[racket-dev] submodules

One more thing, I anticipate that the 'main' module in my "test.rkt"
will be "raco test" and I would extend it to allow you to give a
directory that it will require (if present) all the "test" modules.
You could also have "Test" button in DrRacket.

Jay

On Thu, Mar 8, 2012 at 1:29 PM, Jay McCarthy <jay.mccarthy at gmail.com> wrote:
> I've made a test collecting macro.
>
> https://gist.github.com/2003201
>
> "test.rkt" gives you 'define-test'
>
> (define-test id e ...)
>
> will create a module named 'test' that can see you local bindings
> (like module* #f) at the end of the module that contains all the code
> in "e ...". In addition, you get the (id e ...) form that adds the
> given expressions to the test module.
>
> I expect most uses will look like:
>
> (require racket/test)
> (define-test test (require rackunit))
>
> ....
>
> (define f ...)
> (test ... f tests ...)
>
> ....
>
> (define g ...)
> (test ... g tests ...)
>
> Jay
>
> On Wed, Mar 7, 2012 at 12:07 PM, Jay McCarthy <jay.mccarthy at gmail.com> wrote:
>> I love it---especially for the test collecting macro.
>>
>> I will try to write it and report back.
>>
>> Jay
>>
>> On Wed, Mar 7, 2012 at 10:14 AM, Matthew Flatt <mflatt at cs.utah.edu> wrote:
>>> I've added "submodules" to a version of Racket labeled v5.2.900.1
>>> that's here:
>>>
>>>  https://github.com/mflatt/submodules
>>>
>>> After we've sorted out any controversial parts of the design and after
>>> the documentation is complete, then I'll be ready to merge to the main
>>> Racket repo.
>>>
>>>
>>> Why Submodules?
>>> ---------------
>>>
>>> Using submodules, you can abstract (via macros) over a set of modules
>>> that have distinct dynamic extents and/or bytecode load times. You can
>>> also get a private communication channel (via binding) from a module
>>> to its submodules.
>>>
>>> Some uses:
>>>
>>>  * When you run a module via `racket', if it has a `main' submodule,
>>>   then the `main' module is instantiated --- but not the `main'
>>>   submodules of any other modules used by the starting module.  This
>>>   protocol is implemented for `racket', but not yet for DrRacket.
>>>
>>>  * Languages with separate read-time, configure-time, and run-time
>>>   code can be defined in a single module, with the configure-time and
>>>   read-time code in submodules.
>>>
>>>  * A testing macro could collect test cases and put them into a
>>>   separate `test' submodule', so that testing code is not run or even
>>>   loaded when the module is used normally.
>>>
>>>  * An improved `scribble/srcdoc' can expose documentation through a
>>>   submodule instead of through re-expansion hacks.
>>>
>>>  * If you want to export certain of a module's bindings only to when
>>>   explicitly requested (i.e., not when the module is `require'd
>>>   normally), you can export the bindings from a submodule, instead.
>>>
>>> When I first started talking about these problems last summer, I
>>> called the solution sketch "facets" or "modulets", but the design
>>> has evolved into "submodules".
>>>
>>>
>>> Nesting `module'
>>> ----------------
>>>
>>> Given the term "submodule", the first thing that you're likely to try
>>> will work as expected:
>>>
>>>  #lang racket/base
>>>
>>>  (module zoo racket/base
>>>    (provide tiger)
>>>    (define tiger "Tony"))
>>>
>>>  (require 'zoo)
>>>
>>>  tiger
>>>
>>> Within `module', a module path of the form `(quote id)' refers to the
>>> submodule `id', if any. If there's no such submodule, then `(quote
>>> id)' refers to an interactively declared module, as before.
>>>
>>> Submodules can be nested. To access a submodule from outside the
>>> enclosing module, use the `submod' module path form:
>>>
>>>  #lang racket/base
>>>
>>>  (module zoo racket/base
>>>    (module monkey-house racket/base
>>>      (provide monkey)
>>>      (define monkey "Curious George"))
>>>    (displayln "Ticket, please"))
>>>
>>>  (require (submod 'zoo monkey-house))
>>>
>>>  monkey
>>>
>>> The 'zoo module path above is really a shorthand for `(submod "."
>>> zoo)', where "." means the enclosing module and `zoo' is its
>>> submodule. You could write `(submod "." zoo monkey-house)' in
>>> place of `(submod 'zoo monkey-house)'.
>>>
>>> Note that `zoo' and `monkey-house' are not bound as identifiers in the
>>> module above --- just like `module' doesn't add any top-level
>>> bindings. The namespace of modules remains separate from the namespace
>>> of variables and syntax. Along those lines, submodules are not
>>> explicitly exported, because they are implicitly public.
>>>
>>> When you run the above program, "Ticket, please" is *not* displayed.
>>> Unless a module `require's a submodule, instantiating the module does
>>> not instantiate the submodule. Similarly, instantiating a submodule
>>> does not imply instantiating its enclosing module.
>>>
>>> Furthermore, if you compile the above example to bytecode and run it,
>>> the bytecode for `zoo' is not loaded. Only the bytecode for the
>>> top-level module and `monkey-house' is loaded.
>>>
>>>
>>> Nesting `module*'
>>> -----------------
>>>
>>> Submodules declared with `module' are declared locally while expanding
>>> a module body, which means that the submodules can be `require'd
>>> afterward by the enclosing module. This ordering means, however, that
>>> the submodule cannot `require' the enclosing module. The submodule
>>> also sees no bindings of the enclosing module; it starts with an empty
>>> lexical context.
>>>
>>> The `module*' form is like `module', but it can be used only for
>>> submodules, and it defers the submodule's expansion until after the
>>> enclosing module is otherwise expanded. As a result, a submodule using
>>> `module*' can `require' its enclosing module, while the enclosing
>>> module cannot require the submodule.
>>>
>>> A ".." in a `submod' form goes up the submodule hierarchy, so that
>>> `(submod "." "..")' is a reference to the enclosing module:
>>>
>>>  #lang racket/base
>>>
>>>  (module aquarium racket/base
>>>    (provide fish)
>>>    (define fish '(1 2))
>>>
>>>    (module* book racket/base
>>>      (require (submod "." ".."))
>>>      (append fish '(red blue))))
>>>
>>>  (require (submod 'aquarium book))
>>>
>>> Instead of `require'ing its enclosing module, a `module*' form can use
>>> `#f' as its language, in which case its lexical context starts with
>>> all of the bindings of the enclosing module (implicitly imported)
>>> instead of with an empty lexical context. As a result, the submodule
>>> can access bindings of the enclosing module that are not exported:
>>>
>>>  #lang racket/base
>>>
>>>  (module aquarium racket/base
>>>    (define fish '(1 2))
>>>
>>>    (module* book #f
>>>      (append fish '(red blue))))
>>>
>>>  (require (submod 'aquarium book))
>>>
>>> A common use of `module*' is likely to be with `main', since `racket'
>>> will load a `main' submodule (after `require'ing its enclosing module)
>>> for a module named on its command line. For example, if you run this
>>> program via `racket':
>>>
>>>  #lang racket/base
>>>
>>>  (provide fish)
>>>  (define fish '(1 2))
>>>
>>>  (module* main #f
>>>    (unless (apply < fish)
>>>      (error "fish are not sorted")))
>>>
>>> then you get a "fish are not sorted" error, but if you `require' the
>>> file into another program, you get a `fish' binding with no error.
>>>
>>>
>>> The new `#lang'
>>> ---------------
>>>
>>> The `#lang' reader form was previously defined as a shorthand for
>>> `#reader' where the name after the `#lang' is mangled by adding
>>> "/lang/reader".  With submodules, `#lang' first tries using the name
>>> as-is and checking for a `reader' submodule; if it is found, then the
>>> submodule is used instead of mangling the name with "/lang/reader",
>>> otherwise it falls back to the old behavior.
>>>
>>> So, if you want to define an `ocean' language that is `racket/base'
>>> plus `fish', it's enough to install the following module as "main.rkt"
>>> in an "ocean" collection:
>>>
>>>  #lang racket/base
>>>
>>>  (provide (all-from-out racket/base)
>>>           fish)
>>>  (define fish '(1 2 3))
>>>
>>>  (module reader syntax/module-reader
>>>    #:language 'ocean)
>>>
>>>
>>> Backwards Incompatibility
>>> -------------------------
>>>
>>> The biggest incompatibility is that `resolved-module-path-name' can
>>> return a list when the module path refers to a submodule, in addition
>>> to the old path and symbol results. Most code that calls
>>> `resolved-module-path-name' will have to be updated.
>>>
>>> The `submod' form is a new primitive module-path form, so module name
>>> resolvers also must be updated.  Finally, a load/use-compiled handler
>>> must accept a list as the expected-module name, which usually
>>> indicates that a submodule is being loaded; the list can start with
>>> `#f' to indicate that the module should only be loaded if it can be
>>> loaded independently from bytecode (i.e., without triggering the
>>> declaration of any other submodule, which means not loading from
>>> source). Furthermore, when a submodule is requested, no error should
>>> be raised if the enclosing module is unavailable, which allows
>>> speculative checking for submodule declarations.
>>>
>>> The bytecode format has changed, and the `mod' structure type from
>>> `compiler/zo-parse' has two new fields: one for "pre" submodules
>>> (i.e., those declared with `module') and one for "post" submodules
>>> (i.e., those declared with `module*'). Any code that uses
>>> `compiler/zo-parse' will have to change.
>>>
>>> If you compile a `module' form and it has submodules, then when you
>>> write the bytecode, all of the modules are written together. If the
>>> `module' is not inside a larger top-level sequence, then the printed
>>> form starts with a table that can be used to find any individual
>>> submodule, which is how independent loading of submodules works. If
>>> you just `read' the table in, though, it returns a compiled-module
>>> value that contains submodules, and `eval'ing the compiled module
>>> declares all the submodules, too. This protocol makes lots of
>>> `compile' and `eval' code work without modification. The
>>> `get-module-code' function from `syntax/modcode', meanwhile, gives you
>>> more control, along with functions like module-compiled-submodules' to
>>> get or adjust the submodule list in a compiled-module value.
>>>
>>>
>>> Design Issues
>>> -------------
>>>
>>> The `submod' syntax --- especially "." and ".." --- is arbitrary. The
>>> `submod' name isn't great, but I like it the best among the options
>>> that I tried.  I'm not sure whether the association of "."  and ".."
>>> to filesystem paths is helpfully mnemonic or unhelpfully
>>> confusing. The handling of `quote' paths within a module is also
>>> arbitrary, but it's intended to smooth the connection between the top
>>> level and a module body.
>>>
>>> Overloading `module' for submodules is questionable; again, though, I
>>> like how it roughly matches interactive evaluation. For the
>>> post-submodule form, then, `module*' seems like the obvious
>>> choice.
>>>
>>> As things stand, the ugly pattern `(module* main #f ...)'  would be
>>> common. Probably we should have a macro that expands to `(module* main
>>> #f ...)'. Should the macro be called `main'?
>>>
>>> I haven't tried to build a test-collecting macro or a
>>> `scribble/srcdoc' replacement. I think they will work with this
>>> submodule design, but I can't be sure until we try it.
>>>
>>> _________________________
>>>  Racket Developers list:
>>>  http://lists.racket-lang.org/dev
>>
>>
>>
>> --
>> Jay McCarthy <jay at cs.byu.edu>
>> Assistant Professor / Brigham Young University
>> http://faculty.cs.byu.edu/~jay
>>
>> "The glory of God is Intelligence" - D&C 93
>
>
>
> --
> Jay McCarthy <jay at cs.byu.edu>
> Assistant Professor / Brigham Young University
> http://faculty.cs.byu.edu/~jay
>
> "The glory of God is Intelligence" - D&C 93



-- 
Jay McCarthy <jay at cs.byu.edu>
Assistant Professor / Brigham Young University
http://faculty.cs.byu.edu/~jay

"The glory of God is Intelligence" - D&C 93