[plt-scheme] macrophases

Anton van Straaten wrote:
>> To be able to reuse the function make-prefixed-id in three places in
>> the define-class macro, I resorted to a separate module, imported
>> with require-for-syntax.  This works fine.  Before that, I tried a
>> number of ways to include the function definition within the
>> define-class macro.  But since make-prefixed-id is used at multiple
>> levels - including within a macro generated by the top-level macro -
>> it seems that to make this work, a trick would be needed that I don't
>> know.  Is there such a trick?

Matthew Flatt wrote:
> There's no such trick.
> 
> If we ever get `begin-for-syntax', then you could at least put the
> definition in the same module. But because the function is used by a
> macro-generated macro, the function needs to be defined in some
> module's top level.

Speaking of which: I don't quite understand why PLT Scheme keeps the
transformer top level separate from the normal top level. The MzScheme
manual claims that

    The hierarchy of run times avoids confusion among expansion and
    executing layers that can prevent separate compilation. By ensuring
    that the layers are separate, a compiler or programming environment
    can expand, partially expand, or re-expand a module without
    affecting the module's run-time behavior, whether the module is
    currently executing or not.

I understand the need for a "hierarchy of run times" and separate
storage for transformers. In a way, a program's transformers are
extensions to the compiler rather than part of the program itself.
Therefore, it makes sense to use two different stores for the two
different contexts. However, I don't understand why they use different
scopes.

For example, if you define utility functions in (module foo ...), you
can (require foo) for transformers, normal expressions, or both. In
particular, you can write:

    (module foo
      ; utility functions
      (define (bar) ...)
      ...)
    ; import the functions into transformer and normal namespaces
    (require foo)
    (require-for-syntax foo)
    ; use the functions
    (define-syntax macro-that-calls-bar ...)
    (expression-that-calls-bar ...)

If transformers and normal expressions shared the top-level namespace
(but not storage, of course), then you could write this more simply as

    ; utility functions
    (define (bar) ...)
    ...
    ; use the functions
    (define-syntax macro-that-calls-bar ...)
    (expression-that-calls-bar ...)

The MODULE style would still be handy for cases where you want to
control visibility, e.g., when you use a function for source
transformation only or runtime only.

Here's another way of looking at the issue: On one level, I actually
prefer the MODULE style. I'm a strong advocate of language support for
encoding significant design decisions, and the MODULE style encodes
important details about the differences between "program transformer"
and "normal program" behavior. If that were the whole story, then I'd be
fine with the namespace separation.

However, the MODULE style also has significant costs. The transformers
and the utility functions must reside in different modules. You can put
the modules in the same file, but then they don't play nice with PLT's
compilation and collection tools, which strongly prefer one module per
source file. You can put the modules in different files, but that
complicates source administration and hampers readability. The MODULE
style would be much easier to use with tools that support multiple
modules per file. (Some kind of preprocessor or literate programming
tool might help; it's quite common to generate several intermediate
files from a single literate-programming source file.)

On a slightly different note: I'm confused by the following paragraph in
the MzScheme docs.

    In the absence of let-syntax and letrec-syntax, the hierarchy of run
    times would be limited to three levels, since the transformer
    expressions for run-time imports would have been expanded before the
    importing module must be expanded. The let-syntax and letrec-syntax
    forms, however, allow syntax visible in a for-syntax import's
    transformers to appear in the expansion of transformer expressions
    in the module. Consequently, the hierarchy is bounded in principle
    only by the number of declared modules. In practice, the hierarchy
    will rarely exceed a few levels.

What's special about LET[REC]-SYNTAX that adds more levels to the
compile-run hierarchy?
-- 
Bradd W. Szonye
http://www.szonye.com/bradd