[racket] Flatten syntax and runtime phases?

On 06/15/2012 10:41 PM, Nick Sivo wrote:
> Hi,
>
> I have the following code:
>
> #lang racket
> (require mzlib/defmacro)
>
> (defmacro w/uniq (name . body)
>    `(let ([,name (gensym)])
>       , at body))
>
> (defmacro in (x . choices)
>    (w/uniq g
>       `(let ([,g ,x])
>          (or ,@(map (lambda (c) `(eqv? ,g ,c)) choices)))))
>
> Naturally, it fails:
>
> expand: unbound identifier in module (in phase 1, transformer environment) in: g
>
> Additionally defining w/uniq for-syntax fixes it:
>
> #lang racket
> (require mzlib/defmacro
>           (for-syntax racket
>                       mzlib/defmacro))
>
> (begin-for-syntax
>    (defmacro w/uniq (name . body)
>      `(let ([,name (gensym)])
>         , at body)))
>
> (defmacro w/uniq (name . body)
>      `(let ([,name (gensym)])
>         , at body))
>
> (defmacro in (x . choices)
>    (w/uniq g
>       `(let ([,g ,x])
>          (or ,@(map (lambda (c) `(eqv? ,g ,c)) choices)))))
>
> I know this is Racket's phase separation working as intended.  Now, I
> appreciate why Racket separates phases as well as the advantages of
> hygienic macros, but I'm not writing new code.  I'm trying to enable
> existing code* to run natively in Racket and take advantage of its
> great JIT and some module optimizations (like inlining).
>
> Is there any way to hack the first version to work? One option is to
> build a deeper nest of duplicate macro definitions each time I
> encounter a new one.  It seems possible, and it might even work, but
> it sounds awful.  I've already written a custom reader and have an
> extensive library of language syntax/parse macros, so I'm open to
> anything.

I don't know of a way of doing this within Racket other than running 
each definition at every phase that it might be used at. That might not 
require duplicating code, though.

If you could rely on let*-like scoping for Arc module bodies, then you 
could do it entirely with submodules (new feature since 5.2.1; see 
http://blog.racket-lang.org/2012/06/submodules.html for an introduction).

The idea would be to use the arc language's #%module-begin macro to 
transform

(module M arc
   defn1
   defn2
   defn3))

into

(module M arc
  (#%plain-module-begin

   (module S0 arc)

   (module S1 arc
     (require (submod ".." S0)
              (for-syntax (submod ".." S0)))
     defn1
     (provide (all-from-out (submod ".." S0))
              (all-defined-out)))

   (module S2 arc
     (require (submod ".." S1)
              (for-syntax (submod ".." S1)))
     defn2
     (provide (all-from-out (submod ".." S1))
              (all-defined-out)))

   (module S3 arc
     (require (submod ".." S2)
              (for-syntax (submod ".." S2)))
     defn3
     (provide (all-from-out (submod ".." S2))
              (all-defined-out)))

   (require (submod "." S3))
   (provide (all-from-out (submod "." S3)))))

Each S<N> submodule provides (at phase 0 only) the names defined by all 
definitions defn1 through defn<N>. (For S0, that means zero definitions.)

Each S<N+1> submodule requires S<N> at phases 0 and 1, thus it gets all 
definitions defn1 though defn<N> for both run time and compile-time.

That's if you have strict let*-like scoping. If you want to support 
mutually recursive modules, you could try to group contiguous run-time 
definitions together by head-expanding (ie, calling 'local-expand' with 
a stop list) and accumulating forms as long as they produce 
'define-values' forms. I don't see any way to support mutually recursive 
functions that have macro definitions between them, though.

I also don't see any way to support references from a macro body to 
functions defined later in the module.

This will break 'set!', because variables imported from another module 
cannot be mutated. But you can fix that by defining auxiliary setter 
procedures and providing proxy set-transformer macros in place of the 
variables.

Mutating a variable (or data structure) in one phase still cannot change 
the value in other phases, no matter what you do. In fact, compile-time 
mutation won't work very well at all, since each definition is compiled 
in a new submodule, which should get its own fresh compile-time state.

If you have 'let-syntax' or its equivalent (eg, local 'define-syntax'), 
then phases 0 and 1 might not be enough, since you can have expressions 
at phase 2 (and higher):

   (let-syntax ([m0
                 (let-syntax ([m1 <phase-2-expr>]) <phase-1-expr>)])
     <phase-0-expr>)

If you want to be really *really* crafty, you could try defining your 
own version of 'let-syntax', 'define-syntax', etc to create requires for 
higher phases on demand by calling 'syntax-local-lift-require'. That is,

   (arc-define-syntax id expr)

   == expands to the result of calling ==>

   (syntax-local-lift-require
    #`(for-meta #,(add1 (syntax-local-phase-level)) S<N>)
    #'(racket-define-syntax id expr))

I hope that helps. (I haven't tried any of this, so there are surely 
bugs that need ironing out.)

Ryan