[racket] Flatten syntax and runtime phases?

Thanks for the leads!  I'll look into your suggestions and follow up
with my results.

-Nick

On Fri, Jun 15, 2012 at 10:54 PM, Ryan Culpepper <ryan at cs.utah.edu> wrote:
> On 06/15/2012 10:41 PM, Nick Sivo wrote:
>>
>> Hi,
>>
>> I have the following code:
>>
>> #lang racket
>> (require mzlib/defmacro)
>>
>> (defmacro w/uniq (name . body)
>>   `(let ([,name (gensym)])
>>      , at body))
>>
>> (defmacro in (x . choices)
>>   (w/uniq g
>>      `(let ([,g ,x])
>>         (or ,@(map (lambda (c) `(eqv? ,g ,c)) choices)))))
>>
>> Naturally, it fails:
>>
>> expand: unbound identifier in module (in phase 1, transformer environment)
>> in: g
>>
>> Additionally defining w/uniq for-syntax fixes it:
>>
>> #lang racket
>> (require mzlib/defmacro
>>          (for-syntax racket
>>                      mzlib/defmacro))
>>
>> (begin-for-syntax
>>   (defmacro w/uniq (name . body)
>>     `(let ([,name (gensym)])
>>        , at body)))
>>
>> (defmacro w/uniq (name . body)
>>     `(let ([,name (gensym)])
>>        , at body))
>>
>> (defmacro in (x . choices)
>>   (w/uniq g
>>      `(let ([,g ,x])
>>         (or ,@(map (lambda (c) `(eqv? ,g ,c)) choices)))))
>>
>> I know this is Racket's phase separation working as intended.  Now, I
>> appreciate why Racket separates phases as well as the advantages of
>> hygienic macros, but I'm not writing new code.  I'm trying to enable
>> existing code* to run natively in Racket and take advantage of its
>> great JIT and some module optimizations (like inlining).
>>
>> Is there any way to hack the first version to work? One option is to
>> build a deeper nest of duplicate macro definitions each time I
>> encounter a new one.  It seems possible, and it might even work, but
>> it sounds awful.  I've already written a custom reader and have an
>> extensive library of language syntax/parse macros, so I'm open to
>> anything.
>
>
> I don't know of a way of doing this within Racket other than running each
> definition at every phase that it might be used at. That might not require
> duplicating code, though.
>
> If you could rely on let*-like scoping for Arc module bodies, then you could
> do it entirely with submodules (new feature since 5.2.1; see
> http://blog.racket-lang.org/2012/06/submodules.html for an introduction).
>
> The idea would be to use the arc language's #%module-begin macro to
> transform
>
> (module M arc
>  defn1
>  defn2
>  defn3))
>
> into
>
> (module M arc
>  (#%plain-module-begin
>
>  (module S0 arc)
>
>  (module S1 arc
>    (require (submod ".." S0)
>             (for-syntax (submod ".." S0)))
>    defn1
>    (provide (all-from-out (submod ".." S0))
>             (all-defined-out)))
>
>  (module S2 arc
>    (require (submod ".." S1)
>             (for-syntax (submod ".." S1)))
>    defn2
>    (provide (all-from-out (submod ".." S1))
>             (all-defined-out)))
>
>  (module S3 arc
>    (require (submod ".." S2)
>             (for-syntax (submod ".." S2)))
>    defn3
>    (provide (all-from-out (submod ".." S2))
>             (all-defined-out)))
>
>  (require (submod "." S3))
>  (provide (all-from-out (submod "." S3)))))
>
> Each S<N> submodule provides (at phase 0 only) the names defined by all
> definitions defn1 through defn<N>. (For S0, that means zero definitions.)
>
> Each S<N+1> submodule requires S<N> at phases 0 and 1, thus it gets all
> definitions defn1 though defn<N> for both run time and compile-time.
>
> That's if you have strict let*-like scoping. If you want to support mutually
> recursive modules, you could try to group contiguous run-time definitions
> together by head-expanding (ie, calling 'local-expand' with a stop list) and
> accumulating forms as long as they produce 'define-values' forms. I don't
> see any way to support mutually recursive functions that have macro
> definitions between them, though.
>
> I also don't see any way to support references from a macro body to
> functions defined later in the module.
>
> This will break 'set!', because variables imported from another module
> cannot be mutated. But you can fix that by defining auxiliary setter
> procedures and providing proxy set-transformer macros in place of the
> variables.
>
> Mutating a variable (or data structure) in one phase still cannot change the
> value in other phases, no matter what you do. In fact, compile-time mutation
> won't work very well at all, since each definition is compiled in a new
> submodule, which should get its own fresh compile-time state.
>
> If you have 'let-syntax' or its equivalent (eg, local 'define-syntax'), then
> phases 0 and 1 might not be enough, since you can have expressions at phase
> 2 (and higher):
>
>  (let-syntax ([m0
>                (let-syntax ([m1 <phase-2-expr>]) <phase-1-expr>)])
>    <phase-0-expr>)
>
> If you want to be really *really* crafty, you could try defining your own
> version of 'let-syntax', 'define-syntax', etc to create requires for higher
> phases on demand by calling 'syntax-local-lift-require'. That is,
>
>  (arc-define-syntax id expr)
>
>  == expands to the result of calling ==>
>
>  (syntax-local-lift-require
>   #`(for-meta #,(add1 (syntax-local-phase-level)) S<N>)
>   #'(racket-define-syntax id expr))
>
> I hope that helps. (I haven't tried any of this, so there are surely bugs
> that need ironing out.)
>
> Ryan
>