[racket] Best Way to Really Understand the Compilation and Execution Model?

Thanks for the thorough reply and examples!

I'll definitely look into this, but suspect this approach won't be
friendly to redefinition.  I'll try and see.

Thanks,
Nick

On Mon, Aug 5, 2013 at 4:52 AM, Jens Axel Søgaard <jensaxel at soegaard.net> wrote:
> Hi Nick,
>
> 2013/8/4 Nick Sivo <nicksivo at gmail.com>
>>
>> I'd like to make Arc work as a Racket language (and in the process
>> preserve srclocs and names), and have made some progress[4].  However,
>> I'm hitting a wall trying to implement Arc's macros.  Specifically, I
>> need to find a way for the macros themselves to use previously defined
>> Arc functions as part of their processing. I understand that there are
>> good reasons for Racket's hygiene and syntax phases, but I'm
>> implementing an existing language that has its own strong opinions.
>> I'm almost certain that accomplishing my goal is possible with the
>> right understanding, but despite reading much of the Racket Reference,
>> I don't truly get it yet.
>
>
> The evaluation model of Arc makes it hard to device a general solution. The
> problem is that arc-identifiers are represented
> as symbols which do not contain any source location information.
>
> For a macro like this:
>
>     (mac when (test . body)
>        (list 'if test (cons 'do body)))
>
> which defines a when macro that expands
>
>    (when test body ...) to (if test (do body ...))
>
> you know the source location of the identifier when.
>
>
> However if you have a arc-macro that expands to into a new
>
> arc-macro definition, the arc programmer has no way of
>
> attaching source location information to the generated
>
> arc-identifer.
>
>
> That said an 80% solution is better than nothing.
>
>
>>
>> With that motivation in mind, what's the best way for me to become
>> intimately familiar with Racket's runtime, execution and compilation
>> model? Should I read through and make sure I understand the debugger?
>> The macro stepper?  Is there any kind of guide to Racket Internals?
>
>
> I would read up on modules (and submodules) and experiment
> with define-syntax and begin-for-syntax. Skip the debugger.
>
> Given the way Arc mixes the phases it is hard to compile Arc to anything.
> Consider for example how you would compile Arc-with-macros
> into Arc-without-macros.
>
> If I understand the evaluation model correctly, then you basically can't
> compile an Arc program without running it. If you assume that certain
> names aren't redefined (such as if, mac and others) then it might
> be possible to compile parts.
>
>> Specifically, I need to find a way for the macros themselves to use
>> previously defined Arc functions as part of their processing.
>
> Here is a few examples that illustrate how to make a function
> available both in runtime (phase 0) and at compile time (here in phase 1).
>
> The idea of the final example is to compile (read arc-definitions)
> definitions
> into submodule, which are then required in both phases. This implies
> that the code is run twice...
>
> /Jens Axel Søgaard
>
>
> #lang racket
>
> ; Let us define a simple macro myand and use it as
> ; a running example.
>
> (module ex1 racket
>   (define-syntax (myand stx)
>     (syntax-case stx ()
>       [(_)                 #'#t]
>       [(_ expr0 expr ...)  #'(if expr0 (myand expr ...) #f)]))
>
>   ; Test it:
>   (displayln "Example 1")
>   (myand (= 1 1) (= 2 3))  ; => #f
>   (myand (= 1 1) (= 2 2))  ; => #t
>   ; Conclusion: It works on runtime.
>   )
>
> ; Now can we use in a macro (at expansion time?)
>
> #;(module ex2 racket
>     (define-syntax (myand stx)
>       (syntax-case stx ()
>         [(_)                 #'#t]
>         [(_ expr0 expr ...)  #'(if expr0 (myand expr ...) #f)]))
>
>     (define-syntax (macro-that-uses-myand stx)
>       (syntax-case stx ()
>         [(_ x y z)
>          (if (myand (= 1 1))      ; <= this myand gives the error below
>              #''one-equals-one
>              #''huh?)])))
>
> ; Nope. The example ex2 gives the error:
> ;     myand: unbound identifier in module (in the transformer environment,
> ;     which does not include the macro definition that is visible to
> run-time
> ;     expressions) in: myand
>
> ; In order to make it visible, we need to define it both in the runtime
> ; and transformer environment.
>
> (module ex3 racket
>   (define-syntax (myand stx)
>       (syntax-case stx ()
>         [(_)                 #'#t]
>         [(_ expr0 expr ...)  #'(if expr0 (myand expr ...) #f)]))
>   (begin-for-syntax
>     (require (for-syntax racket))
>     (define-syntax (myand stx)
>       (syntax-case stx ()
>         [(_)                 #'#t]
>         [(_ expr0 expr ...)  #'(if expr0 (myand expr ...) #f)])))
>   (define-syntax (macro-that-uses-myand stx)
>     (syntax-case stx ()
>       [(_ x y z)
>        (if (myand (= 1 1))
>            #''one-equals-one
>            #''huh?)]))
>   (displayln "Example 3")
>   (macro-that-uses-myand 1 2 3)
>   (myand (= 1 1) (= 2 3)))
>
> ; Writing the same macro twice is a bad idea, so we'll let
> ; a macro do the job.
>
> (module define*-mod racket
>   (provide define*)
>
>   (define-syntax (define* stx)
>     (syntax-case stx ()
>       [(_ macro-definition)
>        #'(begin
>            (begin-for-syntax macro-definition)
>            macro-definition)])))
>
> ; Let's test the macro:
>
> (module ex4 racket
>   (require (submod ".." define*-mod)
>            (for-syntax (for-syntax racket))) ; make syntax-case available in
> phase 2
>   (define*
>     (define-syntax (myand stx)
>       (syntax-case stx ()
>         [(_)                 #'#t]
>         [(_ expr0 expr ...)  #'(if expr0 (myand expr ...) #f)])))
>   (define-syntax (macro-that-uses-myand stx)
>     (syntax-case stx ()
>       [(_ x y z)
>        (if (myand (= 1 1))
>            #''one-equals-one
>            #''huh?)]))
>   (displayln "Example 4")
>   (macro-that-uses-myand 1 2 3)
>   (myand (= 1 1) (= 2 3)))
>
> ; This works, but in general code duplication is a Bad Thing (TM).
> ; If we place the definition of myand in a module, we can import it
> ; in both phases.
>
> (module ex5 racket
>   ; A submodule containing a definition of myand:
>   (module myand-mod racket
>     (provide myand)
>     (define-syntax (myand stx)
>       (syntax-case stx ()
>         [(_)                 #'#t]
>         [(_ expr0 expr ...)  #'(if expr0 (myand expr ...) #f)])))
>
>   ; Import it in both phases:
>   (require             (submod "." myand-mod)
>            (for-syntax (submod "." myand-mod)))
>   ; Test it:
>   (define-syntax (macro-that-uses-myand stx)
>     (syntax-case stx ()
>       [(_ x y z)
>        (if (myand (= 1 1))
>            #''one-equals-one
>            #''huh?)]))
>   (displayln "Example 5")
>   (macro-that-uses-myand 1 2 3)
>   (myand (= 1 1) (= 2 3)))
>
> ; Now let's write a macro that can generate the submodule automatically.
>
> (module define/mod-mod racket
>   (provide define/mod)
>   (define-syntax (define/mod stx)
>     (syntax-case stx ()
>       [(_ name definition)
>        #'(begin
>            (module name racket
>              (provide name)
>              definition)
>            #;(require (submod "." name) (for-syntax (submod "." name)))   ;
> footnote *
>            )])))
>
> (module ex6 racket
>   (require (submod ".." define/mod-mod))
>   (define/mod myand
>     (define-syntax (myand stx)
>       (syntax-case stx ()
>         [(_)                 #'#t]
>         [(_ expr0 expr ...)  #'(if expr0 (myand expr ...) #f)])))
>   (require (submod "." myand) (for-syntax (submod "." myand)))           ;
> footnote **
>   ; Test it:
>   (define-syntax (macro-that-uses-myand stx)
>     (syntax-case stx ()
>       [(_ x y z)
>        (if (myand (= 1 1))
>            #''one-equals-one
>            #''huh?)]))
>   (displayln "Example 6")
>   (macro-that-uses-myand 1 2 3)
>   (myand (= 1 1) (= 2 3)))
>
> ; Ideally the line marked ** could be commented out and the one marked * be
> used instead.
> ; If I remember correctly I need to do *something* with the require, but
> what ...
>
> (require 'ex1)
> (require 'ex3)
> (require 'ex4)
> (require 'ex5)
> (require 'ex6)
>
>
>
>
>