[racket] Understanding local-expand with definition contexts

Here’s an alternative solution as a trampolining* macro. The key is to cooperates with the unit macro’s definition-context handling rather than trying to duplicate that effort. That’s especially important when dealing with the unit macro, since it is infeasible to duplicate its handling of unit imports.

* I call macros like this trampoline-style macros by analogy with a technique for implementing proper tail recursion on platforms that don’t directly support tail calls. Rather than the macro controlling the expansion of the unit body from start to finish, it inserts macro calls in the syntax it returns to its context and relinquishes control, and it regains control when its context decides to continue macro expansion.

Here is the code:

(require racket/unit
         (for-syntax racket/base syntax/parse))

(define-syntax (unit/capture-lifts stx)
  (syntax-parse stx
    #:literals (import export)
    [(unit/capture-lifts (import i ...) (export e ...) body ...)
     #'(unit (import i ...) (export e ...)
         (capture-lifts body) ...)]))

(define-syntax (capture-lifts stx)
  (syntax-parse stx
    [(capture-lifts form)
     (case (syntax-local-context)
       [(expression)
        ;; Need the let in case lifts are caught as definitions
        #`(let () #,(local-expand/capture-lifts #'form 'expression null))]
       [else
        (define e-form
          (local-expand/capture-lifts #'form (syntax-local-context) #f))
        (syntax-parse e-form
          #:literal-sets (kernel-literals)
          ;; NOTE: contrary to docs, result may not be a begin form
          ;; if there were no lifts. (FIXME: investigate)
          [(begin form ...)
           #'(begin (capture-lifts form) ... )]
          [e-form
           #'(capture-lifts-in-exprs e-form)])])]))

;; Capture lifts in the expansion of any expr in the given form.
;; PRE: form is head-expanded and not a begin form
(define-syntax (capture-lifts-in-exprs stx)
  (syntax-parse stx
    [(capture-lifts/def form)
     (syntax-parse #'form
       #:literal-sets (kernel-literals)
       ;; Case 1: No phase-0 subexpressions; stop.
       [(define-syntaxes . _) #'form]
       [(module . _) #'form]
       [(begin-for-syntax . _) #'form]
       [(#%provide . _) #'form]
       [(#%require . _) #'form]
       ;; Case 2: Recur to definition's subexpression.
       [(define-values vars rhs)
        #'(define-values vars (capture-lifts rhs))]
       ;; Case 3: Expression: force syntax-local-context to 'expression,
       ;; then capture-lifts.
       [expr
        #'(#%expression (capture-lifts expr))])]))

;; ----------------------------------------
;; Test without import

(define-signature s^ (a b))

(define u
  (unit/capture-lifts (import) (export s^)
    (define-syntax (lifted stx)
      (syntax-parse stx
        [(_ e)
         (syntax-local-lift-expression #'e)]))
    (define a (for/list ([i 2]) (lifted (gensym))))
    (define b (gensym))
    (printf "~s ~s\n" a b)))

;; Check that lifted gensym is within unit, but outside loop:
(invoke-unit u)
(invoke-unit u)

;; ----------------------------------------
;; Test with import

(define-signature t^ (c d))

(define u2
  (unit/capture-lifts (import t^) (export s^)
    (define-syntax (lifted stx)
      (syntax-parse stx
        [(_ e)
         (syntax-local-lift-expression #'e)]))
    (define a (for/list ([i 2]) (lifted (gensym))))
    (define b (gensym))
    (printf "~s ~s ~s ~s\n" a b c d)))

(define c 'used)
(define d 'imports)

;; Check that lifted gensym is within unit, but outside loop:
(invoke-unit u2 (import t^))
(invoke-unit u2 (import t^))

Ryan


On Jan 5, 2015, at 2:47 PM, Matthew Flatt <mflatt at cs.utah.edu> wrote:

> That turns out to be tricky. I've enclosed an implementation, but it
> works only for units that have no imports.
> 
> At Mon, 05 Jan 2015 17:21:37 +0000, Spencer Florence wrote:
>> What I'm trying to do is capture all lifts inside a unit body, instead of
>> having them propagating to the top level.
>> 
>> If I understand what you mean by "add internal-definition context to the
>> bindings listed in `exports`", I can't do that because I need to capture
>> any lifts inside the unit body, but a `(local-expand/capture-lifts #'(unit
>> ...))` would put them outside. But maybe I miss understand you.
>> On Mon Jan 05 2015 at 12:10:53 PM Matthew Flatt <mflatt at cs.utah.edu> wrote:
>> 
>>> I'm not really clear on what you're trying to do. One way to explain
>>> more might be to explain how `test` is meant to differ from `begin`. Or
>>> maybe you can say why it doesn't work to add the internal-definition
>>> context to the bindings listed in `exports` (i.e., to bring everything
>>> into the definition context).
>>> 
>>> At Mon, 05 Jan 2015 16:44:10 +0000, Spencer Florence wrote:
>>>> Is there any way to have the `test` macro work more like `begin`?
>>>> 
>>>> What I am trying to accomplish is something like:
>>>> 
>>>> (define-syntax (make-my-unit stx)
>>>>   (syntax-parse stx
>>>>      [(e ...)
>>>>       (with-syntax ([body (local-expand/capture-lifts stx <something>)])
>>>>         #'(unit (imports ...) (exports ...) body))]))
>>>> 
>>>> Where the defines in #'(e ...) are visible to `unit` so that they can be
>>>> used for exports.
>>>> 
>>>> --spf
>>>> On Mon Jan 05 2015 at 11:32:46 AM Matthew Flatt <mflatt at cs.utah.edu>
>>> wrote:
>>>> 
>>>>> I think it's more a question of what a definition context is supposed
>>>>> to be, rather than how `syntax-local-bind-syntaxes` works.
>>>>> 
>>>>> When you create a new definition context, the context's bindings are
>>>>> visible only to expressions that are also in that context. The `test`
>>>>> form here creates a new definition context in much the same way as
>>>>> `(let () ...)`:
>>>>> 
>>>>> (let ()
>>>>>   (let () (define x 1))
>>>>>   x)
>>>>> 
>>>>> In other words, the final `x` really is out of the scope of the
>>>>> definition of `x`.
>>>>> 
>>>>> At Mon, 05 Jan 2015 16:26:30 +0000, Spencer Florence wrote:
>>>>>> Progress is a new error! I don't think I understand how
>>>>>> syntax-local-bind-syntaxes is supposed to work. I extended the
>>> previous
>>>>>> program:
>>>>>> 
>>>>>> #lang racket
>>>>>> (require (for-syntax syntax/parse))
>>>>>> (define-syntax (test stx)
>>>>>>  (syntax-parse stx
>>>>>>    [(_ e)
>>>>>>     (define ctx
>>>>>>       (if (list? (syntax-local-context))
>>>>>>           (cons (gensym) (syntax-local-context))
>>>>>>           (list (gensym))))
>>>>>>     (define def-ctx (syntax-local-make-definition-context))
>>>>>>     (define expd (local-expand #'e ctx (list #'define-values)
>>> def-ctx))
>>>>>>     (define ids (syntax-parse expd [(def (id) _) (list #'id)]))
>>>>>>     (syntax-local-bind-syntaxes ids #f def-ctx)
>>>>>>     (internal-definition-context-seal def-ctx)
>>>>>>     expd]))
>>>>>> (let ()
>>>>>>  (test (define x 1))
>>>>>>  x)
>>>>>> 
>>>>>> And now I receive the error:  `x: unbound identifier in module in: x`
>>>>>> Looking at the docs for `syntax-local-make-definition-context` it
>>> seems
>>>>>> like I need to provide it with the parent definition-context, but
>>> I'm not
>>>>>> sure how to get a hold of that.
>>>>>> 
>>>>>> --spf
>>>>>> 
>>>>>> On Mon Jan 05 2015 at 10:00:53 AM Matthew Flatt <mflatt at cs.utah.edu>
>>>>> wrote:
>>>>>> 
>>>>>>> The error message is intended for "end users" and turns out to be
>>>>>>> misleading for the implementor of an internal-definition context.
>>> The
>>>>>>> documentation for `define-values` has essentially the same
>>> problem: it
>>>>>>> describes how `define-values` should work in an internal-definition
>>>>>>> context, but it doesn't say how the form interacts with
>>> `local-expand`.
>>>>>>> 
>>>>>>> A `define-values` form will only expand in a module or top-level
>>>>>>> context. To implement an internal-definition context, you must
>>> expand
>>>>>>> only far enough to see `define-values` form; in other words, supply
>>>>>>> `#'define-values` in the stop list. Then, a partially expanded
>>>>>>> `define-values` form must be recognized and handled explicitly,
>>> with
>>>>>>> tools like `syntax-local-bind-syntaxes` or re-writing to
>>>>>>> `letrec-values`, as appropriate for the definition context.
>>>>>>> 
>>>>>>> At Mon, 05 Jan 2015 14:49:03 +0000, Spencer Florence wrote:
>>>>>>>> Hey all,
>>>>>>>> 
>>>>>>>> I'm trying to use 'local-expand', however it seems to think its
>>>>> never in
>>>>>>> a
>>>>>>>> definition context. For example:
>>>>>>>> 
>>>>>>>> (require (for-syntax syntax/parse))
>>>>>>>> (define-syntax (test stx)
>>>>>>>>  (syntax-parse stx
>>>>>>>>    [(_ e)
>>>>>>>>     (define ctx
>>>>>>>>       (if (list? (syntax-local-context))
>>>>>>>>           (cons (gensym) (syntax-local-context))
>>>>>>>>           (list (gensym))))
>>>>>>>>     (local-expand
>>>>>>>>      #'e ctx null
>>>>>>>>      ;; result is the same with this uncommented
>>>>>>>>      #;(syntax-local-make-definition-context))]))
>>>>>>>> (let ()
>>>>>>>>  (test (define x 1))
>>>>>>>>  x)
>>>>>>>> 
>>>>>>>> errors with a "define-values: not in a definition context in:
>>>>>>>> (define-values (x) 1)"
>>>>>>>> 
>>>>>>>> Can anyone provide any insight into what is going on?
>>>>>>>> 
>>>>>>>> --spf
>>>>>>>> ____________________
>>>>>>>>  Racket Users list:
>>>>>>>>  http://lists.racket-lang.org/users
>>>>>>> 
>>>>> 
> <unit-capture.rkt>____________________
>  Racket Users list:
>  http://lists.racket-lang.org/users