[plt-scheme] "identifier used out of context" after two local-expands

--- Kimberley Burchett <kim.burchett at gmail.com> wrote:

> I'm trying to write a macro that takes a single subform and does
> this:
>   1) expands all the macros in that subform except for one
> particular macro
>   2) recursively walks the expanded result
>   3) when the walk encounters the unexpanded macro, expands the
> unexpanded subforms.
> 
> I'm using local-expand for this, because it has a stop-list.
> Steps 1 and 2 seem to work just fine, but I have a problem in
> step #3. 
> When I try to expand the subforms via a second call to 
> local-expand, I get "identifier used out of context" errors.
> This only happens for identifiers that were lexically bound in
> the original subform. 
> I've whittled the problem down to the following small example,
> where the macro I don't want to be expanded is named "stop".
> 
> (define-syntax (phase-one stx)
>   (syntax-case stx ()
>     ((_ e)
>      (let* ((stop-id (datum->syntax-object stx 'stop))
>             (partially-expanded (local-expand #'e
>                                              
> (syntax-local-context)
>                                               (list stop-id))))
>        (phase-two partially-expanded)))))
> 
> (define-for-syntax (phase-two stx)
>   (syntax-case stx ()
>     ((lambda formals e)
>      #`(lambda formals #,(phase-two #'e)))
>     ((i x1 x2 x3)
>      (local-expand #'(i x1 x2 x3) (syntax-local-context) ()))))
> 
> (phase-one (lambda (x) (stop x x x)))
> 
> >>> compile: identifier used out of context in: x
> 
> 
> Can anyone explain to me what I'm doing wrong?

I believe this is what is happening:

Macro expansion uses an environment to keep track of variable
bindings and macro definitions. That is the "context" that the error
message refers to.

The syntactic environment starts out with just the normal Scheme
bindings. Your phase-one macro calls local-expand on its argument.
When local-expand hits the lambda form, it adds an "x" binding to the
syntactic environment, and it adds annotations to the body that say
"expect an 'x' binding". So the "x" identifiers in the
partially-expanded syntax expect to be expanded in an environment
with an "x" binding.

But when local-expand returns, the "current" syntactic environment
revert to the initial environment---no lexical "x" binding.

When phase-two function hits a lambda form, it recurs on the body,
*without extending the syntactic environment.* So when it finally
hits a macro use, it calls local-expand in a syntactic environment
without a binding for "x", and the expander raises an error.

I'll follow up with suggestions on how to rewrite the code to avoid
this problem.

Ryan


PS - Here is another example that illustrates this same problem more
directly:

  (define s1 (expand #'(lambda (x) (add1 x))))
  (define s2 (syntax-case s1 (lambda)
               [(lambda formals body) #'body]))
  (expand s2)