[racket-dev] Strange issue with identifier-binding being wrong for lexical variables

On Wed, Oct 22, 2014 at 10:20 AM, Matthew Flatt <mflatt at cs.utah.edu> wrote:
> I agree that this is broken, but I'd like to put it on hold, because
> its another basic problem with the way the current macro expander
> represents lexical context.
>
> Adjusting the context of the expressions changes the result, because
> its the macro-introduced nature of the `main` definition that triggers
> the bad result from `identifier-binding`.
>
> Expansions that produce this bad `identifier-binding` result probably
> happen up all the time. They don't bother the bytecode compiler,
> because the compiler uses `free-identifier=?` to compare bindings in
> expanded code, keeping track of all of the bindings that are in the
> environment of a given expression. Depending on your use case, that
> might be the way to go for now.

If I understand correctly, I should maintain an environment of
lexically-bound identifiers at every point, and if a given identifier
is in that environment, it should be treated as lexical, even if
`identifier-binding` says that it's a module-bound variable. Only if
an identifier isn't currently bound should I treat it as a
module-bound variable.

Does that sound right? This would be easy enough for me to do.

Sam

>
> At Tue, 21 Oct 2014 22:26:26 -0400, Sam Tobin-Hochstadt wrote:
>> I've found what I think is a bug in the expander where lexical
>> references can get an `identifier-binding` result that suggests that
>> they're module-bound.
>>
>> In particular, you need these three files:
>>
>> bugtest.rkt:
>>
>> (module bugtest "wraptest.rkt")
>>
>> bugtest.scm:
>>
>> (define (gcbench)
>>   (define main #f)
>>   main)
>>
>> (define main #f)
>>
>> wraptest.rkt:
>>
>> #lang racket/base
>> (provide (rename-out (module-begin #%module-begin)))
>>
>> (require racket/include
>>          (for-syntax racket/base))
>>
>> (define-syntax (module-begin stx)
>>   (define name (syntax-property stx 'enclosing-module-name))
>>   #`(#%module-begin
>>      (include #,(format "~a.scm" name))))
>>
>> Then run the macro stepper with macro hiding off on "bugtest.rkt".
>> Click on the reference to `main` inside `gcbench`. You'll see that it
>> says that it's a module-bound variable named `main.2` which is defined
>> in this module.
>>
>> Then try changing the name of the top-level definition in bugtest.scm
>> to `main2`. Re-run the macro stepper, and you'll see that the
>> identifier-binding is now lexical.
>>
>> I tried a few things to get this to go away (such as using
>> `#%plain-module-begin`) which didn't work. Reducing it to two files,
>> ie doing the include directly in bugtest.rkt, made the problem
>> disappear.
>>
>> Changing the body of the `module-begin` macro to:
>>
>> (define-syntax (module-begin stx)
>>   (define name (syntax-property stx 'enclosing-module-name))
>>   #`(#%module-begin
>>      #,(datum->syntax stx `(include ,(format "~a.scm" name)))
>>      #,(datum->syntax stx '(main))))
>>
>> and then providing a bunch of extra stuff made the issue go away.
>> Because there's the workaround, the issue isn't urgent.
>>
>> Sam (and Tobias, who found this bug)
>