[racket] example where untagging behavior of local-expand is important?

On Nov 12, 2013, at 9:32 AM, Ryan Culpepper wrote:

> On 11/12/2013 01:51 AM, John Clements wrote:
>> I was looking over changes to Rust recently, and I saw with some alarm that at least one macro had added an ad-hoc local expansion mechanism.  I sat down with MTWT to see if I could figure out a situation where this might cause a problem. I came up with something, and I'm hoping that one of you can tell me whether it makes sense, and whether there are other situations where this comes up.
>> 
>> For what it's worth, the macro in question is one that tries to glue together strings at expansion time; something like this:
>> 
>> (concat-strings (file-name) ":" (line-number) " there problem is that ~a is not equal to ~a")
>> 
>> ... where (file-name) and (line-number) are macros, and the concat-strings macro wants to join them all together into a format string that can be analyzed. In order to perform this analysis, the two sub-macros must be expanded into strings. It appears to me that syntax-case + compile-time bindings solves this, but Rust has neither of these.
> 
> BTW, this example has an unrelated problem: it reinterprets a file name as (part of) a format string without any quoting/escaping. If the file name contains format directives, bad things may happen. I don't know if that's an issue with the actual macro in question, but beware.
> 
>> Looking over MTWT, it appears to me that the principal complexity of local-expand arises from
>> - stop-lists, and
>> - un- /re-marking
>> 
>> Looking at Rust, it appears to me that stop-lists are not an issue, at least for this particular macro. It may be that the marking behavior is also a non-problem, but I thought I'd try to come up with an example of macros where it *would* make a difference. The best I got was this:
>> 
>> #lang racket
>> 
>> ;; this macro just does a local-expand
>> (define-syntax (does-local-expand stx)
>>   (syntax-case stx ()
>>     [(_ exp) (local-expand #'exp 'expression #f)]))
>> 
>> (define z 13)
>> 
>> (does-local-expand
>>  (let ()
>>    (define-syntax m
>>      (syntax-rules ()
>>        [(m) z]))
>>    (m)))
>> 
>> So, in Racket, if I understand correctly, the mark from the expansion of 'does-local-expand' is removed (or cancelled) before the syntax-local-expand, and restored again afterward. The let, on the other hand, generates an instance of the identifier 'z' that was introduced by the macro 'm', and therefore has 'm's mark on it.
>> 
>> Looking only at the 'z' in the body of the let, then: first it gets a mark m1 upon entry to the expansion of does-local-expand. Then, when the body of does-local-expand calls local-expand, 'z' gets another m1 mark. Then, in the expansion of the use of 'm', it gets the mark (call it m2) on the way *out* of m only (because it wasn't marked on the way into m). Then, the local-expand is finished, and it gets another m1. Finally, expansion of the use of does-local-expand is finished, and it gets another m1. So, the end list of marks is
>> 
>> m1 m1 m2 m1 m1
>> 
>> which is equivalent to just
>> 
>> m2
>> 
>> Now, suppose you *didn't* have the unmarking/re-marking behavior (as currently Rust does not).  In this case, IIUC, you'd wind up with just
>> 
>> m1 m2 m1
>> 
>> which is *not* equivalent to m2.
> 
> Right.
> 
> > This would then make the 'z' in the definition of 'z' not bound-identifier=? to the 'z' that's the result of expanding the 'does-local-expand'.
> 
> Actually, neither (z with {m2}) nor (z with [m1 m2 m1]) is bound-identifier=? to (z with {}), but at that point bound-identifier=? isn't the relevant function; resolve is. In the absence of rename wraps (introduced by binding forms), both of them resolve to the same z as the definition, because resolve just throws away marks outside of a rename.

Right, yes, of course. 
> 
> (The resolve function does use bound-identifier=? internally to determine whether a rename wrap is applicable, but with arguments that are pieces of the identifiers in question.)
> 
>> So: did what I said make sense?  Is there a more compelling example? Am I right in suggesting that for this particular macro, it may not make a difference?
> 
> Yes, for this particular macro, it shouldn't make a difference.
> 
> The extra unmark/remark step matters when binders are involved. For example, if you local-expand to uncover a definition, then create a letrec from the definition and another expression in the body that didn't go through the same local-expand call, then the marks on the binder will be out of sync with the marks on the references in the body.
> 
> Unfortunately, the example that I thought of to illustrate this also seems to require the special mark-ignoring behavior described at the very end of Section 3.8 of MTWT.
> 
> Here's the example anyway, if you're curious:
> 
> ;; (le/var var exp) local-expands exp, then puts the result in a
> ;; let-binding of var to 0
> (define-syntax (le/var stx)
>  (syntax-case stx ()
>    [(_ var exp)
>     #`(let ([var 0]) #,(local-expand #'exp 'expression #f))]))
> 
> (let ([z 13])
>  (le/var z
>    (let ()
>      (define-syntax m
>        (syntax-rules ()
>          [(m) z]))
>      (m))))
> ;; => 0 in Racket

Thanks!

Rust doesn't currently use the "definition context" machinery at all, so this may be moot for us. It sounds like the right thing would be to condense this into a comment in the source, to baffle and frighten future Rust developers. :(

Thanks again,

John