[racket] Again on bindings visibility in eval

Hi Markku,

On Thu, 2011-07-14 at 13:22 +0300, Markku Rontu wrote:
> Hi Maurizio,
> 
> I guess what you are trying to do is possible with macros, but I could
> be wrong as it is difficult to understand what you are doing, because
> we all seem to lack a common vocabulary in this matter. Sorry if I
> misunderstood your intentions.
> 
My difficulty was to show what is my problem without giving lot of
details of my implementation ... it is too huge! The expander function
is recursive and is very complex. So I tried to reproduce in my posts
some simple examples that should give the "idea" of how my huge expander
function works.
 
I agree with you. I also guess what I am trying to do is possible with
macros (only).

> Your expander seems to be a function. I think you are trying to
> generate some syntax from a plain old function. And that is where you
> run into problems that you try to solve with eval. I suspect that you
> should program your expander as a set of utility macros. that build up
> the lambda you need. Perhaps no eval is needed but you do mention that
> you want to do certain optimizations.
Yes, I was used in my past lisp experiences to generate code by means
of: 
`(lambda () ...static-code... ,dynamic-parts ...).
This is what you mean for "generate some syntax from a plain old
function". An this is why I used "eval" to produce the procedure.
You had a good idea ("the expander as a set of utility macros..."), and
I am confident that if I used this approach I wouldn't need the eval.
About optimization... in my case the main optimization is:
1) My macro has to generate a function which has to perform some
recursive tasks 
(for example, it has to do a recursive pattern matching on a set of
elements)
2) Instead of generating a recursive function, I generate one very long
piece of code that expand (I say "inline") all the (potential) recursion
point.
 
> 
> This tutorial helped me to understand some basics of macros. For
> example how to generate code with them. Now Racket has lots of other
> facilities as well but maybe these give you ideas how to ask more
> precise questions?
> http://hipster.home.xs4all.nl/lib/scheme/gauche/define-syntax-primer.txt
> 
Thank you, I will read it...

> On the topic of macros, I find it a bit confusing that to metaprogram
> in Racket, I must use such a different language of macros (with
> syntax-rules et al.), instead of being able to manipulate common data
> structures with plain old Racket functions. The approach in Clojure,
> that the code is just a common data structure, easily manipulatable
> with Clojure code, is intriguing. What I must do with syntax-rules et
> al. brings me memories of template metaprogramming in C++, where I had
> to twist my mind in order to be able to do the most simple of things.
> Writing regular tree manipulation in Racket is easy, but manipulating
> the syntax tree with the macro support is not because of this
> "language switch".
> 
> Once more, to me it seems your expander tries to use plain old Racket
> to come up with a list of symbols, that is what you think your
> expanded program should look like. And then you try to make it a
> macro. I think this is not what the usual Racket way of writing macros
> is.
I agree, I am newish in racket ... and now I understand that 
I am not using in the right (or fully potential) way the racket macros.

Thank you,
Cheers,
Maurizio.
> 
> -Markku
> 
> On Thu, Jul 14, 2011 at 12:30 AM, Maurizio Giordano
> <maurizio.giorda at gmail.com> wrote:
>         Hi Matthias, hi all
>         
>         2011/7/13 Matthias Felleisen <matthias at ccs.neu.edu>
>                 
>                 
>                 [I hadn't forgotten your messages, it's just that
>                 deadlines got in the way.]
>                 
>                 I still don't understand your desire to access the
>                 environmental variables specially:
>                 
>                 1.  if you generate a closure or a struct full of
>                 closures in your macro, the expression that you pass
>                 in captures the variables in its context:
>                 
>                 > (define-syntax-rule (mystuff y expr) (lambda (x)
>                 (displayln `(expr ,y)) (if expr x 0)))
>                 > [  (let ((t 333)) (mystuff t (> t 444)))   22]
>                 ((> t 444) 333)
>                 0
>                 
>                 See it really does get t's value from the context.
>                 
>         Yes it does ... I know that if the macro returns the
>         lambda ...
>         no problem, unfortunately my macro returns something like:
>         (eval `(lambda ...))
>         Now the question is: Can I rewrite my macro to generate the
>         same code
>         without using the "(eval `(lambda  ...))"?
>         I don't know, I feel yes ... but at the moment I don't know
>         how to
>         redesign it.
>         My lambda code is not a predefined template
>         (the quasiquote) with some small parts to be instantiated
>         (with unquoting).
>         First, the lambda code is very large,
>         Second, it is recursively produced by an expander function
>         (used in the macro).
>         When you call the macro, you don't know in advance how many
>         times 
>         the expander will call itself. It is something like:
>         
>         
>         (define-syntax-rule replace
>            ...
>            (eval `(lambda (x) 
>                          ... static part ...
>                          ,(expander ...)   ; this inject a part of the
>         lambda code   
>                          ...)))
>         
>         
>         (define (expander ...)
>           `(let ((...))
>                ... static part ...
>                ,(expander ...)             ; injection once again   
>                ...))
>                 
>                 2a. do you have reason to access variables other than
>                 throwing them into the environment for eval? In that
>                 case your problem is solved.
>                  
>                 2b. if not, what are the reasons to guess at variable
>                 names in the context of the macro USE (not
>                 definition)?
>                 
>         I try to answer to both questions:
>         
>         
>         My macro is the primitive of a new language:
>         
>         
>         < elem1, elem2, ... , (replace x y by (+ x y) if (> x
>         othervar)) >
>         
>         
>         the replace macro has the following inputs:
>         x, y = free variables to be matched on a set of elements
>         (<...>)
>         (+ x y) = a form to be inserted in the set inspite of x and y
>         (> x othervar) = a conditional form: here x is matched (bound)
>         locally,
>         othervar is  a symbol "outside" the set (that is the scheme
>         top env or
>         any other inner env like a let)
>         This is why I need that the produced lambda USE outside
>         symbols.
>         At the moment, in my implementation, the lambda can use
>         outside
>         symbols if they are defined in the top env ... I would like to
>         have it also in
>         inner environments. 
>          
>           
>                 ;; ----------------
>                 
>                 
>                 SEPARATE QUESTION: Now you also write that you process
>                 the code before you throw it to eval. Is it possible
>                 to write functions on syntax that process the code and
>                 perform the optimizations that Racket doesn't perform
>                 for you?
>                 
>         That's an interesting question: yes ... I process A LOT the
>         code before
>         I throw it to the eval ... as I said you it is like a compiler
>         (... my first research area... many years ago!)
>         First: I generate recursively the code with a my expander
>         function (not the one of racket)  
>         Second: I (try) to generate an efficient code to simulate the
>         runtime of the language (the chemical language) I have
>         implemented on top of racket.
>         If you look at the generated code, you can figure out several
>         optimizations.
>         One is "inlining": if I have a lambda (in my case a chemical
>         rule) that will be executed many times (on all possible
>         combinations of elements in a set), 
>         and each time the lambda may use recursion, than, from my
>         experience it is
>         better to explode recursive calls in recursive inlining of
>         code.
>         From your question I understand that you mean "optimizations
>         of racket
>         code" that racket does not do.
>         
>         
>         Absolutely an interesting discussion... thanks.
>         
>         
>         Cheers,
>         Maurizio.
>         
>           
>                 -- Matthias
>                 
>                 
>                 
>                 
>                 
>                 
>                 
>                 On Jul 13, 2011, at 3:54 PM, Maurizio Giordano wrote:
>                 
>                 > Hi all, hi Matthias,
>                 >
>                 > I would like to come back to my (still unsolved)
>                 problem I
>                 > proposed some days ago:
>                 >
>                 > when using define-syntax in a inner scope of
>                 bindings (like let),
>                 > in this example:
>                 >
>                 > (define-syntax mymacro
>                 >   (syntax-rules ()
>                 >     [(_ x)
>                 >       (eval `(lambda (w) (print (quote x)) (if x w
>                 #f)))]))
>                 >
>                 > (let* ((s 3) (f (mymacro (> s 0)))) (f 5))
>                 > reference to undefined identifier: s
>                 >
>                 > of course if you:
>                 >
>                 > (define s 3)
>                 >
>                 > in the top environment, the error will not appear.
>                 > This is even more clear if you expand the macro:
>                 >
>                 > (syntax->datum (expand '(mymacro (> s 0))))
>                 > ... you see that "s" is a %top binding.
>                 >
>                 > I know that if I make the macro to return directly
>                 the lambda,
>                 > it works. Nevertheless, in my implementation, I
>                 still need to use
>                 > the "(eval (quasiquote (lambda ...)))".
>                 > Why? my macro is like a compiler
>                 > that generates a lambda code very huge: the code  is
>                 recursive in some of its part,
>                 > but, for efficiency reasons, i preferred to inline
>                 all recursive calls. So I use a
>                 > function "expander" that makes recursive inlining
>                 (or injecting) of code).
>                 > It is more or less something like:
>                 >
>                 > `(lambda (...) ... static-code...
>                  ,(expander ...) ...)
>                 >
>                 > where "expander call itself with different
>                 parameters.
>                 >
>                 > This is just to know your opinion... if the "(eval
>                 (quasiquote (lambda ...)))"
>                 > cannot see local bindings like in let*, than I have
>                 to choice:
>                 > 1) renounce to the feature.
>                 > 2) looking for alternative methods to generate my
>                 code with recursive inlining
>                 >
>                 > Than you,
>                 > Cheers,
>                 > Maurizio.
>                 
>                 
>         
>         
>         _________________________________________________
>          For list-related administrative tasks:
>          http://lists.racket-lang.org/listinfo/users
>