[racket] Formal Presentation and initial doubts.

Will has some bits right and quite a few bytes wrong. 

0. It is not about macros. It is about having a good API to the compiler. 

1. It is not about hygiene alone. As John points out, there are other aspects of Racket's syntax extension system that make it easy, for instance pattern matching. 

2. It is about the expressive power of the API and ease of use. 

;; --- 

Let's take these points in turn. 

-- Plain old macros are about adding a new form of syntax to the language. Here is an example: 

(define-synatx-rule 
  (block ((define x e-rhs) ...) e-body)
 ;; --> 
  (letrec ((print-all (lambda (x) (displayln `(,x ...)))) (x e-rhs) ...) e-body))

The three lines say (1) I am a simple syntax extension, (2) whenever the compiler sees this pattern, then (3) it rewrites it to this template. 

EXERCISE: write 'block' in the Common Lisp macro system. 



-- Macros substitute code into each other in two different directions. The code from the use site flows into the macro and the macro flows into use context. Here are three scenarios for block: 

> (block ((x 1) (y 2)) (+ x y)) 
no problem, works like a charm

> (let ((displayln list) block ((x 1) (y 2)) (+ x y)))
QUESTION: should 'displayln' in the syntax rule refer to the 'displayln' defined in the lexical scope of this particular use site? Or should it refer to the 'displayln' that the macro author had in mind? 

> (let ((print-all (lambda () (write "hello world"))) (block ((x 1) (y 2)) (print-all))
QUESTION: should the inner 'print-all' refer to the 'print-all' that the rewriting of 'block' injects into the abstract syntax tree or should it refer to the 'print-all' that the programmer defined in the use context of 'block' here? 

HYGIENE sets up a reasonable default for these last two scenarios. Clearly, the 'displayln' should take its meaning from the macro definition site, and the inner 'print-all' should refer to the one defined in the use context. 

I don't think you need to understand the implementation of hygiene to figure out what happens here. 

Also, hygiene as presented here adds new expressive power over the Common Lisp macros. You simply cannot even express the idea in CL that 'displayln' should take its meaning from the definition site of the macro. Period. 



--- While hygiene works most of the time, it can be the wrong default. A programmer may wish to say that a new linguistic construct like 'block' ought to have the power to inject a new bound variable into a programmer's code. Ergo, we need a way to override the hygiene default. 



-- Even if you don't buy the need for 'hygiene' power, you ought to accept the idea that language extensions should signal behave as if they were baked into the language. In particular, 'block' doesn't warn its clients that the sequence of x ... should be repetition-free. 

> (block ((x 1) (x 2)) x)

QUESTION: what happens here? ANSWER: you get an error message from a 'letrec' but you don't see a 'letrec' anywhere. Simple macro systems, such as those of Clojure, Common Lisp, and Scheme force serious macro programmers to write many lines of extra code to get the error messages for 'block' formulated in terms of 'block'. But these pieces of error-checking code appear over and over again, and they resemble each other to the dot. What do good programming language designers do? They abstract over these repetitions. That's what Racket's syntax-parse form does. 




-- Syntax systems are about building language extensions as language restrictions and languages period. If you accept that, macros aren't enough. First you want to encapsulate syntactic revisions as libraries that others can use. That means you need to have a mechanism to export macros across library boundaries. Second you want the ability to remove existing syntax, to re-define existing syntax, and to communicate among the trees of an AST forest. Racket supports all of this in its syntax extension system. Finally, there are a few other tweaks that don't deserve spelling out here, but again, many of these tweaks were recognized as useful to compiler-API programmers and have found their way into Racket's syntax system. 


Now look back at these steps. Clearly you can learn the syntax system of Racket in several layers. The above is the simplest path through these layers but I am sure there are others that could be of value. You can stop any time and you have added useful expressive power to your vocabulary. And you have something to look forward to when you have free time on your hand again or your uses of the syntax system tell you that there must be a better way to get things done. 


Think back to the time before you understood closures. I am sure you couldn't even imagine them. Now imagine them away. See you can't do that either. 

-- Matthias















On Mar 21, 2014, at 6:49 PM, Daniel Prager wrote:

> Thank-you for raising this topic.
> 
> > I don't understand well its macros (are completely different for me than CL macros).
> 
> Me neither. And I'm somewhat torn. The trade-off as I understand it is that CL-style macros are *much* easier to learn, but prone to aliasing while hygenic macros are technically superior because they compose better and you can't accidentally hit aliasing problems. But when it comes to learning, the advantages are reversed.
> 
> William Byrd, in this 2012 interview with Michael Fogus, comments on some of these issues:
> Clojure's macro system appears to me to be a kludge, trying to patch up Common Lisp's defmacro without going all the way to real hygienic macros. I don't mean to be overly critical--I'm not happy with the state of Scheme, either, which has not one but two hygienic macro systems, neither of which is ideal.
>  
> The more powerful Scheme macro system, syntax-case, gives the programmer the full power of Scheme at macro expansion time, along with the ability to "bend" hygiene when desired. Syntax-case is powerful enough to define Common Lisp's defmacro (and almost certainly powerful enough to define Clojure's defmacro as well). Unfortunately, syntax-caseis notoriously complicated. As far as I can tell, mastering syntax-case requires a mental model equivalent in complexity to the macro expander's implementation. As a result, Dan, Oleg, and I refuse to use syntax-case.
>  
> The other Scheme macro system, syntax-rules, is strictly less powerful than syntax-case, and can in fact be implemented in syntax-case. However, the mental model required to use syntax-rules is much simpler, as it is essentially a term-rewriting system with hygiene. For vanilla forms such as let, and, and or, syntax-rules works beautifully. For incredibly complicated macros, such as the match pattern matcher used in Kent Dybvig's compiler course at Indiana, the full power of syntax-case is probably required. For almost all other macros, it seems like a system with just a little more power than syntax-rules would be both sufficient and ideal. For example, often a macro writer wishes to concatenate two symbols--currently this requires using syntax-case, even if the rest of the macro is trivial.
>  
> If a hygienic macro system that hits the sweet spot between syntax-rules and syntax-caseis created, I hope it will be adopted by Schemers and Clojurers alike. A few steps in this direction include the Dylan macro system and Ryan Culpepper's syntax-parse. Also needed is a way to learn how to write non-trivial hygienic macros that doesn't involve earning a PhD in programming languages from Indiana or Northeastern.
> 
> The Racket and Scheme inner circle say that hygienic macros are the future, but I wonder whether a worse-is-better argument can be mounted for CL and/or Clojure macros.
> 
> Personally, I only write Racket macros sparingly, but would like to develop my level of understanding of comfort.
> 
> What are some good ways to smooth out the learning curve?
> 
> 
> Dan
> 
> 
> 
> On Sat, Mar 22, 2014 at 12:42 AM, Matthias Felleisen <matthias at ccs.neu.edu> wrote:
> 
> Welcome.
> 
> Why don't you take a look at Greg's macro write-up
> 
>  http://www.greghendershott.com/fear-of-macros/
> 
> and see whether this helps you wrap your head around macros.
> 
> 
> 
> 
> On Mar 21, 2014, at 9:48 AM, Alejandro Zamora Fonseca <terefv at ltu.sld.cu> wrote:
> 
> > Hi!!
> > My name is
> > Alejandro, i'm a Cuban Computer Scientist, new to Lisp(having 1 year using CL), and wishing change to Racket.
> > I see the list and I hope make me a better Lisp programmer with your kindly help.
> >
> > I have Racket installed in my PC, I like the language and tools, but yet I don't understand well its macros(are completely different for me than CL macros) and continuations.
> >
> > Greetings for all.
> >
> > Alejandro
> >
> >
> >
> >
> > --
> >
> > Este mensaje le ha llegado mediante el servicio de correo electronico que ofrece Infomed para respaldar el cumplimiento de las misiones del Sistema Nacional de Salud. La persona que envia este correo asume el compromiso de usar el servicio a tales fines y cumplir con las regulaciones establecidas
> >
> > Infomed: http://www.sld.cu/
> >
> > ____________________
> > Racket Users list:
> > http://lists.racket-lang.org/users
> 
> 
> ____________________
>   Racket Users list:
>   http://lists.racket-lang.org/users
> 
> 
> 
> -- 
> Daniel Prager
> Agile/Lean Coaching, Software Development and Leadership
> Startup: www.youpatch.com
> Twitter: @agilejitsu 
> Blog: agile-jitsu.blogspot.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.racket-lang.org/users/archive/attachments/20140321/1445fb0e/attachment-0001.html>