[racket] variables within macros

Let's take a step at a time. Don't lose patience. It will help 
you get where you want to be, and it may help us figure out how
to do better with this aspect of Racket, which is by far its most
innovative part and unrivaled in the world of PLs. 


On Jan 18, 2013, at 12:36 PM, Tim Brown wrote:

> On 18/01/13 15:54, Matthias Felleisen wrote:
>> Is there a single stumbling block or do you feel overwhelmed by
>> the broad API to the syntax system? -- Matthias
> 
> It's a combination of the two...
> 
> My single stumbling block is "binding" identifiers -- I'm never sure if
> I'm going to end up with an identifier referencing the same value/entity/
> thing. 


Like Scheme, Racket supports hygienic syntax extensions. What this 
means is that -- in principle -- you should rarely, if ever, worry 
about binding issues. Most of the time, they should just work. 


> A concrete example is that I am trying to write a #lang language. 

Honestly, I would never ever try to write a #lang first. 
That's more than most people want. 

The syntax system has two distinct roles (and possibly more): 

1. extend the Racket language so that developers don't have to 
repeat themselves all the time. This is about syntactic abstraction. 
You can also think of this step as extending the language. 

Abstracting over common syntactic patterns is usually a matter of 
writing a syntactic extension with one of several tools (syntax rules, 
syntax cases, syntax parse) and -- on some occasions -- to deposit 
additional definitions at the module level. That's about it. 

2. create entirely new languages. To so so, Racket enables you 
to start from an existing language and 
 -- to add constructs 
 -- to subtract constructs 
 -- to rewrite modules non-locally 
 -- to expand an existing module for further analysis 
Some of these objectives are achieved with syntactic abstractions; 
for others you really need some of the additional features of the API. 


> This is a bit of a big bite to have taken out of the macro system, but I
> have to start non-triviality somewhere. Equally, though extending the
> language is something that Racket encourages; and as far as I can tell,
> this is (or should be) a trivial extension.


You seem to want to extend Racket with a define/test construct. 
I propose to think of this as as syntactic extension of Racket. 
Let me start by arguing against thinking of this as a new language. 

See my line of reasoning at the end of the message. I didn't want
to pollute the interior of the message with all this code. 


> Thank you for listening :-)

Listening to you helps us. Thanks for speaking up. -- Matthias

GOAL: Let's say we wish to be able to write 

(define/test (add-one x)
  (tests (check-equal? (add-one 1) 2) 
         (check-equal? (add-one -1) 0)
         (check-equal? (add-one 2) 3)
         (check-equal? (add-one 2) 4))
  ;; -- IN -- 
  (+ x 1))

because we believe in self-documentation of functions via tests. 
Viva la design recipe. 

------------------------------------------------------------------
STEP 1: write a syntax rule 

#lang racket

(define-syntax-rule 
  (define/test (f x ...) (tests t ...) e ...)
  (begin
    (module+ test 
      (require rackunit) ;; we usually need rackunit for the t ...
      t ...)
    (define (f x ...) e ...)))

;; --- program proper --- 

(define/test (add-one x)
  (tests (check-equal? (add-one 1) 2) 
         (check-equal? (add-one -1) 0)
         (check-equal? (add-one 2) 3)
         (check-equal? (add-one 2) 4))
  (+ x 1))

DrRacket immediately shows that this fails with 

> check-equal?: unbound identifier in module in: check-equal?


Why? HYGIENE says that the "t ... " part comes from the source, 
the require part injects new bindings from somewhere else, so we need
to protect them from each other. 

Here is one way to justify this answer. You really may wish to 
make define/test orthogonal to which testing framework you use: 
 -- rackunit 
 -- test engine
 -- the Eli tester 

You say: BOGUS. I just want check-equal? to be bound. 

But the fix is easy. 

------------------------------------------------------------------
STEP 2: write a syntax rule and bind check-equal? in the source 

#lang racket

(define-syntax-rule 
  (define/test (f x ...) (tests t ...) e ...)
  (begin
    (module+ test 
      t ...)
    (define (f x ...) e ...)))

;; --- program proper --- 

(module+ test 
  (require rackunit))

(define/test (add-one x)
  (tests (check-equal? (add-one 1) 2) 
         (check-equal? (add-one -1) 0)
         (check-equal? (add-one 2) 3)
         (check-equal? (add-one 2) 4))
  (+ x 1))

And indeed, if you mouse over rackunit, DrRacket says that it 
binds all four occurrences of check-equal? -- which is what it 
should do. 

RUN. The fourth test fails, and drracket locates the failed test. 
Things work the way they should. 

Ah you say, but I want to provide this construct via a project-wide
library. Okay, let's split this file into two pieces. 

------------------------------------------------------------------
STEP 3: write a syntax rule in a separate library, provide it 

---------------------------------------------------
LIBRARY: 

#lang racket

(provide 
 ;; syntax: define/test
 ;; (define/test (f x ...) (tests t ...) e ...)
 ;; expands to a combination of two items:
 ;; -- the function (define (f x ...) e ...) 
 ;; -- tests t ... in submodule test 
 define/test)

;; ------------------------------------------------
;; implementation 

(define-syntax-rule 
  (define/test (f x ...) (tests t ...) e ...)
  (begin
    (module+ test 
      t ...)
    (define (f x ...) e ...)))

---------------------------------------------------
PROJECT FILE: 

#lang racket

(require "test-define.rkt")

(module+ test
  (require rackunit))

(define/test (add-one x)
  (tests (check-equal? (add-one 1) 2) 
         (check-equal? (add-one -1) 0)
         (check-equal? (add-one 2) 3)
         (check-equal? (add-one 2) 4))
  (+ x 1))

Works like a charm. We have created a well-documented, 
project-wide library that does what we want. 

Now you may object to a repeated syntactic pattern: 
 ** every time you import 'test-define.rkt', you also 
    need to write (module+ test (require rackunit)). 


------------------------------------------------------------------
STEP 4: write a syntax rule in a separate library, provide it 

---------------------------------------------------
LIBRARY: 

#lang racket 

(require rackunit)

(provide 
 (all-from-out rackunit)
 ;; ------------------------------------------
 ;; syntax: define/test
 ;; (define/test (f x ...) (tests t ...) e ...)
 ;; expands to a combination of two items:
 ;; -- the function (define (f x ...) e ...) 
 ;; -- tests t ... in submodule test 
 define/test)

;; ------------------------------------------------
;; implementation 

(define-syntax-rule 
  (define/test (f x ...) (tests t ...) e ...)
  (begin
    (module+ test 
      t ...)
    (define (f x ...) e ...)))


---------------------------------------------------
PROJECT FILE: 

#lang racket

(require "test-define.rkt")

(define/test (add-one x)
  (tests (check-equal? (add-one 1) 2) 
         (check-equal? (add-one -1) 0)
         (check-equal? (add-one 2) 3)
         (check-equal? (add-one 2) 4))
  (+ x 1))

Now this one is even more charming: 

 1. we have a syntax library 
 2. we have no pattern left; 
	the library takes care of it all 

Is there an objection left? I could see that you may wish to 
argue that the all-from-out pollutes the entire module scope
with the bindings from rackunit. You really just waned those
in the test submodule. 

Here is my reply: I think the objection is not well-founded. 
When I see software engineering problems with this "pollution", 
I will believe it and I will work out a solution. 

PRINCIPLE: stick to syntactic extensions until you really really
need new languages. Keep new languages rare. 

IF I found such a software engineering reason, I would try to 
figure out how to solve it w/o resorting to a new language. 


Let us know what you think up to this point.