[racket-dev] generalized `begin-for-syntax'

The `begin-for-syntax' form in v5.1.3.7 supports variable and macro
definitions within a module for all phases N >= 0 (finally!).


As a simple example, you can use `struct' for-syntax without having to
add `#:omit-define-syntaxes':

 #lang racket

 (begin-for-syntax
  (require racket/match)

  (struct posn (x y)) ; binds `posn' as syntax

  (match (posn 1 2)
    [(posn x y) ; uses `posn' as syntax
     (displayln (list x y))]))


You can also define macros at the top level of a module to
be used by macro implementations within the module:

 #lang racket

 (begin-for-syntax
  ;; like `syntax-rules', but quasiquotes the template:
  (define-syntax-rule (syntax-rules~ () [pat tmpl] ...)
    (lambda (stx)
      (syntax-case stx ()
        [pat #`tmpl]
        ...)))
  ;; plain old compile-time helper:
  (define (check-id stx)
    (unless (identifier? stx)
      (raise-syntax-error #f "not an identifier" stx))
    stx))

 (define-syntax define-five
   (syntax-rules~ ()
     [(_ id) (define #,(check-id #'id) 5)]
     [(_ id 5) (define-five id)]))

 (define-five V)
 (define-five wu 5)
 (+ V wu)


Perhaps most usefully, you can define a syntax class in the same module
as a macro that uses `syntax-parse':

 #lang racket
 (require (for-syntax syntax/parse))

 (begin-for-syntax
  (define-syntax-class two
    (pattern [x y])))

 (define-syntax (ark stx)
   (syntax-parse stx
     [(_ a:two ...)
      #'(list '(a ...))]))

 (ark [snake snake] [tiger tiger] [ant ant])


A `begin-for-syntax' form can contain `begin-for-syntax' forms with
phase-2 definitions, and so on.

----------------------------------------

As part of this change, the grammar of fully expanded expressions has
changed. It no longer includes `define-values-for-syntax', but
`begin-with-syntax' is a core form, and `begin-with-syntax' forms can
be nested.

Code that uses `kernel-syntax-case' with a `define-values-for-syntax'
case needs to be changed to one with a `begin-with-syntax' case. (I've
changed the ones in the main code repo.) Fortunately, most code that
uses `kernel-syntax-case' only handles expressions, in which case there
is no `define-values-for-syntax' case.

The `syntax-local-module-defined-identifiers' function now returns a
hash table mapping phase levels to lists, instead of two lists.

Some structs from `compiler/zo-structs' have changed: `def-for-syntax'
is gone, `seq-for-syntax' is added, some `mod' fields have changed
(such as a `syntax-bodies' instead of `syntax-body'), and
`def-syntaxes' has a new field.

The order of expansion for a module body has changed slightly. A
`begin-for-syntax' form is expanded with the same two-pass algorithm as
a module body overall, which means that partial expansion is used to
expose definitions and meta-forms before expanding expressions. All
`provide' (or, more precisely, `#%provide') forms across all phases are
delayed until the module is otherwise expanded, and then the
`#%provide' forms are expanded in the order that they appear within the
module (independent of phase).