[plt-scheme] Transforming expanded syntax from the top

On 8/26/06, Casey Klein <clklein at calpoly.edu> wrote:
> I'm working on an algebraic stepper for the ProfessorJ languages.  My
> initial attempt annotates the module produced by ProfessorJ before
> it's expanded (via a #%module-begin macro), because this
> transformation needs to distinguish "user" expressions from those
> composing ProfessorJ's run-time system, and this is more complicated
> after expansion.
>
> Transforming pre-expansion, however, turns out to have some problems,
> and so I'd instead prefer to transform the module body after it's
> expanded (working with the entire body provides the context to
> identify user expressions), but I'm unsure how to tie into the macro
> system at this point in expansion -- i.e., at the body of the expanded
> module.

Hi Casey,

I've managed to exactly this, for my own project.  It's not especially
straightforward.

First, if you want your modules to be able to define new macros, then
you need to do the expansion in phases instead of all at once.
Matthew pointed me at
collects/lang/private/contracts/contracts-module-begin.ss as an
example of how this can be done (look at the bottom of the file, where
module-begin and module-continue are defined, and pay attention to
anything that says "lift").  Basically you take the module body, do a
local-expand with a stop list that define-values, require, provide,
etc, and then process each top-level declaration individually.  Be
sure to include define-values in the stop list, otherwise you'll get
an error during expansion (and you'll have a very hard time figuring
out where the error is coming from).  You probably also want to
include #%app and #%top in your stop-list, so that you don't expand
top-level expressions (otherwise you'll run into various errors about
references to undefined identifiers).

When you traverse the top-level definitions, you'll have trouble
detecting top-level expressions.  The only reliable way of detecting
them is to eliminate all the other alternatives (require, provide,
define-values, etc).  You can't just look for calls to #%app, since
built-in primitives like "if" or "begin" won't be wrapped with #%app.

Now, if you also want to expand function definitions (instead of just
stopping once you get to the top-level define-values form), then
you'll need to perform a second local-expand whenever you come across
an expression.  Be sure to include #%top in your stop-list during this
local-expand, otherwise you'll run into "identifier not defined"
errors when expanding recursive functions or references to functions
defined later in the file.  Don't include #%app in the stop-list,
otherwise expressions won't get expanded at all.

Finally, if you're doing any modification of the resulting expressions
(as opposed to just examining them, or adding new definitions), then
you will very likely run into problems with certificates.  This is
somewhat complicated, but basically if you modify any certified
expression, then you lose its certificate.  This will cause an error
during expansion, which makes debugging very difficult.  To get around
this, you will need to perform one final local-expand just before you
return the resulting syntax.  Wrap this local-expand in a
with-handlers clause, and catch exn:fail:syntax.  That's the only way
to know whether you've angered off the certificate manager or not.

Be sure to use the correct context for all the various local-expands.
The first one, which stops as soon as it hits a top-level declaration,
should use a 'module context.  The second one, which expands function
bodies, should use 'top-level (not 'expression -- otherwise you don't
get the behavior that turns local defines into letrecs).

I think that's most of the issues I've encountered.

--
Kimberley Burchett
http://www.kimbly.com/