[racket] Inner workings of source annotation

That's it!

There is even a little documentation so you could try to re-use the
traversal function separate of the GUI if you want. Let me know if
there's something more to improve in this direction.

Robby

On Fri, Jun 20, 2014 at 8:10 PM, Jonker, Todd <jonker at amazon.com> wrote:
> Thank you for the tips pointing me in the right direction.
>
> Apologies for not giving more context; that would certainly help. I’ve been
> embedded deeply in Racket-space for a couple years now, but lawyers prevent
> me from talking about my work and I’ve only recently been given permissions
> to ask questions in public. Please forgive my being indirect and opaque when
> I’d much rather dish the dirt.
>
> I have a fairly intimate understanding of the macro expander, and understand
> that the resulting syntax objects retain source location, lexical context,
> and various properties.  Very soon after sending my email it dawned on me
> that I’d mistakenly assumed that the editor retained the original source
> objects and built a visual model atop that, but now I see that’s not the
> case. It looks like the association between the expanded source objects and
> the original source text is done solely through the source-location. Fair
> enough.
>
> Since my last mail I discovered drracket/private/syncheck/traversals which
> seems to be the meat of it. I see that it recursively parses the expanded
> code (core syntax forms), builds several indices and cross-references, then
> sends the results to the editor layer via various syncheck-annotations
> methods.
>
> In other words I think I’ve got a handle on it now.
>
> Thanks again everyone!
>
> —Todd
>
> From: Matthias Felleisen <matthias at ccs.neu.edu>
> Date: Friday, June 20, 2014 at 5:22 PM
> To: "Todd V. Jonker" <jonker at amazon.com>
> Cc: "users at racket-lang.org" <users at racket-lang.org>
> Subject: Re: [racket] Inner workings of source annotation
>
>
> On Jun 20, 2014, at 5:07 PM, Jonker, Todd wrote:
>
> Greetings,
>
> I’m interested in learning how DrRacket’s “Check Syntax” feature works.
>
> As I understand it, the process is (1) perform full macro-expansion on the
> source code; (2) map elements of the expanded code back to the source code;
> (3) use that mapping to apply annotations/markup to the source code. The
> documentation at http://docs.racket-lang.org/tools/Check_Syntax.html seems
> to cover step (3), but I’m struggling to understand how (2) works.
>
> In other words, I’m trying to learn how the lexical information in the
> expanded syntax objects is mapped back to the appropriate parts of the
> original syntax objects. I’m reading the code to try to figure out the
> algorithm but hoping that there’s a paper that describes it more directly.
>
> Thanks in advance for any pointers,
>
>
>
>
> Hi Todd,  good to see you on the list.
>
> (1) local-expand is your friend, (2) the syntax system already propagates
> source location information (see syntax/loc in the docs), and (3) if I am
> guessing correctly you may want to look at "syntax property" lists in the
> docs just in case you want to implement a language of your own that needs to
> communicate between layers of expansion.
>
> The Typed Racket example is the best documented in this regard. And the best
> paper to read is
>
>   Languages as Libraries
>   http://www.ccs.neu.edu/racket/pubs/#pldi11-thacff
>
> If you play with Typed Racket, you will see that DrRacket 'lifts through'
> the syntax. It will do so for any language you create in the #lang spirit
> (with 2) and even macros.
>
> There is also a paper on DrScheme and its implementation, which explains a
> bit about other aspects of implementing these things. It's probably much
> less helpful
>
> If this doesn't help, perhaps you can ask more specific questions. --
> Matthias
>
>