<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>Thank you for the tips pointing me in the right direction.</div>
<div><br>
</div>
<div>Apologies for not giving more context; that would certainly help. I’ve been embedded deeply in Racket-space for a couple years now, but lawyers prevent me from talking about my work and I’ve only recently been given permissions to ask questions in public.
Please forgive my being indirect and opaque when I’d much rather dish the dirt.</div>
<div><br>
</div>
<div>
<div>I have a fairly intimate understanding of the macro expander, and understand that the resulting syntax objects retain source location, lexical context, and various properties. Very soon after sending my email it dawned on me that I’d mistakenly assumed
that the editor retained the original source objects and built a visual model atop that, but now I see that’s not the case. It looks like the association between the expanded source <i>objects</i> and the original source <i>text</i> is done solely through
the source-location. Fair enough.</div>
<div><br>
</div>
<div>Since my last mail I discovered drracket/private/syncheck/traversals which seems to be the meat of it. I see that it recursively parses the expanded code (core syntax forms), builds several indices and cross-references, then sends the results to the editor
layer via various syncheck-annotations methods.</div>
</div>
<div><br>
</div>
<div>In other words I think I’ve got a handle on it now.</div>
<div><br>
</div>
<div>Thanks again everyone!</div>
<div><br>
</div>
<div>—Todd</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Matthias Felleisen <<a href="mailto:matthias@ccs.neu.edu">matthias@ccs.neu.edu</a>><br>
<span style="font-weight:bold">Date: </span>Friday, June 20, 2014 at 5:22 PM<br>
<span style="font-weight:bold">To: </span>"Todd V. Jonker" <<a href="mailto:jonker@amazon.com">jonker@amazon.com</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:users@racket-lang.org">users@racket-lang.org</a>" <<a href="mailto:users@racket-lang.org">users@racket-lang.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [racket] Inner workings of source annotation<br>
</div>
<div><br>
</div>
<div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<div><br>
</div>
<div>On Jun 20, 2014, at 5:07 PM, Jonker, Todd wrote:</div>
<div><br class="Apple-interchange-newline">
<blockquote type="cite">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>Greetings,</div>
<div><br>
</div>
<div>I’m interested in learning how DrRacket’s “Check Syntax” feature works.</div>
<div><br>
</div>
<div>As I understand it, the process is (1) perform full macro-expansion on the source code; (2) map elements of the expanded code back to the source code; (3) use that mapping to apply annotations/markup to the source code. The documentation at
<a href="http://docs.racket-lang.org/tools/Check_Syntax.html">http://docs.racket-lang.org/tools/Check_Syntax.html</a> seems to cover step (3), but I’m struggling to understand how (2) works.</div>
<div><br>
</div>
<div>In other words, I’m trying to learn how the lexical information in the expanded syntax objects is mapped back to the appropriate parts of the original syntax objects. I’m reading the code to try to figure out the algorithm but hoping that there’s a paper
that describes it more directly.</div>
<div><br>
</div>
<div>Thanks in advance for any pointers,</div>
</div>
</blockquote>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Hi Todd, good to see you on the list. </div>
<div><br>
</div>
<div>(1) local-expand is your friend, (2) the syntax system already propagates source location information (see syntax/loc in the docs), and (3) if I am guessing correctly you may want to look at "syntax property" lists in the docs just in case you want to
implement a language of your own that needs to communicate between layers of expansion. </div>
<div><br>
</div>
<div>The Typed Racket example is the best documented in this regard. And the best paper to read is </div>
<div><br>
</div>
<div> Languages as Libraries </div>
<div> <a href="http://www.ccs.neu.edu/racket/pubs/#pldi11-thacff">http://www.ccs.neu.edu/racket/pubs/#pldi11-thacff</a></div>
<div><br>
</div>
<div>If you play with Typed Racket, you will see that DrRacket 'lifts through' the syntax. It will do so for any language you create in the #lang spirit (with 2) and even macros. </div>
<div><br>
</div>
<div>There is also a paper on DrScheme and its implementation, which explains a bit about other aspects of implementing these things. It's probably much less helpful </div>
<div><br>
</div>
<div>If this doesn't help, perhaps you can ask more specific questions. -- Matthias</div>
<div><br>
</div>
<div><br>
</div>
</div>
</div>
</div>
</span>
</body>
</html>