[racket] New grammar for highlighting Racket code on GitHub

From: Jens Axel Søgaard (jensaxel at soegaard.net)
Date: Thu Dec 18 10:31:00 EST 2014

Hi All,

Github has until recently used a lexer for Racket from Pygments to
recognize and color tokens
for programs written in Racket. Unfortunately they have stopped using
Pygments and now
use TextMate grammars. TextMate is an editor that is famous for its
support for a plethora
of programming languages. To support many languages TextMate (read
Allan Odgaard)
created a file format to describe language grammars. Being a success
in TextMate, the
editors SublimeText[123] has chosen to support the same grammars. Github's
SublimeText copy (the editor Atom) also supports these grammars. Since the two
favorite editors in the Racket community are DrRacket and Emacs, the
Racket grammar
needed an upgrade. I have thus attempted to improve the grammar, see link below.
Before making a pull request to Github, I like to run the new
highlighter by you in order
to fix any oversights first.

Since the TextMate grammars are based on regular expressions (mostly being
applied to one line at a time), there are some restrictions on the
kind of constructs that can
be recognized. For example, nested multiline comments are at best
tricky (I am tempted to
say impossible) to lex parse properly. For the same reason s-expressions
comments (prefixed with #;) are handled by just coloring the #;.

Here is an example of the current colorer:


Try to color different files and report any mistakes.

Questions:         1) All datums get the same color.
                               Would it make sense to color strings

                           2) All syntax bound in #lang racket are
colored as keywords.
                               Should this be restricted to commonly
used control constructs only?

Jens Axel Søgaard

Posted on the users mailing list.