[racket] Anyone familiar with mercurial? Add syntax coloring to github?

From: Greg Hendershott (greghendershott at gmail.com)
Date: Tue Aug 14 19:36:05 EDT 2012

I submitted a pull request,
<https://bitbucket.org/birkenfeld/pygments-main/pull-request/94/add-lexer-for-racket-language>.

On Mon, Aug 13, 2012 at 11:25 AM, Greg Hendershott
<greghendershott at gmail.com> wrote:
>> And a terrible shame that you cannot reuse the syntax colorer we
>> already have! (but you may find the source code in
>> collects/syntax-color/scheme-lexer.rkt to be useful?)
>
> Yes I suppose there's duplication among DrRacket, Quack, pygments, et
> al. A central web service could be a neat way to go in some
> always-connected future.
>
> Meanwhile, Jens helped me again (thanks!) pointing me to how to
> generate the `keywords' and `built-ins' lists using
> `namespace-mapped-symbols'. I incorporated that. Noticed and fixed a
> few bugs.
>
> I'll marinade in it for a day or two, then consider giving pygments a
> pull request.
>
> BTW I'm trying to keep in mind this is a syntax highlighter, not
> checker. I want something better than the Scheme highlighter, but
> sometimes "the perfect is the enemy of the good". Trying to handle
> every number literal variation perfectly, is me probably already
> trying too hard.
>
> On Mon, Aug 13, 2012 at 8:53 AM, Robby Findler
> <robby at eecs.northwestern.edu> wrote:
>> Its great to see someone working on this!
>>
>> And a terrible shame that you cannot reuse the syntax colorer we
>> already have! (but you may find the source code in
>> collects/syntax-color/scheme-lexer.rkt to be useful?)
>>
>> Robby
>>
>> On Mon, Aug 13, 2012 at 7:46 AM, Greg Hendershott
>> <greghendershott at gmail.com> wrote:
>>> I spent more time on this, including trying to recognzie all the
>>> variations of numbers like these:
>>>
>>> (values
>>>  ;; #b
>>>  #b1.1
>>>  #b-1.1
>>>  #b1e1
>>>  #b1/1
>>>  #b0/1
>>>  #b1e-1
>>>  #b101
>>>  #b2 ;highlight as error
>>>
>>>  ;; #d
>>>  #d-1.23
>>>  #d1.123
>>>  #d1e3
>>>  #d1e-22
>>>  #d1/2
>>>  #d-1/2
>>>  #d1
>>>  #d-1
>>>  #dZ ;highlight as error
>>>
>>>  ;; No # reader prefix -- same as #d
>>>  -1.23
>>>  1.123
>>>  1e3
>>>  1e-22
>>>  1/2
>>>  -1/2
>>>  1
>>>  -1
>>>
>>>  ;; #e
>>>  #e-1.23
>>>  #e1.123
>>>  #e1e3
>>>  #e1e-22
>>>  #e1
>>>  #e-1
>>>  #e1/2
>>>  #e-1/2
>>>  #eZ ;highlight as error
>>>
>>>  ;; #i always float
>>>  #i-1.23
>>>  #i1.123
>>>  #i1e3
>>>  #i1e-22
>>>  #i1/2
>>>  #i-1/2
>>>  #i1
>>>  #i-1
>>>  #iZ ;highlight as error
>>>
>>>  ;; #o
>>>  #o777.777
>>>  #o-777.777
>>>  #o777e777
>>>  #o777e-777
>>>  #o3/7
>>>  #o-3/7
>>>  #o777
>>>  #o-777
>>>  #08 ;highlight as error
>>>
>>>  ;; #x
>>>  #x-f.f
>>>  #xf.f
>>>  #x-f
>>>  #xf
>>>  #xG ;highlight as error
>>>
>>>  )
>>>
>>> In this way (Python):
>>>
>>>             ## numbers: Keep in mind Racket reader hash prefixes,
>>>             ## which can denote the base or the type. These don't map
>>>             ## neatly onto pygment token types; some judgment calls
>>>             ## here.  Note that none of these regexps attempt to
>>>             ## exclude identifiers that start with a number, such as a
>>>             ## variable named "100-Continue".
>>>
>>>             # #b
>>>             (r'#b[-+]?[01]+\.[01]+', Number.Float),
>>>             (r'#b[01]+e[-+]?[01]+', Number.Float),
>>>             (r'#b[-+]?[01]/[01]+', Number),
>>>             (r'#b[-+]?[01]+', Number.Integer),
>>>             (r'#b\S*', Error),
>>>
>>>             # #d OR no hash prefix
>>>             (r'(#d)?[-+]?\d+\.\d+', Number.Float),
>>>             (r'(#d)?\d+e[-+]?\d+', Number.Float),
>>>             (r'(#d)?[-+]?\d+/\d+', Number),
>>>             (r'(#d)?[-+]?\d+', Number.Integer),
>>>             (r'#d\S*', Error),
>>>
>>>             # #e
>>>             (r'#e[-+]?\d+\.\d+', Number.Float),
>>>             (r'#e\d+e[-+]?\d+', Number.Float),
>>>             (r'#e[-+]?\d+/\d+', Number),
>>>             (r'#e[-+]?\d+', Number),
>>>             (r'#e\S*', Error),
>>>
>>>             # #i is always inexact-real, i.e. float
>>>             (r'#i[-+]?\d+\.\d+', Number.Float),
>>>             (r'#i\d+e[-+]?\d+', Number.Float),
>>>             (r'#i[-+]?\d+/\d+', Number.Float),
>>>             (r'#i[-+]?\d+', Number.Float),
>>>             (r'#i\S*', Error),
>>>
>>>             # #o
>>>             (r'#o[-+]?[0-7]+\.[0-7]+', Number.Oct),
>>>             (r'#o[0-7]+e[-+]?[0-7]+', Number.Oct),
>>>             (r'#o[-+]?[0-7]+/[0-7]+', Number.Oct),
>>>             (r'#o[-+]?[0-7]+', Number.Oct),
>>>             (r'#o\S*', Error),
>>>
>>>             # #x
>>>             (r'#x[-+]?[0-9a-fA-F]+\.[0-9a-fA-F]+', Number.Hex),
>>>             # the exponent variation (e.g. #x1e1) is N/A
>>>             (r'#x[-+]?[0-9a-fA-F]+/[0-9a-fA-F]+', Number.Hex),
>>>             (r'#x[-+]?[0-9a-fA-F]+', Number.Hex),
>>>             (r'#x\S*', Error),
>>>
>>> Talk about brain burn. :)
>>>
>>> Also, Jens pointed out I should also handle curly braces. () = [] = {}
>>>
>>> https://bitbucket.org/greghendershott/pygments-main
>>>
>>> On Sun, Aug 12, 2012 at 6:50 PM, Sam Tobin-Hochstadt <samth at ccs.neu.edu> wrote:
>>>> Wonderful!  This will make the github experience much nicer.
>>>>
>>>> On Sun, Aug 12, 2012 at 6:36 PM, Greg Hendershott
>>>> <greghendershott at gmail.com> wrote:
>>>>> So I followed up on this.
>>>>>
>>>>> https://bitbucket.org/greghendershott/pygments-main/changeset/240e51e2da13b079482f6b61c215280224f89f06
>>>>>
>>>>> ~~~~~
>>>>> Add RacketLexer.
>>>>>
>>>>> Previously Racket files were handled by SchemeLexer. Instead, use a
>>>>> proper RacketLexer, which handles Racket more appropriately:
>>>>>
>>>>>  1. Treat square brackets like parentheses.
>>>>>  2. Expanded list of keywords.
>>>>>  3. Different file extensions, MIME types, etc.
>>>>>  4. Handle #:keyword arguments.
>>>>>  5. Handle more number literals (e.g. #xFF, #o777, 2e2, #e232, etc.).
>>>>>  6. Handle #| ... |# multiline comments (although NOT nested).
>>>>> ~~~~~
>>>>>
>>>>> Before I give them a pull request, I wanted to give folks here a
>>>>> chance to critique it (or even flatly veto it).
>>>>>
>>>>> Again, the goal is that eventually this would flow through to GitHub,
>>>>> and improve the readability of Racket repos and gists.
>>>>>
>>>>> On Thu, Jun 2, 2011 at 12:49 PM, John Clements
>>>>> <clements at brinckerhoff.org> wrote:
>>>>>>
>>>>>> On Jun 2, 2011, at 6:43 AM, Eli Barzilay wrote:
>>>>>>
>>>>>>> An hour ago, Vincent St-Amour wrote:
>>>>>>>> At Thu, 2 Jun 2011 06:30:00 -0400,
>>>>>>>> Greg Hendershott wrote:
>>>>>>>>>  aliases = ['scheme', 'scm', 'ss', 'racket', 'rkt']
>>>>>>>>
>>>>>>>> rktl would probably fit in there too.
>>>>>>>
>>>>>>> (This is just in case someone takes it on more seriously: IIRC,
>>>>>>> another issue with pygments was either ignoring square brackets, or
>>>>>>> highlighting them as errors.)
>>>>>>
>>>>>> Gosh, sounds complicated.  I don't think any of us could handle that!
>>>>>>
>>>>>>
>>>>>> John (goad, goad) Clements
>>>>>>
>>>>>>
>>>>>> _________________________________________________
>>>>>>   For list-related administrative tasks:
>>>>>>   http://lists.racket-lang.org/listinfo/users
>>>>> ____________________
>>>>>   Racket Users list:
>>>>>   http://lists.racket-lang.org/users
>>>>
>>>>
>>>>
>>>> --
>>>> sam th
>>>> samth at ccs.neu.edu
>>> ____________________
>>>   Racket Users list:
>>>   http://lists.racket-lang.org/users

Posted on the users mailing list.