[racket] Is there a technical or social reason that parser-tools/lexer doesn't have an "else"?

From: Jon Zeppieri (zeppieri at gmail.com)
Date: Mon May 21 20:20:01 EDT 2012

  On Mon, May 21, 2012 at 7:54 PM, Danny Yoo <dyoo at hashcollision.org> wrote:
> I've been frustrated with trying to compose lexers together.  The
> situation is that I'd like to define a lexer that knows how to give up
> gracefully, and to give another lexer a go at processing a port.
> Ideally, I'd like to say something like this:
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> (define lexer-1
>  (lexer ...
>           [else (lexer-2 ip)]))
> (define lexer-2
>  (lexer ...
>           [else (error 'i-dunno)]))
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

parser-tools/lex-sre has a sequence regular expression (:), which can
match an empty sequence. (It does warn you that it accepts an empty
sequence, since it's guaranteed to succeed, but it doesn't consume

#lang racket
(require parser-tools/lex
         (prefix-in : parser-tools/lex-sre))

(define lexer1
   ["a"  lexeme]
   [(::) (lexer2 input-port)]))

(define lexer2
   ["b" lexeme]))

(define (test1)
  (define in (open-input-string "ab"))
  (values (lexer1 in)
          (lexer1 in)))

(define (test2)
  (define in (open-input-string "b"))
  (lexer1 in))

Posted on the users mailing list.