[plt-scheme] lex problem

From: Stephen Bloch (sbloch at adelphi.edu)
Date: Tue Sep 29 12:11:11 EDT 2009

I'm trying to teach myself to use the scanner and parser tools in  
time to provide my students a parser for a C-like language so they  
can write an interpreter for it.

I'm calling
(lexer (some-ws (get-token input-port))
          ((eof) 'eof)
          (#\+ (token-plus-operator))
          ((concatenation #\" (repetition 0 +inf.0 (complement #\")) # 
           (token-str-lit-token (chop-fl lexeme)))

So this should recognize anything that starts with a quotation mark,  
ends with a quotation mark, and doesn't contain any other quotation  
marks, as a "str-lit-token".

I call the lexer on the input "\"hello\" + \"there\""
and it gives me back a single str-lit-token with value "hello\" +  
rather than (as I expected) a str-lit-token, a plus-operator, and  
another str-lit-token.

What gives?

Stephen Bloch
sbloch at adelphi.edu

Posted on the users mailing list.