[plt-scheme] lex problem
Stephen Bloch wrote:
> I'm trying to teach myself to use the scanner and parser tools in time
> to provide my students a parser for a C-like language so they can
> write an interpreter for it.
>
> I'm calling
> (lexer (some-ws (get-token input-port))
> ((eof) 'eof)
> ...
> (#\+ (token-plus-operator))
> ...
> ((concatenation #\" (repetition 0 +inf.0 (complement #\")) #\")
> (token-str-lit-token (chop-fl lexeme)))
> )
>
> So this should recognize anything that starts with a quotation mark,
> ends with a quotation mark, and doesn't contain any other quotation
> marks, as a "str-lit-token".
>
> I call the lexer on the input "\"hello\" + \"there\""
> and it gives me back a single str-lit-token with value "hello\" +
> \"there"
> rather than (as I expected) a str-lit-token, a plus-operator, and
> another str-lit-token.
>
> What gives?
>
Sort of a wild guess but does this work instead?
replace (complement #\") with
(intersection (complement #\") any-char)