[racket] Performance help

On Jan 2, 2015, at 11:38 AM, Jens Axel Søgaard <jensaxel at soegaard.net> wrote:

> Another trick: the regular expression matchers in Racket works both
> on strings and ports, so  train  can be written
> 
>    (define (train fname)
>       (freqs (words (open-input-file fname))))
> 
> I am not sure whether it will give a speedup though.


Dang, I started this and forgot to follow through. 

I hadn't thought of the regexp-split step though
I suspect downcasing is still needed for normalization. 

-- Matthias