[plt-scheme] Compression dictionary

2009/10/5 Eli Barzilay <eli at barzilay.org>:
> Does anyone know of a good and simple method for building a dictionary
> for compression?

> Explanation: The documentation index file was ~2mb initially, and now
> it's up to 3mb.  In addition, some thing I did for compression make
> loading it slower (like nested arrays which are used like Sexprs) so
> I'm revising the whole thing.

> Example>

> "foo_bar"
>  "meh_blah_foo_blah"

I understand the tokens are "foo", "bar", "meh", and, "blah".
How many bytes do you need to store the tokens alone?
How many different tokens are there?

-- 
Jens Axel Søgaard