[plt-scheme] (fast) reading of data files into a hash-table - how?

From: Jens Axel Søgaard (jensaxel at soegaard.net)
Date: Sat Dec 31 15:30:51 EST 2005

Hi Yoav,

>I need to load a large amount of precomputed data.
>It is bassically a list of key:value pairs, where each key is a list
>of symbols and each value is a number. I have about 14000 such keys.
>
>I tried to just read the data line by line and put it into a
>hash-table, but it turns out creating a hash-table with 'equal, and
>then filling it with 14000 values takes way to much processing time.
>Is there any way to speed things up?
>  
>
Chongkai's suggestion was quite clever. If that approach doesn't work 
consider
either serializing the entire hash table to disk, so you don't have to 
rehash it each time,
or try another data structure - e.g. a trie. What kind of data is in 
your lists? Do you
have a total order on your keys?

-- 
Jens Axel Søgaard




Posted on the users mailing list.