[plt-scheme] (fast) reading of data files into a hash-table - how?

From: Jens Axel Søgaard (jensaxel at soegaard.net)
Date: Sat Dec 31 17:24:35 EST 2005

For fun I decided to try inserting 14000 lists of 6 strings
of length 10 into the finite-map data structure provided by
Galore. And then do 10000 random lookups. On my machine I get:

 > cpu time: 1752 real time: 1762 gc time: 0
 > cpu time: 711 real time: 721 gc time: 0


(require (planet "finite-map.scm" ("soegaard" "galore.plt"))
         (lib "67.ss" "srfi")
         (lib "42.ss" "srfi")
         (lib "27.ss" "srfi"))

; random-char : -> char
;   return a random char
(define random-char
  (let* ([alphabet "abcdeghijklmnopqrstuvwxyz"]
         [len      (string-length alphabet)])
    (lambda ()
      (string-ref alphabet (random len)))))

; random-string : natural -> string
;   return a random string of length n
(define (random-string n)
  (string-ec (: i n) (random-char)))

; random-strings : natural -> (list string)
;   return a random list of length m of strings of length n
(define (random-strings m n)
  (list-ec (: i m) (random-string n)))

; 14000 lists of 6 strings of length 10
(define lists-of-strings
  (list-ec (: i 14000) (random-strings 6 10)))

; 10000 random keys for lookup timing
(define random-keys
  (list-ec (: i 10000)
           (list-ref lists-of-strings
                     (random (length lists-of-strings)))))

(collect-garbage)
(define finite-map
  ; make a finite map, with the above random keys,
  ; use (random 10) to generate the associated values
  (time (foldl (lambda (s fm)
                 (insert s (random 10) fm))
               (empty)
               lists-of-strings)))

(collect-garbage)
; time 10000 random lookups
(time (for-each (lambda (key)
                  (get key finite-map))
        random-keys))

Happy New Year!
Jens Axel





Posted on the users mailing list.