[racket] Newbie performance question

From: Richard Cole (richard.j.cole at gmail.com)
Date: Sun Oct 12 14:47:12 EDT 2014

When reading binary file formats is there a way to do this that is
performant? Can I mmap the file and then define arrays of structs that
overlay the in memory structure?

Here is a longer explanation of what I mean.

So I tried writing a program that loads in 3d models from a binary format.
The binary format has lots of arrays of structs that contains floats, and
ints and so forth. So I defined a few helper functions

(define (read-n-things in n read-fun)
  (for/vector #:length n ([i (in-range n)]) (read-fun in)))

Then I would do stuff like this

(struct mesh-vertex (v n c st tangent int-bone-index float-bone-index))
(define (read-mesh-vertex in num-uv-layers has-tangent)
  (mesh-vertex
   (read-vector-3f in)
   (read-vector-3f in)
   (read-vector-4c in)
   (read-vector-nf in (* 2 num-uv-layers))
   (if has-tangent
       (read-n-things in (* num-uv-layers 4) read-float)
       #f)
   (read-n-things in 4 read-uint16)
   (read-n-things in 4 read-float)
  ))

and read-vector-3f and friends are like this

(define (read-vector-3f in)
  (read-n-things in 3 read-float))

and read-float is like this

(define (read-float in)
  (floating-point-bytes->real (read-bytes 4 in) #f))

The thing, this ends up being pretty slow. It takes seconds to load in a
single model composed of on the order to 100k points.

Is there a way in racket to mmap the file into memory and read floats etc
at given offsets. After reading the file it needs to be transformed into a
suitable format for rendering with opengl. Right now I translate into a
second in memory format suitable for feeding to opengl. Potentially if I
can compute strides (memory distances between successive data points)
easily then I can feed the model directly into opengl without transforming.

regards,

Richard.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.racket-lang.org/users/archive/attachments/20141012/9d4d973b/attachment.html>

Posted on the users mailing list.