[racket-dev] Reading and writing binary data

30 minutes ago, Tobias Hammer wrote:
> 
> after reading this thread on make-sized-byte-string to convert an
> arbitrary cpointer very efficiently into a byte string

"Efficiently" is not really the right word here -- it basically
creates a byte string using a pointer to an existing block of memory,
which means that you can deal with it directly in racket code.


> i came up with the idea to use it for writing structured binary data
> to a file:
> 
> (define-cstruct _S ([a _int]))
> (define s (make-S 3210))
> 
> (with-output-to-file "test.bin" #:mode 'binary #:exists 'replace
>    (lambda ()
>      (display (make-sized-byte-string s (ctype-sizeof _S)))))
> 
> Like this you can write large chunks of binary data very fast.

Two things:

1. Note that a file that you create in this way is not portable -- it
   can depend on platform specific things like endiannes and padding.


> Where i am struggling is how to read it in again. My current
> approach reads it in as a byte string and copies every byte into the
> c-struct.  Not very efficient and not very elegant:

You can use `cast':

  #lang racket
  (require ffi/unsafe)
  (define-cstruct _S ([a _int] [b _byte] [c _int]))
  (with-output-to-file "test.bin" #:exists 'replace
    (lambda ()
      (write-bytes (make-sized-byte-string (make-S 6543210 4 24680)
                                           (ctype-sizeof _S)))
      (void)))
  (let ([p (cast (file->bytes "test.bin") _bytes _S-pointer)])
    (list (S-a p) (S-b p) (S-c p)))


One last note -- if you just want to dump some data in a binary format
that is more efficient than text, then you can use the `fasl' library,
for example:

  (require racket/fasl)
  (s-exp->fasl '(6543210 4 24680))

gives you a bytestring with a binary representation of any racket
data.  It obviously won't be as efficient the above representation,
but should be much closer to it than using text.  If you want to use
this for large data files, then it's worth trying both and timin them.
Keep in mind that the main cost of fasl files is when you read them
but then it's just plain racket data; in contrast, with an ffi struct
you pay the price when you access the data.

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!