[plt-scheme] More PLT Scheme as an Alternative to Matlab

On Aug 13, Doug Williams wrote:
> 
> For the PLT implementors. I'm using SRFI 4 as the underlying
> representation for storing the arrays. They store the data in the
> formats we want, but don't seem to be processed efficiently.
> Although efficiency isn't a real concern at this stage, it will be
> for some people at some point. Any idea of what we can do better is
> more than welcome.

The srfi-4 representation can be more efficient in some cases, and
less in other cases...  The basic idea is that the vector is saved
directly as the C vector, so memory is more efficient than plain
Scheme values.  For example, a floating point number in Scheme is
(usually) a boxed object, so the machine representation is a pointer
to the value.  Here's a quick list of things that are relevant.

* If you're dealing with external code a lot (for example, and
  interface to some algebra library), then you save a lot of
  translation overhead since the data is (usually) in a format that
  the external library can use as is.

* However, if you're doing a lot of work *inside* Scheme, then things
  can become very inefficient.  For example, say that you want to loop
  over a floating point array and multiply all of the values by a
  given factor.  What will happen in this case is: each time you read
  a value, you allocate a Scheme object for the fp number, multiplying
  that allocates a new one, then saving it copies the result back to
  the array, leaving the two allocated values to be GCed.  (There are
  some JIT-level optimizations for fp numbers, I don't know if it can
  save one of these allocations in this case.)

  So the bottom line is: using the srfi-4 representation can work well
  if you usually operate on the data as an opaque blob of values, and
  only get back the Scheme values at the end.

* Another issue here is with mutation -- if the external call changes
  values, then the changes will be visible on the Scheme side with no
  further work.

* However, you should be careful as usual with C code:
  - If you deal with a library that can allocate such a vector and
    hand it back to you -- then this memory will not be visible to the
    GC and will need an explicit `free'.
  - srfi-4 vectors are allocated as usual, which means that the 3m GC
    will move them around.  This can be important if the C code might
    retain a pointer to such a vector -- when a later call to the C
    code is done, the block is likely to be in a different place, with
    the usual C response (segfault in the best case, complete mess in
    the worst case).

* Another point to consider is huge arrays -- if you expect to have
  such arrays, then it might be better to do the allocation explicitly
  outside of the GC -- using `malloc' in 'raw mode.  This will require
  registering finalizers to free them, but the benefits are:
  - You can have a huge array without worrying about the 3m GC that
    usually requires double the space that you're using.
  - Dealing with an external library that allocates an array becomes
    easy, since it's dealt with in the same way.
  - No issues with arrays moving around, for external code that keeps
    pointers to it.  (I can imagine this being a real benefit if there
    are libraries that can use multiple cores and calling a callback
    function when the operation is done.)

* Finally, it sounds like it might be more convenient to use the
  `cvector' representation from the foreign interface: this is very
  similar to the srfi-4 representation, except that the type is stored
  in the cvector rather than having a separate type for each kind of
  vector.

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!