[plt-scheme] graphs / shared / memory!!

From: Eli Barzilay (eli at barzilay.org)
Date: Thu Oct 20 22:40:35 EDT 2005

On Oct 20, Yoav Goldberg wrote:
> > >> Have you actually measured the memory consumption disadvantaged before
> > >> you jump to this conclusion?
> Ok. I stand corrected.
> 
> My little benchmark[*] proved without a doubt that lists are far
> more efficient -- about 9mb mem usage for mzscheme using lists,
> while the streams version got to about 150mb mem usage (and run for
> quite a while!) when I killed the proccess.

Like Matthias said, using streams can make it tricky to debug memory
consumption, but the program that you wrote makes it easy to see the
problem: you're generating huge streams and holding them all in memory
until the end of the code -- this means that you're paying the price
of building promises, thunks, etc, all for using them as if they're
lists, so it's no wonder lists are much lighter.

Streams would really shine in a situation where you *don't* need to
use more than some small portion of it at any time.  For example, if
you want to play some list over and over, you write the code that
turns the list into an infinitely repeating stream, and then play that
stream -- but you do that while being careful not to hold a reference
to the beginning of the stream since that means that you accumulate
more memory.

> (I wonder how Python's generators will match up.. I might write a
> python version tomorrow..)

The Python approach of using iterators is also similar to infinite
streams, except that you have `processes' instead of streams.  The
iterator macros in Swindle can be used for that -- and that included
in the sample code I sent you a while ago, after you first had notes
play.

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                  http://www.barzilay.org/                 Maze is Life!


Posted on the users mailing list.