[racket] Racket Virtual Machine runs out of memory

Thank you Pierpaolo I am very interested in seeing your boolean vectors library.
Thank you Jon for the suggestion of bitwise operations.
Danny I'll look at Racket-bitsyntax.
Thank you Eli for the suggestion of using a big byte-string.
And  thank you Matthias for showing how to ensure the GC .

Cheers,
Harry Spier

On Fri, Jul 20, 2012 at 4:14 PM, Pierpaolo Bernardi <olopierpa at gmail.com> wrote:
> Btw, I have a boolean vectors library that you can use as is or as a
> starting point. I cannot send it to you until monday, but you can find
> an old copy in the archives of this ml, googling for my name + boolean
> vector.
>
>
>
> 2012/7/20, Harry Spier <vasishtha.spier at gmail.com>:
>> Thanks Eli,
>>
>> I probably have to go the route you and others suggest.  But I think I
>> still have a problem.  Even the single operation of MagickExportPIxels
>> to export the pixel data of this page to manipulate fills at least 4/7
>> of the memory before failure.  And there is no guarantee that pages
>> wont have more pixel data.  They probably will.  And as Matthias says
>> the list doesn't seem "that large".
>>
>> Also I have 4 or 5 transformation stages of the pixel data.  Will
>> putting in Garbage Collection commands to get rid of transformations
>> of the data I've already used help?
>>
>> What sets the amount of memory the Racket virtual machine uses.  Is it
>> a Racket parameter, Is it a function of the amount of RAM in the
>> machine?  Is this more a Windows problem and will switching to Linux
>> help etc.?
>>
>> Harry Spier
>>
>> On Fri, Jul 20, 2012 at 1:09 PM, Eli Barzilay <eli at barzilay.org> wrote:
>>> 10 minutes ago, Harry Spier wrote:
>>>> #lang racket
>>>> (define l (time (build-list (* 7091 5023) (λ (x) 1))))
>>>> (system "PAUSE")
>>>>
>>>> ABORTS with Racket Virtual Machine run out of memory
>>>
>>> IME, the exact size where things fail is not important -- if you're
>>> getting anywhere close to it, then you should revise the code to use
>>> less memory.
>>>
>>> There was the option that was raised for using integers, which might
>>> be inconvenient -- even with one (huge) integer for each row.
>>> Instead, I think that it would be convenient to use one big byte
>>> string for the whole array, and write some accessor functions to
>>> address the contents as a matrix.  The exact format of the byte string
>>> can be one byte per 1/0 pixel or even more compactly, one bit per
>>> pixel.  The choice should depend on whatever libraries you're using
>>> with the same data, to minimize translation work.  (Dealing with bits
>>> will make the access code a bit more complicated.)
>>>
>>> Currently, you're using one cons cell for each number, which is
>>> probably somewhere around 3 pointers -- which is about 12 bytes per
>>> bit.  So just a one byte for each number would be a 12x factor, with
>>> one bit per 1/0, you're at ~100x saving, which would be significant
>>> enough to reduce other processing times.
>>>
>>> --
>>>           ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
>>>                     http://barzilay.org/                   Maze is Life!
>>
>> ____________________
>>   Racket Users list:
>>   http://lists.racket-lang.org/users
>>
>
> --
> Inviato dal mio dispositivo mobile