[racket] Remote execution in Racket

You may wish to consider an alternative architecture. 
As someone who has used dumps, starting in a 1980 Prolog, 
I think it's a nice thing but highly overrated. We have 
several people on this list who distribute highly intensive
comps on systems, and they are all experimenting with different 
ways of getting it done. Perhaps some of them will pipe in 
-- Matthias



On Aug 20, 2010, at 10:20 AM, Jay McCarthy wrote:

> There's no way to dump or load images in Racket. You'll have to
> implement state dumping/loading in your application.
> 
> Jay
> 
> On Fri, Aug 20, 2010 at 2:52 AM, Frederick Ross <madhadron at gmail.com> wrote:
>> I find myself with a rather peculiar problem, and I'd love anyone
>> else's input on it.
>> 
>> I'm writing a piece of software for our core facility.  Among the
>> things it does is wrap up a bunch of unix commands into one atomic
>> unit, handle their inputs and outputs, and keep track of all the files
>> and commands involved in an SQLite database.  The thing about
>> bioinformatics is that a few of the operations (okay, one: sequence
>> alignment) are computationally hard.  It's the only reason anyone
>> thinks the field isn't trivial.  So we have a big Linux cluster next
>> door running a batch submission system called LSF.  You basically give
>> it arguments the way you would an exec call, but instead of running it
>> locally, it queues it on the cluster, runs it, does some tracking of
>> it, and sends you an email when it's done.
>> 
>> I want to take the local executions I'm doing and be able to submit
>> them as batch jobs to the cluster instead.  My initial idea was along
>> these lines:
>> 
>> 1. Create a temporary directory (the execution will have one anyway,
>> just use that).  The scratch space where I would do this is shared
>> across the cluster.
>> 3. Executions are wrapped in little language in my system, so have the
>> macro instead create a uniquely named function to postpone the
>> execution.
>> 2. Dump an image of the current session into the temporary directory.
>> 3. Submit a batch job to start Racket, load the dumped image, and run
>> the postponed function.
>> 
>> This is wasteful, but the waste is negligible compared to the size of
>> the computing task that would usually be submitted this way.
>> 
>> Unfortunately, I can't figure out how to dump images from Racket.  Is
>> it possible?  Or am I having brain creep from Common Lisp?  Or if
>> anyone can think of a cleaner way to do this, I would really love to
>> hear it.
>> 
>> --
>> Frederick Ross
>> Bioinformatics and Biostatistics Core Facility
>> Life Sciences, EPFL
>> http://bbcf.epfl.ch/
>> +41 21 693 14 39
>> _________________________________________________
>>  For list-related administrative tasks:
>>  http://lists.racket-lang.org/listinfo/users
>> 
> 
> 
> 
> -- 
> Jay McCarthy <jay at cs.byu.edu>
> Assistant Professor / Brigham Young University
> http://teammccarthy.org/jay
> 
> "The glory of God is Intelligence" - D&C 93
> _________________________________________________
>  For list-related administrative tasks:
>  http://lists.racket-lang.org/listinfo/users