[racket] Remote execution in Racket

From: Frederick Ross (madhadron at gmail.com)
Date: Fri Aug 20 04:52:49 EDT 2010

I find myself with a rather peculiar problem, and I'd love anyone
else's input on it.

I'm writing a piece of software for our core facility.  Among the
things it does is wrap up a bunch of unix commands into one atomic
unit, handle their inputs and outputs, and keep track of all the files
and commands involved in an SQLite database.  The thing about
bioinformatics is that a few of the operations (okay, one: sequence
alignment) are computationally hard.  It's the only reason anyone
thinks the field isn't trivial.  So we have a big Linux cluster next
door running a batch submission system called LSF.  You basically give
it arguments the way you would an exec call, but instead of running it
locally, it queues it on the cluster, runs it, does some tracking of
it, and sends you an email when it's done.

I want to take the local executions I'm doing and be able to submit
them as batch jobs to the cluster instead.  My initial idea was along
these lines:

1. Create a temporary directory (the execution will have one anyway,
just use that).  The scratch space where I would do this is shared
across the cluster.
3. Executions are wrapped in little language in my system, so have the
macro instead create a uniquely named function to postpone the
execution.
2. Dump an image of the current session into the temporary directory.
3. Submit a batch job to start Racket, load the dumped image, and run
the postponed function.

This is wasteful, but the waste is negligible compared to the size of
the computing task that would usually be submitted this way.

Unfortunately, I can't figure out how to dump images from Racket.  Is
it possible?  Or am I having brain creep from Common Lisp?  Or if
anyone can think of a cleaner way to do this, I would really love to
hear it.

-- 
Frederick Ross
Bioinformatics and Biostatistics Core Facility
Life Sciences, EPFL
http://bbcf.epfl.ch/
+41 21 693 14 39


Posted on the users mailing list.