[racket] Remote execution in Racket

On Fri, Aug 20, 2010 at 8:04 PM, YC <yinso.chen at gmail.com> wrote:
> It seems like all you need is a batch architecture in racket, is that
> correct?  I am assuming the following:

Not correct.  Thanks to all those who answered with ideas, and they're
all very good ideas.  Dumping images and loading them remotely is a
dirty, ugly hack.  I admit it openly and freely, and would never
consider it if I weren't *severely* constrained by the cluster this
has to run on.

To give you some idea:

I can't start servers for dispatch on the various compute nodes.  It's
not a question of policy, it's a question of being impossible.  The
admins can't even set new iptables rules on all the compute nodes of
the cluster, which I also asked them to do at one point for the
purpose of secure database access.

When I use the batch system, I really do only get some criteria for
acceptable hosts to run on and a set of command lines arguments to be
exec'd.

There is really no way to get any kind of callback from the batch
execution system.  It doesn't even pass signals on in any kind of
meaningful way.

Creating new batch executions should be doable from a REPL.

The Racket system doing dispatch in this way is the *second* choice.
The first choice was a system put together in Haskell, but it is
essentially impossible to get the Glasgow Haskell Compiler to run on
the cluster.  I lost a lot of time on that one.  Nor can I develop
static binaries elsewhere and drop them on the cluster, because the
batch system's development headers are proprietary and only on the
cluster.

-- 
Frederick Ross
Bioinformatics and Biostatistics Core Facility
Life Sciences, EPFL
http://bbcf.epfl.ch/
+41 21 693 14 39