[racket] remote tasks

HP Wei wrote at 05/13/2012 03:53 PM:
> Suppose I am on a master machine A
> and there are two other machines B and C.
>
> On A,  in racket,  I would like to programatically initiate
> one server on B and another on C.
> The 'server' is presumably a 'repl' that can execute a block
> of code, sent from A.
> [ Let's say security is not an issue here. ]
>
> i.e. this is the intention:
>
>     invoke-server-on B  and C  (via rsh or ssh)
>     send-code B   (asynchronously)
>     send-code C   (asynchronously)
>     wait-for-result-from B and C
>     ...
>     kill-server-on B and C

How about the following?

* Master machine has a thread for each slave machine.

* Each slave machine handling thread in the master does the following: 
first uses the "subprocess" procedure (or similar) to SSH (or "rsh") to 
the slave machine and start the slave program, keeps that connection 
open, and then runs in a loop, writing Racket expressions to stdin of 
that remote process over SSH, and reading results from the stdout of 
that process.

* The slave program on each machine can read Racket expressions from 
stdin and write the results to stdout in a format readable by the "read" 
procedure.  You might be able to do this in a single thread.

* Regarding killing the slave process, you can send an expression that 
tells a slave to die, and/or you can have the process die when you kill 
the SSH connection.

That's a very simple setup that might do everything you need, and it 
could be implemented from this outline by a second-year CS student.

If, however, you also want to do things like have the slave signal any 
exceptions back to the master, permit the master to interrupt a slave, 
take advantage of custodians for better resilience, have work queues, 
etc., then there would be additional work you have to do.  All the 
details and subtleties of doing this well can't practically be covered 
in an off-the-cuff email list post (considering that highly-skilled 
people make their livelihoods doing such things well).

BTW, if you use SSH, you'll want to verify that small payloads get 
padded so that they move promptly even if there is nothing else to 
send.  Offhand, I don't recall the SSH way to do that, but you should 
look it up, so that you're not increasing latency or even intermittently 
deadlocking a slave connection.

Neil V.