[racket] remote tasks

On 05/15/2012 09:37 AM, Eli Barzilay wrote:
> Yesterday, Kevin Tew wrote:
>> Attached is a distributed places program that will do what you want.
>>
>> It requires the latest checkout from git head.  You must have ssh
>> public-key authentication setup on all the nodes.  For easy use, it
>> also requires that racket and remote-eval.rkt be installed in the
>> same place on all three machines.  It communicates with plain
>> sockets, so it assumes a secure environment.
>>
>> Let me know what problems you have or if it works for you.
>> I would start by testing it out using just localhost.
> Looks like there's a good bit of magic is done in the library or
> possibly in the `define-named-remote-server' macro -- so it seems that
> it would be good to add this example with lots of explaining comments
> into the docs.  BTW, the description of `define-named-remote-server'
> looks very confusing.  My guess that it might do some work that is
> worth describing is based on the fact that it seems that it is what
> defines the `echo-server-evalit' and `make-echo-server' --?
I'll add more detail to the reference documentation

>
> More issues:
>
> * When I ran this for the first time, I got
>
>      tcp-listen: listen on 6340 failed (Address already in use; errno=98)
>
>    and it was stuck.  (And no other process is using that port.)  When
>    I killed it (with just C-c, in an Emacs shell), it left a couple of
>    processes:
>
>    24863 eli 0.0 0.0    784 S ? 00:00:00 /usr/bin/ssh localhost /var/tmp/racket/bin/racket -tm /var/tmp/racket/collects/racket/place/distributed/launch.r
>    24871 eli 0.0 0.0 139740 S ? 00:00:00 /var/tmp/racket/bin/racket -tm /var/tmp/racket/collects/racket/place/distributed/launch.rkt spawn 6340
>
>    I manually killed the second and the first disappeared too.
>
>    The "/var/tmp" path means that this was started from the nightly
>    build -- is there some test that uses that port?  If so, then it's a
>    bad idea to risk leaving a running process and worse to have it open
>    a port -- can you add code that checks that the process is dead and
>    kill it if it isn't?
There were some tests that used to run nightly, but they have been 
disabled for a while.
This shouldn't be a problem anymore for nightly builds.
>
> * Openning a tcp port for each node seems pretty bad in general -- if
>    it's ssh-ing to the other machine, why not use the standard IO ports
>    and avoid the potential problems?  (Eg, for the build script I would
>    never use a system that just opens a random port -- even if all of
>    the machines are protected by a firewall, a stuck build process
>    would be very problematic.)
The distributed system, by default, builds a complete graph between the 
nodes.
The star network of ssh connections is not what I wanted to implement.
>
> * The hello-world example at the top of the doc page could also use
>    commentage.  More than that, it's missing some documentation that is
>    at a level that is a little higher -- I can't make things out in the
>    current text.
Noted, I'll add more detail.
>
> * The documentation says: "The same user account is used across all
>    nodes in the distributed network" -- is that really needed?  If it
>    uses ssh to connect, then "ssh foo" could use a different username
>    if my .ssh/config sets that up, and in addition is there any problem
>    with "ssh foo at bar"?
It will use a different username if your .ssh/config sets that up.
It doesn't support a "user at host" hostname string, so that won't work.

>
> * Later it says: "All machines run the same version of Racket" -- if
>    this is required because it uses zo to pass around data, then it's
>    better to say that explicitly.  (Otherwise it's not clear: should it
>    be exactly the same racket build for the same platform on both
>    machines?  Maybe it should only have the same functionality wrt the
>    places libraries?)
I'm being conservative for now.  Currently, serialization
just uses read and write.
>
> * Would it work on all platforms?  If it runs ssh, is there a way to
>    specify an alternative executable name?
Yes, there is a #:ssh-bin-path keyword argument.
>
> * Sidenote: you should use @filepath{.ssh/config} and
>    @filepath{.ssh/authorized_keys} to typeset them as paths, and
>    probably worth mentioning something about other possible
>    architectures (maybe on OSX it's in some ~/Library?).
I've added @filepath usage
>
> * Another side-note: you have a few things like
>
>      @(define place-thunk-function
>          (make-splice
>           (list
>            @p{
>      ...some text...
>            })) )
>
>    There's no need for that -- you can indent the text to make the
>    scribble source more readable.  (That's one of the advantages of the
>    @-forms.)
FIXED
>