[racket] parallel-map, for/parallel etc.

Hey there!

I have a half-baked implementation of something like this on PLaneT, but
it doesn't use places or any of the nice things that Racket has. You can
replace for/list loops with for/work loops and, under some (major)
constraints, have an instant networked mapreduce setup.

http://planet.racket-lang.org/display.ss?package=riot.plt&owner=gcr

To try this out, open one terminal and run:
    racket -p gcr/riot/server
This starts a "tracker" which coordinates all your computing resources,
matching the programs that need results to the programs that compute them.

Then, save the following to simple-demo.rkt:

;; -------------------------
#lang racket
(require (planet gcr/riot))
(define (run)
  (for/work ([i (in-range 10)])
    (sleep 3) ; or some big task
    (* 10 i)))
(module+ main
  (connect-to-riot-server! "localhost")
  (displayln (run))
  ;; Note that (run) must be a separate function
  (displayln "Complete"))
;; ---------------------------------

Open a few more terminals, and in the same folder that contains
simple-demo.rkt, run a few instances of:
    racket -p gcr/riot/worker -- localhost
Now you can run simple-demo.rkt, which will register ten units of work
that will be coordinated by the tracker and fulfilled by the workers.
Note that the more workers you add, the faster the task completes.

You can also add workers on other machines if you copy simple-demo.rkt
there.

Docs: http://planet.racket-lang.org/package-source/gcr/riot.plt/1/1/planet-docs/riot/index.html

It has limitations (eg. all results kept in memory, you can only refer
to serializable structures, and your for/work loops must be in separate
functions) but it's quite fun to play with.

Harry Spier <vasishtha.spier at gmail.com> writes:
> This post from May 2011 says:
> "parallel-for-each, parallel-sort, parallel-filter, for/parallel and the
> likes for sequences. ... are next on our radar."
>
> Are there still plans to develop these parallel processing functions.
>
> Thanks,
> Harry Spier
> ____________________
>   Racket Users list:
>   http://lists.racket-lang.org/users