<div class="gmail_quote">On Mon, Dec 7, 2009 at 1:08 PM, Eli Barzilay <<a href="mailto:eli@barzilay.org">eli@barzilay.org</a>> wrote: <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> <div class="im">On Dec 7, James Swaine wrote: > I'm pleased to announce the initial release of parallel futures, a > construct for fine-grained parallelism in PLT. </div>Random notes: 1. Your message mentioned requiring `scheme/futures' -- it should be `scheme/future'. </blockquote><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> 2. Is it possible to move `processor-count' into the core and make it available for non-future builds too? That function is useful even without futures (for example, for code that runs subprocesses for all cores -- in my scripts I just read a from from /proc, and having a portable replacement would be nice). For the sake of code that wants to use futures, I think that the current function is better named `futures-count'. </blockquote><div> This sounds fine to me. processor-count should always have the same behavior, regardless of whether futures are enabled. But I'm not sure why you would need a separate function for futures-count, unless you're intending it to be related to question (6) below? </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> 3. Can you please include the command line for building with futures support? This way, you can have it show both `--enable-futures' and `--disable-mred' to avoid confusion. </blockquote><div> Yes, will do. </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> 4. You should also mention what Windows users should do, even if it's "not use futures", since on windows there is no configure step. 5. The term `unsafe' was used with a particular meaning by the foreign interface, then came the scheme/unsafe/ops library with a slightly different but related meaning. The futures documentation uses `unsafe' with a completely different meaning -- so it would be better to find some other term, or at least make that very clear. (This applies in general too -- for example, in your reply to Jay you mention "unsafe primitives"... A question about the confusion with `scheme/unsafe/ops' or `scheme/foreign' would also be difficult to answer when the only term you have is "unsafe".) </blockquote><div> Good point - perhaps it'd be best to use other terminology to avoid confusion here. </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> 6. It would be nice to be able to get a list of the currently running futures? (With threads, this is possible through custodians -- with futures being global this won't make much sense.) 7. There should be some way to disable futures. I'm thinking of situations like running sandboxed code, where you don't want untrusted code to be able to grab cpus. Of course it would be nice to be able to have a finer control over it (like how many cores each future sees) -- but with nested futures being forbidden there doesn't seem to be much point in that. 8. Looks like there is no way to kill a running future. There should be one. To make it practical, it should also be killed on a custodian shutdown -- but again, it's not clear to me how that might look when futures are inherently global to the process. 9. Worse, it looks like when a future is touched, it's not really joining the main execution thread. For example, when I run your suggested example: <div class="im"> (for-each touch (for/list ([i (in-range 0 (processor-count))]) (future loop))) </div> I get my 4 cores spinning away happily, but hitting break leaves them *all* running. I'd expect the first future to stop as a result. And with future values giving you only the ability to touch, this means that your example is a way to render your mzscheme process useless until it exits. 10. (This will be a long one.) The "unsafe" primitives make some sense as far as using them. (For example, it's clear that using any IO will suspend the future.) But there is no mention at all of what happens when there are potential race conditions. Is writing to memory going to block a future in any way? How is reading handled? Since this is not documented, I ran some experiments. (My machine has two double core Xeons cpus at 3ghz, in case it matters.) First, I did this: #lang scheme (require scheme/future) (define x (let ([v (vector 0 0 0 0)]) (define ((spin n)) (let loop () (vector-set! v n (add1 (vector-ref v n))) (loop))) (cons v (map (compose future spin) '(0 1 2 3))))) (sleep 10) x (exit) and the printout had similar numbers, between 53M and 58M. Realizing that there's nothing useful I can do with the future values, I changed it to: (define x (let ([v (vector 0 0 0 0)]) (define ((spin n)) (let loop () (vector-set! v n (add1 (vector-ref v n))) (loop))) (for-each (compose future spin) '(0 1 2 3)) v)) and this produced very different numbers, with one counter being at around 110M, two more around 50-60M, and one around 10M. The high one was usually the first, and the low one was usually the last. I couldn't really attribute this to delays in firing off the futures, because one of the results was #(74M 49M 98M 10M). Next I tried to rephrase it, to avoid delays in starting them as soon as possible: (define x (let ([v (vector 0 0 0 0)]) (define (spin n) (future (lambda () (let loop () (vector-set! v n (add1 (vector-ref v n))) (loop))))) (for-each spin '(0 1 2 3)) v)) and the results went back to all being around 60M. The *sum* of all counters is roughly the same (around 230M), but I can't think of any way the previous example would have such huge differences. Next, I tried this: (define x (let ([v (vector 0 0)]) (define (spin n) (future (lambda () (let loop () (vector-set! v n (add1 (vector-ref v n))) (loop))))) (for-each spin '(0 1 1 1)) v)) and the counts were 63M and 77M resp. This *looks* like reading and writing from the same memory location is "safe", and you get the usual races -- so the three cores that use the second counter keep running into races of "read N, write N+1 after another thread wrote N+1 or more", resulting in races mostly making the counts the same, but with some "success rate" that leads to a higher number. If this is correct, and together with the fact that the usual synchronization primitives in mzscheme are useless in futures (since they seem to be "unsafe"), then this is a pretty important thing to describe in the docs. Without being able to sync, then people should be told very explicitly that the system avoids race conditions that violate the mzscheme vm, but it knows nothing about your own races, which means that you should be extra careful to use futures for "very" independent tasks. </blockquote><div> An excellent point - I'll update the documentation to include something to this effect. We've had several discussions about adding concurrency primitives with futures - but you're right, as of now you're restricted to independent tasks because there really isn't an effective way to do inter-thread communication. </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: <a href="http://barzilay.org/" target="_blank">http://barzilay.org/</a> Maze is Life! </blockquote></div>