<html> <head> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <div class="moz-text-html" lang="x-western"> On 03/29/2012 12:00 PM, Nick Shelley wrote: <blockquote cite="mid:CAPrQK3DZ501QHOoNtmBp2QuSq+zZoSZfV_pTqp4=d8NAhtcO6g@mail.gmail.com" type="cite">In my limited experience with parallel computing, it seems like the master-worker paradigm is somewhat common. However, it seems like Racket's places (or at least the way events are done with place-channels) makes this inconvenient. Since there is nothing that I can find in the result of syncing on a place-channel that would allow me to then send some new work to the same channel, I have to implement this manually by having each channel keep track of it's place in the list of workers and send that information explicitly. This approach seems hacky and more prone to bugs.</blockquote> <br> You need to use wrap-evt or handle-evt with a closure that includes the channel.<br> <blockquote cite="mid:CAPrQK3DZ501QHOoNtmBp2QuSq+zZoSZfV_pTqp4=d8NAhtcO6g@mail.gmail.com" type="cite"> <div> <br> </div> <div>I think it would make more sense for the result of syncing on a place-channel to return the channel itself as a result. This would make an explicit call to place-channel-get necessary, which may be a downside, but the upside is I could then put something on the same channel or do whatever else I may want with it.</div> </blockquote> This is one way to do it, however we choose to make place-channels work like plain channels in Racket.<br> Its a trade off.  Either you have to explicitly call place-channel-get or you have to use wrap-evt or handle-evt to maintain a reference to the channel.<br> <blockquote cite="mid:CAPrQK3DZ501QHOoNtmBp2QuSq+zZoSZfV_pTqp4=d8NAhtcO6g@mail.gmail.com" type="cite"> <div><br> </div> <div>If changing how place-channel events are treated isn't feasible, I think it would at least be useful to provide an abstraction that makes master-worker more convenient. One idea might be a sync-channel function that returns the channel.</div> </blockquote> wrap-evt and handle-evt are those abstractions.<br> <br> Place-channel is meant to be a simple primitive, enabling more complicated parallel constructs to be built on top of place-channels.<br> We don't yet have a master-worker library. It would be nice if we did.<br> <br> Some example you can look at include:<br> The places paper at <a class="moz-txt-link-freetext" href="http://www.cs.utah.edu/plt/publications/dls11-tsffd.pdf">http://www.cs.utah.edu/plt/publications/dls11-tsffd.pdf</a>, demonstrates a master-worker parallel build using a jobqueue and handle-evt.<br> The file collects/setup/parallel-do.rkt is a very complicated example of a master-worker implementation using work-queues and wrap-evt<br> <br> <blockquote cite="mid:CAPrQK3DZ501QHOoNtmBp2QuSq+zZoSZfV_pTqp4=d8NAhtcO6g@mail.gmail.com" type="cite"> <div>As a sort of side note, it would be nice to be able to treat this problem similar to a user thread problem where you can conceptually imagine having infinite workers and just giving one chunk of work to each of them. I tried to do this at first, but because of the way places are implemented, I ran out of file descriptors relatively quickly (and the overhead of starting a new VM for each chunk of work might have been too much anyway, I don't know). I don't know if an abstraction like that is possible or useful in general, but it may be something to consider.</div> <div><br> </div> </blockquote> Places are too resource expensive to spawn one for each work item.  You have to use a work-queue and a small number of worker places.<br> <br> Kevin<br> <blockquote cite="mid:CAPrQK3DZ501QHOoNtmBp2QuSq+zZoSZfV_pTqp4=d8NAhtcO6g@mail.gmail.com" type="cite"> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_________________________ Racket Developers list: <a class="moz-txt-link-freetext" href="http://lists.racket-lang.org/dev">http://lists.racket-lang.org/dev</a> </pre> </blockquote> <br> </div> <br> <br> On 03/29/2012 12:00 PM, Nick Shelley wrote: <blockquote cite="mid:CAPrQK3DZ501QHOoNtmBp2QuSq+zZoSZfV_pTqp4=d8NAhtcO6g@mail.gmail.com" type="cite">In my limited experience with parallel computing, it seems like the master-worker paradigm is somewhat common. However, it seems like Racket's places (or at least the way events are done with place-channels) makes this inconvenient. Since there is nothing that I can find in the result of syncing on a place-channel that would allow me to then send some new work to the same channel, I have to implement this manually by having each channel keep track of it's place in the list of workers and send that information explicitly. This approach seems hacky and more prone to bugs. <div> <br> </div> <div>I think it would make more sense for the result of syncing on a place-channel to return the channel itself as a result. This would make an explicit call to place-channel-get necessary, which may be a downside, but the upside is I could then put something on the same channel or do whatever else I may want with it.</div> <div><br> </div> <div>If changing how place-channel events are treated isn't feasible, I think it would at least be useful to provide an abstraction that makes master-worker more convenient. One idea might be a sync-channel function that returns the channel.</div> <div><br> </div> <div>As a sort of side note, it would be nice to be able to treat this problem similar to a user thread problem where you can conceptually imagine having infinite workers and just giving one chunk of work to each of them. I tried to do this at first, but because of the way places are implemented, I ran out of file descriptors relatively quickly (and the overhead of starting a new VM for each chunk of work might have been too much anyway, I don't know). I don't know if an abstraction like that is possible or useful in general, but it may be something to consider.</div> <div><br> </div> <div>-Nick</div> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_________________________ Racket Developers list: <a class="moz-txt-link-freetext" href="http://lists.racket-lang.org/dev">http://lists.racket-lang.org/dev</a> </pre> </blockquote> <br> </body> </html>