<div dir="ltr">Oh, yes. Continuation barriers are a better way to do basically what I wrote. Sorry, I should have mentioned that; I had forgotten about them.<div><br></div><div>Robby</div></div><div class="gmail_extra"><br>
<br><div class="gmail_quote">On Fri, Aug 9, 2013 at 1:05 PM, David Vanderson <span dir="ltr">&lt;<a href="mailto:david.vanderson@gmail.com" target="_blank">david.vanderson@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">

  
    
  
  <div text="#000000" bgcolor="#FFFFFF">
    Is this the sort of situation that continuation barriers were made
    for?  Do you have any guidance about using them?<div class="im"><br>
    <br>
    #lang racket<br>
    <br>
    (define (only-once thunk)<br></div>
      (dynamic-wind<br>
       (λ () (displayln &quot;pre-thunk&quot;))<br>
       (λ () (call-with-continuation-barrier thunk))<br>
       (λ () (displayln &quot;post-thunk&quot;))))<br>
    <br>
    (only-once (λ () (displayln &quot;hi&quot;)))<div class="im"><br>
    <br>
    (let ([saved-k #f])<br>
      (only-once (λ () (let/cc k (set! saved-k k)<br></div>
                         (displayln &quot;saving continuation&quot;))))<br>
      (displayln &quot;invoking continuation...&quot;)<br>
      (saved-k 11))<br>
    <br>
    Thanks,<br>
    Dave<div><div class="h5"><br>
    <br>
    <div>On 08/08/2013 12:10 PM, Robby Findler
      wrote:<br>
    </div>
    <blockquote type="cite">
      <div dir="ltr">As to the interactions with dynamic-wind and
        continuations: l did, as you figured out, intend for you to use
        only the post-thunk in the dynamic-wind (to close the pipes). In
        principle, you could use the pre-thurnk to try to restore the
        pipe, but there really isn&#39;t enough information to do this
        correctly in all cases. 
        <div>
          <br>
        </div>
        <div>I see that you&#39;ve protected things a little bit by using
          the port-closed? predicate as a guard, but if you did that to
          protect against possible continuation re-entry, then probably
          you&#39;re better off adding something to the pre-thunk that
          explicitly raises an error saying that it isn&#39;t allowed to
          re-enter. Something like this:</div>
        <div><br>
        </div>
        <div>
          <div>#lang racket</div>
          <div><br>
          </div>
          <div>(define (only-once thunk)</div>
          <div>  (define already-in-once? #f)</div>
          <div>  (dynamic-wind</div>
          <div>   (λ ()</div>
          <div>     (when already-in-once? (error &#39;only-once &quot;no no&quot;)))</div>
          <div>   (λ ()</div>
          <div>     (set! already-in-once? #t)</div>
          <div>     (thunk))</div>
          <div>   void))</div>
          <div><br>
          </div>
          <div>(only-once (λ () &quot;hi&quot;))</div>
          <div><br>
          </div>
          <div>(let ([saved-k #f])</div>
          <div>  (only-once (λ () (let/cc k (set! saved-k k))))</div>
          <div>  (saved-k 11))</div>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
      </div>
      <div class="gmail_extra"><br>
        <br>
        <div class="gmail_quote">On Tue, Aug 6, 2013 at 11:16 AM, JP
          Verkamp <span dir="ltr">&lt;<a href="mailto:racket@jverkamp.com" target="_blank">racket@jverkamp.com</a>&gt;</span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div dir="ltr">I&#39;ve never actually used <font face="courier
                new, monospace">dynamic-wind</font>, although it does
              look interesting / like what I need. A few questions /
              caveats though:
              <div>
                <br>
              </div>
              <div>- Should the pipe be created in the <font face="courier new, monospace">pre-thunk</font> or
                before the <font face="courier new, monospace">dynamic-wind</font>
                entirely? The thunks don&#39;t seem to share scope, so I&#39;m
                guessing the latter, but that seems a bit odd. I&#39;m
                guessing the <font face="courier new, monospace">pre-thunk</font>
                is for an entirely different use case though when you
                are actually dealing with closing and reopening
                resources are the like as control gets passed around.<br>
                <div><br>
                  <div>- Doesn&#39;t <font face="courier new, monospace">dynamic-wind</font>
                    break if the user messes with continuations during
                    the <font face="courier new, monospace">value-thunk</font>?
                    So far as I understand, when control passes out, <font face="courier new, monospace">post-thunk</font> is
                    called and then <font face="courier new, monospace">pre-thunk</font>
                    on the way back in, but that means that when control
                    returns the port will be closed. I don&#39;t know how
                    often this will come up, but it seems to break if I
                    nest a thread inside of the <font face="courier
                      new, monospace">with-gzip</font> call. Granted, my
                    version did as well because of the <font face="courier new, monospace">close-input-port</font>
                    call. Is this just expected behavior?</div>
                </div>
                <div><br>
                </div>
                <div>(And yes, it works fine in the more likely /
                  sensible case of wrapping the entire <font face="courier new, monospace">with-gzip</font> in a
                  thread in both cases.)</div>
                <div><br>
                </div>
                <div>- So far as <font face="courier new, monospace">error</font>
                  rather than <font face="courier new, monospace">raise</font>,
                  <font face="courier new, monospace">raise</font> was
                  my original guess. But that added another layer of
                  indirection to the stack trace which I didn&#39;t at first
                  notice (I thought I wasn&#39;t even catching the error).
                  It makes sense to have that though in the long run.</div>
              </div>
              <div><br>
              </div>
              <div>That all being said, how does this version look?</div>
              <div><br>
              </div>
              <div>
                <div>
                  <div><font face="courier new, monospace">(define
                      (with-gunzip thunk)</font></div>
                  <div><font face="courier new, monospace"> 
                      (define-values (pipe-from pipe-to) (make-pipe))</font></div>
                </div>
                <div><font face="courier new, monospace">  (dynamic-wind</font></div>
                <div><font face="courier new, monospace">   void</font></div>
                <div><font face="courier new, monospace">   (λ ()</font></div>
                <div><font face="courier new, monospace">   
                     (gunzip-through-ports (current-input-port) pipe-to)</font></div>
                <div>
                  <div><font face="courier new, monospace">   
                       (close-output-port pipe-to) </font></div>
                  <div><font face="courier new, monospace">   
                       (parameterize ([current-input-port pipe-from])</font></div>
                </div>
                <div><font face="courier new, monospace">     
                     (thunk)))</font></div>
                <div><font face="courier new, monospace">   (λ ()</font></div>
                <div><font face="courier new, monospace">     (unless
                    (port-closed? pipe-to) (close-output-port pipe-to))</font></div>
                <div><font face="courier new, monospace">     (unless
                    (port-closed? pipe-from) (close-input-port
                    pipe-from)))))</font></div>
              </div>
            </div>
            <div>
              <div>
                <div class="gmail_extra"><br>
                  <br>
                  <div class="gmail_quote">On Tue, Aug 6, 2013 at 11:47
                    AM, Robby Findler <span dir="ltr">&lt;<a href="mailto:robby@eecs.northwestern.edu" target="_blank">robby@eecs.northwestern.edu</a>&gt;</span>
                    wrote:<br>
                    <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                      <div dir="ltr">You might consider using
                        dynamic-wind instead of that with-handlers. Or,
                        instead of (error &#39;with-gunzip ...) just do
                        (raise exn). That way you won&#39;t lose the stack
                        information in the original exception (which is
                        likely the one a user would want).
                        <div>
                          <br>
                          Robby</div>
                      </div>
                      <div class="gmail_extra"><br>
                        <br>
                        <div class="gmail_quote">
                          <div>
                            <div>On Tue, Aug 6, 2013 at 10:40 AM, JP
                              Verkamp <span dir="ltr">&lt;<a href="mailto:racket@jverkamp.com" target="_blank">racket@jverkamp.com</a>&gt;</span>
                              wrote:<br>
                            </div>
                          </div>
                          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                            <div>
                              <div>
                                <div dir="ltr">Figured it out and
                                  cleaned it up. It turns out that I was
                                  using <font face="courier new,
                                    monospace">with-handlers</font>
                                  oddly, but reading further though the
                                  documentation it works as expected.
                                  Here&#39;s a new version (generalized to
                                  any input-port):
                                  <div>
                                    <br>
                                  </div>
                                  <div>
                                    <div><font face="courier new,
                                        monospace">(define (with-gunzip
                                        thunk)</font></div>
                                    <div><font face="courier new,
                                        monospace">  (define-values
                                        (pipe-from pipe-to) (make-pipe))</font></div>
                                    <div><font face="courier new,
                                        monospace">  (with-handlers
                                        ([exn:fail?</font></div>
                                    <div><font face="courier new,
                                        monospace">                   (λ
                                        (err)</font></div>
                                    <div><font face="courier new,
                                        monospace">                   
                                         (close-output-port pipe-to)</font></div>
                                    <div><font face="courier new,
                                        monospace">                   
                                         (close-input-port pipe-from)</font></div>
                                    <div><font face="courier new,
                                        monospace">                   
                                         (error &#39;with-gunzip
                                        (exn-message err)))])</font></div>
                                    <div><font face="courier new,
                                        monospace">   
                                        (gunzip-through-ports
                                        (current-input-port) pipe-to)</font></div>
                                    <div><font face="courier new,
                                        monospace">   
                                        (close-output-port pipe-to)</font></div>
                                    <div><font face="courier new,
                                        monospace">    (parameterize
                                        ([current-input-port pipe-from])</font></div>
                                    <div><font face="courier new,
                                        monospace">      (thunk))</font></div>
                                    <div><font face="courier new,
                                        monospace">    (close-input-port
                                        pipe-from)))</font></div>
                                  </div>
                                  <div><br>
                                  </div>
                                  <div>If anyone&#39;s interested in a more
                                    in depth write up / source code for
                                    this and with-gzip:</div>
                                  <div>- writeup: <a href="http://blog.jverkamp.com/2013/08/06/adventures-in-racket-gzip/" target="_blank">http://blog.jverkamp.com/2013/08/06/adventures-in-racket-gzip/</a></div>
                                  <div>- source: <a href="https://github.com/jpverkamp/small-projects/tree/master/blog/with-gzip.rkt" target="_blank">https://github.com/jpverkamp/small-projects/tree/master/blog/with-gzip.rkt</a></div>

                                </div>
                                <div>
                                  <div>
                                    <div class="gmail_extra"><br>
                                      <br>
                                      <div class="gmail_quote">On Mon,
                                        Aug 5, 2013 at 5:36 PM, JP
                                        Verkamp <span dir="ltr">&lt;<a href="mailto:racket@jverkamp.com" target="_blank">racket@jverkamp.com</a>&gt;</span>
                                        wrote:<br>
                                        <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                          <div dir="ltr">Thanks! <font face="courier new,
                                              monospace">make-pipe</font>
                                            isn&#39;t something that I&#39;ve
                                            had to use otherwise, so I
                                            missed the optional
                                            parameter. That does
                                            certainly seem to help.
                                            <div><br>
                                            </div>
                                            <div>
                                              Here&#39;s my first take of <font face="courier new,
                                                monospace">with-input-from-gzipped-file</font>:</div>
                                            <div><br>
                                            </div>
                                            <div>
                                              <div>
                                                <div><font face="courier
                                                    new, monospace">(define
                                                    (with-input-from-gzipped-file
                                                    filename thunk
                                                    #:buffer-size
                                                    [buffer-size #f])</font></div>
                                                <div><font face="courier
                                                    new, monospace"> 
                                                    (call-with-input-file
                                                    filename</font></div>
                                                <div><font face="courier
                                                    new, monospace">   
                                                    (lambda (file-from)</font></div>
                                                <div><font face="courier
                                                    new, monospace">   
                                                      (define-values
                                                    (pipe-from pipe-to)
                                                    (make-pipe
                                                    buffer-size))</font></div>
                                                <div><font face="courier
                                                    new, monospace">   
                                                      </font></div>
                                                <div><font face="courier
                                                    new, monospace">   
                                                      (thread </font></div>
                                                <div><font face="courier
                                                    new, monospace">   
                                                          (λ () </font></div>
                                                <div><font face="courier
                                                    new, monospace">   
                                                           
                                                    (gunzip-through-ports
                                                    file-from pipe-to)</font></div>
                                                <div><font face="courier
                                                    new, monospace">   
                                                           
                                                    (close-output-port
                                                    pipe-to)))</font></div>
                                                <div><font face="courier
                                                    new, monospace">   
                                                      </font></div>
                                                <div><font face="courier
                                                    new, monospace">   
                                                     
                                                    (current-input-port
                                                    pipe-from)</font></div>
                                                <div><font face="courier
                                                    new, monospace">   
                                                      (thunk)</font></div>
                                                <div><font face="courier
                                                    new, monospace">   
                                                      (close-input-port
                                                    pipe-from))))</font></div>
                                              </div>
                                              <div><br>
                                              </div>
                                              <div>The main thing
                                                missing is that there&#39;s
                                                no error handling (where
                                                the pipe should still be
                                                closed). At the very
                                                least, if I try to call
                                                this on a non-gzipped
                                                file, it breaks on the <font face="courier new,
                                                  monospace">gunzip-through-ports</font>
                                                line. Theoretically,
                                                some variation of <font face="courier new,
                                                  monospace">with-handlers</font>
                                                should work (<font face="courier new,
                                                  monospace">error</font>
                                                should raise an <font face="courier new,
                                                  monospace">exn:fail?</font>,
                                                yes?), but it doesn&#39;t
                                                seem to be helping.</div>
                                              <div><br>
                                              </div>
                                              <div>Any help with that?</div>
                                              <div><br>
                                              </div>
                                              <div>Alternatively, I&#39;ve
                                                now found this: <a href="http://planet.racket-lang.org/display.ss?package=gzip.plt&amp;owner=soegaard" target="_blank">http://planet.racket-lang.org/display.ss?package=gzip.plt&amp;owner=soegaard</a></div>

                                              <div><br>
                                              </div>
                                              <div>It seems to do
                                                exactly what I need,
                                                albeit without the
                                                call-with-* forms, but
                                                that&#39;s easy enough to
                                                wrap. With some very
                                                basic testing, it does
                                                seem to be buffering
                                                though, although it is a
                                                bit slower than the
                                                above. Not enough to
                                                cause trouble though.</div>
                                            </div>
                                          </div>
                                          <div>
                                            <div>
                                              <div class="gmail_extra"><br>
                                                <br>
                                                <div class="gmail_quote">On
                                                  Mon, Aug 5, 2013 at
                                                  4:51 PM, Ryan
                                                  Culpepper <span dir="ltr">&lt;<a href="mailto:ryanc@ccs.neu.edu" target="_blank">ryanc@ccs.neu.edu</a>&gt;</span>
                                                  wrote:<br>
                                                  <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                                    <div>On 08/05/2013
                                                      04:29 PM, JP
                                                      Verkamp wrote:<br>
                                                      <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                                        Is there a nice
                                                        / idiomatic way
                                                        to work with
                                                        gzipped data in
                                                        a streaming<br>
                                                        manner (to avoid
                                                        loading the
                                                        rather large
                                                        files into
                                                        memory at once).
                                                        So<br>
                                                        far as I can
                                                        tell, my code
                                                        isn&#39;t doing
                                                        that. It hangs
                                                        for a while on
                                                        the<br>
                                                        call to
                                                        gunzip-through-ports,
                                                        long enough to
                                                        uncompress the
                                                        entire file,<br>
                                                        then reads are
                                                        pretty quick
                                                        afterwords.<br>
                                                        <br>
                                                        Here&#39;s what I
                                                        have thus far:<br>
                                                        <br>
                                                        #lang racket<br>
                                                        <br>
                                                        (require
                                                        file/gunzip)<br>
                                                        <br>
                                                        (define-values
                                                        (pipe-from
                                                        pipe-to)
                                                        (make-pipe))<br>
                                                        (with-input-from-file
                                                        &quot;test.rkt.gz&quot;<br>
                                                           (lambda ()<br>
                                                           
                                                         (gunzip-through-ports
                                                        (current-input-port)
                                                        pipe-to)<br>
                                                             (for ([line
                                                        (in-lines
                                                        pipe-from)])<br>
                                                             
                                                         (displayln
                                                        line))))<br>
                                                      </blockquote>
                                                      <br>
                                                    </div>
                                                    You should probably
                                                    1) limit the size of
                                                    the pipe (to stop it
                                                    from inflating the
                                                    whole file at once)
                                                    and 2) put the
                                                    gunzip-through-ports
                                                    call in a separate
                                                    thread. The gunzip
                                                    thread will block
                                                    when the pipe is
                                                    full; when your
                                                    program reads some
                                                    data out of the
                                                    pipe, the gunzip
                                                    thread will be able
                                                    to make some more
                                                    progress. Something
                                                    like this:<br>
                                                    <br>
                                                    (define-values
                                                    (pipe-from pipe-to)
                                                    (make-pipe 4000))<br>
                                                    (with-input-from-file
                                                    &quot;test.rkt.gz&quot;<br>
                                                      (lambda ()<br>
                                                        (thread
                                                    <div><br>
                                                            (lambda ()<br>
                                                             
                                                      (gunzip-through-ports
                                                      (current-input-port)
                                                      pipe-to)<br>
                                                    </div>
                                                           
                                                    (close-output-port
                                                    pipe-to)))
                                                    <div><br>
                                                          (for ([line
                                                      (in-lines
                                                      pipe-from)])<br>
                                                            (displayln
                                                      line))))<br>
                                                      <br>
                                                      <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                                        As an additional
                                                        problem, that
                                                        code doesn&#39;t
                                                        actually work.<br>
                                                        in-lines seems
                                                        to be waiting
                                                        for an
                                                        eof-object? that<br>
                                                        gunzip-through-ports
                                                        isn&#39;t sending.
                                                        Am I missing
                                                        something? It
                                                        ends up<br>
                                                        just hanging
                                                        after reading
                                                        and printing the
                                                        file.<br>
                                                      </blockquote>
                                                      <br>
                                                    </div>
                                                    The docs don&#39;t say
                                                    anything about
                                                    closing the port, so
                                                    you&#39;ll probably have
                                                    to do that yourself.
                                                    In the code above, I
                                                    added a call to
                                                    close-output-port.<span><font color="#888888"><br>
                                                        <br>
                                                        Ryan<br>
                                                        <br>
                                                      </font></span></blockquote>
                                                </div>
                                                <br>
                                              </div>
                                            </div>
                                          </div>
                                        </blockquote>
                                      </div>
                                      <br>
                                    </div>
                                  </div>
                                </div>
                                <br>
                              </div>
                            </div>
                            ____________________<br>
                              Racket Users list:<br>
                              <a href="http://lists.racket-lang.org/users" target="_blank">http://lists.racket-lang.org/users</a><br>
                            <br>
                          </blockquote>
                        </div>
                        <br>
                      </div>
                    </blockquote>
                  </div>
                  <br>
                </div>
              </div>
            </div>
            <br>
            ____________________<br>
              Racket Users list:<br>
              <a href="http://lists.racket-lang.org/users" target="_blank">http://lists.racket-lang.org/users</a><br>
            <br>
          </blockquote>
        </div>
        <br>
      </div>
      <br>
      <fieldset></fieldset>
      <br>
      <pre>____________________
  Racket Users list:
  <a href="http://lists.racket-lang.org/users" target="_blank">http://lists.racket-lang.org/users</a>
</pre>
    </blockquote>
    <br>
  </div></div></div>

</blockquote></div><br></div>