<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Is this the sort of situation that continuation barriers were made
for? Do you have any guidance about using them?<br>
<br>
#lang racket<br>
<br>
(define (only-once thunk)<br>
(dynamic-wind<br>
(λ () (displayln "pre-thunk"))<br>
(λ () (call-with-continuation-barrier thunk))<br>
(λ () (displayln "post-thunk"))))<br>
<br>
(only-once (λ () (displayln "hi")))<br>
<br>
(let ([saved-k #f])<br>
(only-once (λ () (let/cc k (set! saved-k k)<br>
(displayln "saving continuation"))))<br>
(displayln "invoking continuation...")<br>
(saved-k 11))<br>
<br>
Thanks,<br>
Dave<br>
<br>
<div class="moz-cite-prefix">On 08/08/2013 12:10 PM, Robby Findler
wrote:<br>
</div>
<blockquote
cite="mid:CAL3TdOOWCAj5cxV-7FWMvP+CwP7ELAkxsAEdjUMR1eHNkQq7mA@mail.gmail.com"
type="cite">
<div dir="ltr">As to the interactions with dynamic-wind and
continuations: l did, as you figured out, intend for you to use
only the post-thunk in the dynamic-wind (to close the pipes). In
principle, you could use the pre-thurnk to try to restore the
pipe, but there really isn't enough information to do this
correctly in all cases.
<div>
<br>
</div>
<div>I see that you've protected things a little bit by using
the port-closed? predicate as a guard, but if you did that to
protect against possible continuation re-entry, then probably
you're better off adding something to the pre-thunk that
explicitly raises an error saying that it isn't allowed to
re-enter. Something like this:</div>
<div><br>
</div>
<div>
<div>#lang racket</div>
<div><br>
</div>
<div>(define (only-once thunk)</div>
<div> (define already-in-once? #f)</div>
<div> (dynamic-wind</div>
<div> (λ ()</div>
<div> (when already-in-once? (error 'only-once "no no")))</div>
<div> (λ ()</div>
<div> (set! already-in-once? #t)</div>
<div> (thunk))</div>
<div> void))</div>
<div><br>
</div>
<div>(only-once (λ () "hi"))</div>
<div><br>
</div>
<div>(let ([saved-k #f])</div>
<div> (only-once (λ () (let/cc k (set! saved-k k))))</div>
<div> (saved-k 11))</div>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Tue, Aug 6, 2013 at 11:16 AM, JP
Verkamp <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:racket@jverkamp.com" target="_blank">racket@jverkamp.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">I've never actually used <font face="courier
new, monospace">dynamic-wind</font>, although it does
look interesting / like what I need. A few questions /
caveats though:
<div>
<br>
</div>
<div>- Should the pipe be created in the <font
face="courier new, monospace">pre-thunk</font> or
before the <font face="courier new, monospace">dynamic-wind</font>
entirely? The thunks don't seem to share scope, so I'm
guessing the latter, but that seems a bit odd. I'm
guessing the <font face="courier new, monospace">pre-thunk</font>
is for an entirely different use case though when you
are actually dealing with closing and reopening
resources are the like as control gets passed around.<br>
<div><br>
<div>- Doesn't <font face="courier new, monospace">dynamic-wind</font>
break if the user messes with continuations during
the <font face="courier new, monospace">value-thunk</font>?
So far as I understand, when control passes out, <font
face="courier new, monospace">post-thunk</font> is
called and then <font face="courier new, monospace">pre-thunk</font>
on the way back in, but that means that when control
returns the port will be closed. I don't know how
often this will come up, but it seems to break if I
nest a thread inside of the <font face="courier
new, monospace">with-gzip</font> call. Granted, my
version did as well because of the <font
face="courier new, monospace">close-input-port</font>
call. Is this just expected behavior?</div>
</div>
<div><br>
</div>
<div>(And yes, it works fine in the more likely /
sensible case of wrapping the entire <font
face="courier new, monospace">with-gzip</font> in a
thread in both cases.)</div>
<div><br>
</div>
<div>- So far as <font face="courier new, monospace">error</font>
rather than <font face="courier new, monospace">raise</font>,
<font face="courier new, monospace">raise</font> was
my original guess. But that added another layer of
indirection to the stack trace which I didn't at first
notice (I thought I wasn't even catching the error).
It makes sense to have that though in the long run.</div>
</div>
<div><br>
</div>
<div>That all being said, how does this version look?</div>
<div><br>
</div>
<div>
<div class="im">
<div><font face="courier new, monospace">(define
(with-gunzip thunk)</font></div>
<div><font face="courier new, monospace">
(define-values (pipe-from pipe-to) (make-pipe))</font></div>
</div>
<div><font face="courier new, monospace"> (dynamic-wind</font></div>
<div><font face="courier new, monospace"> void</font></div>
<div><font face="courier new, monospace"> (λ ()</font></div>
<div><font face="courier new, monospace">
(gunzip-through-ports (current-input-port) pipe-to)</font></div>
<div class="im">
<div><font face="courier new, monospace">
(close-output-port pipe-to) </font></div>
<div><font face="courier new, monospace">
(parameterize ([current-input-port pipe-from])</font></div>
</div>
<div><font face="courier new, monospace">
(thunk)))</font></div>
<div><font face="courier new, monospace"> (λ ()</font></div>
<div><font face="courier new, monospace"> (unless
(port-closed? pipe-to) (close-output-port pipe-to))</font></div>
<div><font face="courier new, monospace"> (unless
(port-closed? pipe-from) (close-input-port
pipe-from)))))</font></div>
</div>
</div>
<div class="HOEnZb">
<div class="h5">
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Tue, Aug 6, 2013 at 11:47
AM, Robby Findler <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:robby@eecs.northwestern.edu"
target="_blank">robby@eecs.northwestern.edu</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">You might consider using
dynamic-wind instead of that with-handlers. Or,
instead of (error 'with-gunzip ...) just do
(raise exn). That way you won't lose the stack
information in the original exception (which is
likely the one a user would want).
<div>
<br>
Robby</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">
<div>
<div>On Tue, Aug 6, 2013 at 10:40 AM, JP
Verkamp <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:racket@jverkamp.com"
target="_blank">racket@jverkamp.com</a>></span>
wrote:<br>
</div>
</div>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div>
<div>
<div dir="ltr">Figured it out and
cleaned it up. It turns out that I was
using <font face="courier new,
monospace">with-handlers</font>
oddly, but reading further though the
documentation it works as expected.
Here's a new version (generalized to
any input-port):
<div>
<br>
</div>
<div>
<div><font face="courier new,
monospace">(define (with-gunzip
thunk)</font></div>
<div><font face="courier new,
monospace"> (define-values
(pipe-from pipe-to) (make-pipe))</font></div>
<div><font face="courier new,
monospace"> (with-handlers
([exn:fail?</font></div>
<div><font face="courier new,
monospace"> (λ
(err)</font></div>
<div><font face="courier new,
monospace">
(close-output-port pipe-to)</font></div>
<div><font face="courier new,
monospace">
(close-input-port pipe-from)</font></div>
<div><font face="courier new,
monospace">
(error 'with-gunzip
(exn-message err)))])</font></div>
<div><font face="courier new,
monospace">
(gunzip-through-ports
(current-input-port) pipe-to)</font></div>
<div><font face="courier new,
monospace">
(close-output-port pipe-to)</font></div>
<div><font face="courier new,
monospace"> (parameterize
([current-input-port pipe-from])</font></div>
<div><font face="courier new,
monospace"> (thunk))</font></div>
<div><font face="courier new,
monospace"> (close-input-port
pipe-from)))</font></div>
</div>
<div><br>
</div>
<div>If anyone's interested in a more
in depth write up / source code for
this and with-gzip:</div>
<div>- writeup: <a
moz-do-not-send="true"
href="http://blog.jverkamp.com/2013/08/06/adventures-in-racket-gzip/"
target="_blank">http://blog.jverkamp.com/2013/08/06/adventures-in-racket-gzip/</a></div>
<div>- source: <a
moz-do-not-send="true"
href="https://github.com/jpverkamp/small-projects/tree/master/blog/with-gzip.rkt"
target="_blank">https://github.com/jpverkamp/small-projects/tree/master/blog/with-gzip.rkt</a></div>
</div>
<div>
<div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon,
Aug 5, 2013 at 5:36 PM, JP
Verkamp <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:racket@jverkamp.com"
target="_blank">racket@jverkamp.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div dir="ltr">Thanks! <font
face="courier new,
monospace">make-pipe</font>
isn't something that I've
had to use otherwise, so I
missed the optional
parameter. That does
certainly seem to help.
<div><br>
</div>
<div>
Here's my first take of <font
face="courier new,
monospace">with-input-from-gzipped-file</font>:</div>
<div><br>
</div>
<div>
<div>
<div><font face="courier
new, monospace">(define
(with-input-from-gzipped-file
filename thunk
#:buffer-size
[buffer-size #f])</font></div>
<div><font face="courier
new, monospace">
(call-with-input-file
filename</font></div>
<div><font face="courier
new, monospace">
(lambda (file-from)</font></div>
<div><font face="courier
new, monospace">
(define-values
(pipe-from pipe-to)
(make-pipe
buffer-size))</font></div>
<div><font face="courier
new, monospace">
</font></div>
<div><font face="courier
new, monospace">
(thread </font></div>
<div><font face="courier
new, monospace">
(λ () </font></div>
<div><font face="courier
new, monospace">
(gunzip-through-ports
file-from pipe-to)</font></div>
<div><font face="courier
new, monospace">
(close-output-port
pipe-to)))</font></div>
<div><font face="courier
new, monospace">
</font></div>
<div><font face="courier
new, monospace">
(current-input-port
pipe-from)</font></div>
<div><font face="courier
new, monospace">
(thunk)</font></div>
<div><font face="courier
new, monospace">
(close-input-port
pipe-from))))</font></div>
</div>
<div><br>
</div>
<div>The main thing
missing is that there's
no error handling (where
the pipe should still be
closed). At the very
least, if I try to call
this on a non-gzipped
file, it breaks on the <font
face="courier new,
monospace">gunzip-through-ports</font>
line. Theoretically,
some variation of <font
face="courier new,
monospace">with-handlers</font>
should work (<font
face="courier new,
monospace">error</font>
should raise an <font
face="courier new,
monospace">exn:fail?</font>,
yes?), but it doesn't
seem to be helping.</div>
<div><br>
</div>
<div>Any help with that?</div>
<div><br>
</div>
<div>Alternatively, I've
now found this: <a
moz-do-not-send="true"
href="http://planet.racket-lang.org/display.ss?package=gzip.plt&owner=soegaard"
target="_blank">http://planet.racket-lang.org/display.ss?package=gzip.plt&owner=soegaard</a></div>
<div><br>
</div>
<div>It seems to do
exactly what I need,
albeit without the
call-with-* forms, but
that's easy enough to
wrap. With some very
basic testing, it does
seem to be buffering
though, although it is a
bit slower than the
above. Not enough to
cause trouble though.</div>
</div>
</div>
<div>
<div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On
Mon, Aug 5, 2013 at
4:51 PM, Ryan
Culpepper <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:ryanc@ccs.neu.edu" target="_blank">ryanc@ccs.neu.edu</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div>On 08/05/2013
04:29 PM, JP
Verkamp wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
Is there a nice
/ idiomatic way
to work with
gzipped data in
a streaming<br>
manner (to avoid
loading the
rather large
files into
memory at once).
So<br>
far as I can
tell, my code
isn't doing
that. It hangs
for a while on
the<br>
call to
gunzip-through-ports,
long enough to
uncompress the
entire file,<br>
then reads are
pretty quick
afterwords.<br>
<br>
Here's what I
have thus far:<br>
<br>
#lang racket<br>
<br>
(require
file/gunzip)<br>
<br>
(define-values
(pipe-from
pipe-to)
(make-pipe))<br>
(with-input-from-file
"test.rkt.gz"<br>
(lambda ()<br>
(gunzip-through-ports
(current-input-port)
pipe-to)<br>
(for ([line
(in-lines
pipe-from)])<br>
(displayln
line))))<br>
</blockquote>
<br>
</div>
You should probably
1) limit the size of
the pipe (to stop it
from inflating the
whole file at once)
and 2) put the
gunzip-through-ports
call in a separate
thread. The gunzip
thread will block
when the pipe is
full; when your
program reads some
data out of the
pipe, the gunzip
thread will be able
to make some more
progress. Something
like this:<br>
<br>
(define-values
(pipe-from pipe-to)
(make-pipe 4000))<br>
(with-input-from-file
"test.rkt.gz"<br>
(lambda ()<br>
(thread
<div><br>
(lambda ()<br>
(gunzip-through-ports
(current-input-port)
pipe-to)<br>
</div>
(close-output-port
pipe-to)))
<div><br>
(for ([line
(in-lines
pipe-from)])<br>
(displayln
line))))<br>
<br>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
As an additional
problem, that
code doesn't
actually work.<br>
in-lines seems
to be waiting
for an
eof-object? that<br>
gunzip-through-ports
isn't sending.
Am I missing
something? It
ends up<br>
just hanging
after reading
and printing the
file.<br>
</blockquote>
<br>
</div>
The docs don't say
anything about
closing the port, so
you'll probably have
to do that yourself.
In the code above, I
added a call to
close-output-port.<span><font
color="#888888"><br>
<br>
Ryan<br>
<br>
</font></span></blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
<br>
</div>
</div>
____________________<br>
Racket Users list:<br>
<a moz-do-not-send="true"
href="http://lists.racket-lang.org/users"
target="_blank">http://lists.racket-lang.org/users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
<br>
____________________<br>
Racket Users list:<br>
<a moz-do-not-send="true"
href="http://lists.racket-lang.org/users" target="_blank">http://lists.racket-lang.org/users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">____________________
Racket Users list:
<a class="moz-txt-link-freetext" href="http://lists.racket-lang.org/users">http://lists.racket-lang.org/users</a>
</pre>
</blockquote>
<br>
</body>
</html>