[plt-scheme] Re: web-server: connection timeout with open file handles

Hi Jay,

Thanks for looking at this. Answers below:

> I presume that you are using adjust-timeout! ? Or are you using a
> dispatcher that wraps another and does something to the connection
> while it runs?

Yes. It's a one-line hack to web-server/http/response.ss:

===================================================================
--- http/response.ss	(revision 13220)
+++ http/response.ss	(working copy)
@@ -154,7 +154,9 @@
             ((response/incremental-generator bresp)
              (lambda chunks
                (fprintf o-port "~x\r\n"
                        (apply + 0 (map data-length chunks)))
+              (adjust-connection-timeout! conn 5)
                (for-each (lambda (chunk) (display chunk o-port))  
chunks)
                (fprintf o-port "\r\n")))
             ; one \r\n ends the last (empty) chunk and the second \r 
\n ends the (non-existant) trailers

My code is reading and writing 4kb chunks of the file so 5 seconds is  
easily enough. All downloads seem to complete properly now, even if  
they are very large.

I thought this hack would address the file handle retention issue but  
it looks like it hasn't, at least on our Debian server. Everything  
works okay on my Mac laptop but I might not have hit the right  
combination of factors to make it break.

> Regarding the file handles, I presume that are your file handles  
> from call-with-input-file?
>  [...] Can you track down what kind of file handles are being  
> retained? I don't think that
> there's a problem with call-with-input-file, but I'd like more info.

call-with-input-file certainly seems to be the culprit. Here are some  
lines from lsof:

COMMAND   PID    USER   FD   TYPE   DEVICE     SIZE     NODE NAME
# ... blah blah blah ...
mzscheme 9300 untyped  130u   REG      9,0  5346912   950289 /data/fs/ 
module/sbs524/lecture2.pdf
mzscheme 9300 untyped  131r   REG      9,0  5346912   950289 /data/fs/ 
module/sbs524/lecture2.pdf
mzscheme 9300 untyped  132u   REG      9,0  3087554   950290 /data/fs/ 
module/sbs524/lecture3.pdf
mzscheme 9300 untyped  133u   REG      9,0  3087554   950290 /data/fs/ 
module/sbs524/lecture3.pdf
# ... blah blah blah ...

These are the files (lecture notes and so on) that are being read with  
call-with-input-file. I get one line for each file download. Some  
lines (not all) are retained after the download is complete. Only  
larger downloads seem to be retained (6+ megabytes, or 1+ minute of  
download underload).

> When a servlet runs, its resources are owned by the servlet's
> custodian, so they are shared by the entire servlet and not just the
> instance or connection. This means that if you don't free resources,
> it won't be removed until the servlet is, which only happens when you
> conf/refresh-servlets.

Returning gracefully from call-with-input-file should do this, right?  
Can I do anything to make sure the file handle is released if there is  
an abrupt exit?

I'll add a debugging message to make sure there is a graceful return.  
I'll also see if sleeping for a second after call-with-input-file  
finished helps at all.

Cheers,

-- Dave

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.racket-lang.org/users/archive/attachments/20090422/1807e872/attachment.html>