[plt-scheme] web-server: connection timeout with open file handles

Hi all,

I'm having a couple of issues with timeouts in the web server. This is  
kind of related to the "dynamic-wind & kill-thread" thread.

I'm using a servlet and make-response/incremental to serve files from  
the file system to web users. I've attached simplified test code below  
(tested on PLT 4.1.4.1).

Three problems:

   - large file downloads can take a long time, and can be killed by  
the web server's connection timeout (default 60 seconds);

   - when the connection times out, the servlet thread is killed and  
leaves the file handle open permanently ("lsof -p <PID>").

   - over time, the web server accrues open file handles until it hits  
the OS-imposed limit ("ulimit -n")  and stops working properly  
(connecting to Postgres in my case).

I remember back in the day we had the same problem with files served  
using dispatch-files.ss. The solution there was to insert a call to  
adjust-connection-timeout! to increase the connection timeout for  
large files.

Here it's not so simple because make-response/incremental can do more  
than just serve static files.

I'd like to be able to call adjust-connection-timeout! from servlet  
code, but I don't think that feature is available from inside a  
servlet (no access to the connection object).

Cheers,

-- Dave

===== test-server.ss =====

#lang scheme/base

(require scheme/runtime-path
          web-server/servlet-env
          web-server/http)

; Plain text file of >60 bytes in length:
(define-runtime-path test-file "test-file.txt")

; Serve the text file at an artificiually slow rage (1 byte/sec):
(define (make-test-response)
   (make-response/incremental
    200
    "Okay"
    (current-seconds)
    #"text/plain"
    null
    (lambda (write)

      ; This is 4096 in my actual production code:
      (define BUFFER-SIZE 1)
      (define buffer (make-bytes BUFFER-SIZE))

      (call-with-input-file test-file
        (lambda (input)
          (let loop ([counter 0])

            (sleep 1)
            (printf "... ~as~n" counter)

            ; Send some data to the browser:
            (let ([num-read (read-bytes! buffer input)])
              (cond [(eof-object? num-read)
                     (void)]
                    [(= num-read BUFFER-SIZE)
                     (write buffer)
                     (loop (add1 counter))]
                    [else
                     (write (subbytes buffer 0 num-read))
                     (loop (add1 counter))]))))))))

; Main server loop:
(serve/servlet
  (lambda (request)
    (printf "Serving file.~n")
    (begin0 (make-test-response)
            (printf "Finished serving file.~n"))))

===== end test-server.ss =====

===== test-file.txt =====

0123456789
0123456789
0123456789
0123456789
0123456789
0123456789
0123456789
0123456789
0123456789
0123456789

===== end test-file.txt =====

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.racket-lang.org/users/archive/attachments/20090422/e4c6588f/attachment.html>