[plt-scheme] Socket memory leak under Windows

This reminds me of a (very) similar problem with the web-server. That
memory leak probably happens because of the memory leaks in the socket
connections (or disconnections). But wasn't there a known solution to
the web-server problem?

ifconfig

-----Original Message-----
From: plt-scheme-admin at list.cs.brown.edu
[mailto:plt-scheme-admin at list.cs.brown.edu] On Behalf Of Blake McBride
Sent: Monday, February 09, 2004 4:47 AM
To: plt-scheme at list.cs.brown.edu
Subject: [plt-scheme] Socket memory leak under Windows

  For list-related administrative tasks:
  http://list.cs.brown.edu/mailman/listinfo/plt-scheme


Greetings,

I have encountered a significant memory leak using sockets in
MzScheme 206 under Windows.  I have spent several days on this problem
and discovered some interesting facts.  I believe there is some sort
of conflict between MzScheme's use of native threads in concert with
the Windows socket API.  Any help or pointers would be greatly
appreciated.

I have reduced the problem down to simply making repeated socket
connection and disconnections.  The two functions I am running
(one per instance of MzScheme) are as follows:


(define server
   (lambda ()
     (let ((srv (tcp-listen 8888)))
       (let loop ()
	(let-values (((in out) (tcp-accept srv)))
		    (close-input-port in)
		    (close-output-port out))
	(loop)))))

(define client
   (lambda ()
     (let-values (((in out) (tcp-connect "localhost" 8888)))
		(close-input-port in)
		(close-output-port out)
		(sleep .1)
		(client))))

The program is designed to make a single connection over and over.
There is no use of scheme threads since only a single connection is
being made at a time and I wished to make the test program as simple
as possible.  (The real program handles multiple connections via
threads.)

When this client/server scenario is run under Windows one can monitor
the memory usage of the server process and it will continually
increase its memory footprint.  If run long enough it will render the
machine useless.  Given this situation it is not possible to use
MzScheme in a socket server environment under Windows.  Presuming
others have successfully used MzScheme as a socket server under
Windows I believe they must either reboot often enough to make the
small leak unimportant or they make a relatively small number of
connections (i.e. not in the thousands per hour).

After several days of investigation I have discovered the following:

1.  The problem exists under Windows and not under Linux.

2.  The problem also exists in MzScheme 103 (so has probably always
     been there)

3.  I wrote an equivalent server in C using the Windows API (as
     MzScheme does internally) and there was no leak.

4.  I re-build MzScheme 103 as "Win32 Release" instead of "Win32
     Threads" and the problem went away but of course I wouldn't be
     able to use threads anymore.  This is interesting because my test
     program doesn't use any threads although I believe MzScheme does
     some things with threads if they are compiled in.  That is exactly
     what seems to be the key.

5.  I looked at the MzScheme network code in depth.  It seems to be
     correct and have no memory leaks.

6.  I ran numerous tests designed to determine if MzScheme has
     interpreter or memory management leaks.  I couldn't find any.

7.  I took my tested C socket code and linked it into MzScheme so that
     scheme functions could use my tested socket code instead of
     MzScheme's.  The leak remained even though the C code by itself
     proved solid.  However, if I changed my functions to do nothing
     (no real socket calls but keep all the scheme and scheme/c
     interface code) there was no leak.  This helped me validate that
     it wasn't the interpreter or my interface code.


I am using this application in a production environment so this is a
very very significant problem for me.  Any help or pointers are
greatly appreciated.

Sincerely,

Blake McBride
blake at integra-online.com