[plt-scheme] web-server memory use

Thanks for you help so far, guys. Jay - could you send me the code  
you knocked up with Matthew's make-cust-box, please? I'm not sure I  
understand what you're talking about in terms of implementation.

It looks as if there are at least two angles to this:



-- How to get memory accounting working in web-server.

It looks as if this won't be too difficult to achieve. Even without  
the direct implementation Matthew was talking about, this would be  
very useful as a debugging tool to determine how memory-heavy our  
code is.

Using only (current-memory-use), it's difficult to see how much  
memory is taken up by permanent data in  instances and continuations  
and how much is just out-of-scope variables waiting to be collected.



-- How to make sure LRU behaves "well".

I'm not sure whether or not you were saying there was a problem with  
this. I haven't really got the full picture in my head yet.

The LRU in our web-server uses the value returned by (current-memory- 
use) to determine the rate at which it needs to kill off  
continuations. If memory is not freed up properly when continuations  
are deleted, then (current-memory-use) will increase regardless of  
LRU's efforts. As far as I can see this all works, but I haven't done  
any real stress testing yet.

In my investigations, I noticed that LRU deletes continuations but  
never instances. Once all continuations are removed from an instance,  
it just sits there in the hash table. I don't know if this would  
cause a big drain on memory... I haven't ascertained yet what the  
instance structure's data field refers to.



Past experiences have hinted that our software for Queen Mary,  
University of London may be taxing web-server more than other  
applications that I've heard about. This makes me nervous about our  
problems with continuation expiry. Our next release will contain a  
number of critical pages, and we can't afford to have them go wrong.  
I'm keen to get these issues resolved and make things as stable as  
possible. We've exchanged a number of ideas about this here at  
Untyped, but we need to work out exactly what's happening before we  
can make an informed decision.

Let me know what I can do to help.

Many Thanks,

-- Dave