[racket] How to reduce my Racket web server's memory usage?

On Mon, Feb 2, 2015 at 5:06 PM, George Neuner <gneuner2 at comcast.net> wrote:
> On 2/2/2015 12:58 PM, Jay McCarthy wrote:
>
> I think 80MB is pretty typical. If you convert to "#lang web-server",
> you should be able to trade RAM for disk space, but I doubt you could
> get less than 40MB and 80MB out of 512M is pretty decent.
>
> Jay
>
>
> Would that difference all be from not caching continuations, or is there
> something else?

Yes, the "#lang web-server" continuations can be serialized to disk
and not stored in memory.

> On a related note, does sharing a file through  #:servlet-namespace
> actually save memory, or is it just about putting names into the custodian
> environment?  I'm a bit unclear as to whether custodians really manage code
> (ala Java loaders) or just namespaces.   I have a couple of large utility
> files shared this way, but since I still have to require them everywhere and
> you have mentioned previously (IIUC) that servlet instances run under
> separate custodians, I am wondering just how it works.

The only namespace savings would come from global variables (including
closures) in the modules being initialized one time. I doubt this
would be a significant source of savings unless there were A LOT of
servlets.

> Thus far I've been using #lang racket.  I haven't bumped into memory
> limitations, but the executable footprint  (resident+swap) is ~260MB and
> growing.

I think you might see improvement by using "#lang racket/base"

> Currently I have no continuation servlets - all are run to
> completion - and so right now I have  #:stateless? #t   and  #:manager
> (create-none-manager #f)  in the call to serve/servlet, but there are 43
> servlets and counting, plus several auxiliary threads and a large(ish) dbms
> connection pool.  Probably the number of servlets will double before I'm
> done.  I have managed to avoid using continuations via SQL wizardry, but I
> don't know how far I can push that - the web designer keeps wanting to shift
> more session management responsibility onto the server side.

That sounds like a lot of servlets. I think I would try to consolidate.

>
> I haven't played with  #lang web-server  at all, so forgive me if these are
> stupid questions:
>
> Where I have servlets that share auxiliary functions I have grouped them
> together in a single source file.  From the docs it appears that the
> (provide version stuffer start)  3 argument form is special vs the normal
> n-argument module provide.

It is not, but the names are special when loaded with the
plt-web-server cmdline tool. (So you can just use serve/servlet or
something else from web-server/web-server.)

>  Can there be multiple web language servlets with
> different specs provided from a single source file?

Yes, provided you don't load them with the cmdline tool, but even in
that case, there can just be one level of indirection.

> Does switching to the web language impact using serve/servlet and
> dispatch-rules?

You can still use those, although there is only a benefit when you are
using continuations via send/suspend and send/suspend/dispatch.

> Can web language and racket language servlets be mixed in the same
> application (gradual transition) or is it all or nothing?

They can be gradually mixed. That was the main theoretical challenge
after serializing the continuations at all.

Jay

> Thanks,
> George
>
>



-- 
Jay McCarthy
http://jeapostrophe.github.io

           "Wherefore, be not weary in well-doing,
      for ye are laying the foundation of a great work.
And out of small things proceedeth that which is great."
                          - D&C 64:33