[plt-scheme] overhead of lots of dynamic closure creation in high-performance environment?
Is there any rule-of-thumb I could use when reasoning about the
performance implications of using lots of dynamic creation of closures
and one-shot application of them?
I need high-performance processing of a protocol that is chunked at
arbitrary points across packets in a lower protocol layer. I want to
integrate the parsing of the higher-level protocol into that of the
lower-level protocol, rather than assembling the lower-level chunks and
doing a second pass for the higher-level.
Doing this efficiently in a C-like style requires some headache-inducing
record-keeping and code.
Doing this by dynamically creating a closure (for immediate application,
or to be stuck in a hash table keyed on the stream ID), which in turn
dynamically creates another closure, and so on until the protocol is
consumed, looks like it will be pretty elegant. And efficient, if
closure creation is inexpensive.
For example:
(letrec (;; ...
(make-complete-name-paramsproc
(lambda (params name val-size)
(lambda (content content-i)
;; !!! do some processing and return a closure created by
;; make-partial-val-paramsproc
)))
(make-partial-val-paramsproc
(lambda (params name buf used)
(lambda (content content-i)
;; !!!
)))
;;...
So, for each Web page served, this part of the processing would
typically involve, say, 30 closures created dynamically, applied once,
and immediately left for GC.
The entire purpose of this mechanism that would use dynamic closures is
to greatly *speed up* Web serving, so I'm looking for a better intuition
for how efficient dynamic closures are this is before I code it all.
Thanks,
Neil
--
http://www.neilvandyke.org/