[plt-scheme] Building a proxy in plt-scheme

Thomas-Xavier MARTIN <txm at m4x.org> writes at 10:05 19-Oct-2004 +0200:
> I would be grateful for any advice or suggestions on how to
> architecture such a project in PLT-Scheme, which libraries and tools I
> should look at, etc.

One of the practical considerations with HTTP proxies is that they
absolutely have to work (not crash, not hang, not slow things down, not
leak memory) or they quickly become intolerable to the user.  I first
found this when using a Java-based HTTP proxy framework, Muffin, which
(at least in 1997) had unacceptably imperfect reliability.  The bugs
made me stop using my own little tool, which was otherwise very useful
(keyword searches on content of Web pages you'd seen at some specified
point in the past).

If I were going to do this as a logging-only proxy in pure PLT Scheme
without using any C, I would expect to spend more development time on
efficient I/O buffers than on the HTTP protocol handling, since fast I/O
is important and HTTP passthrough is easy.  As I passed HTTP traffic
through with minimal rewriting, I'd log it into a fast SQL database as
large text fields and blobs.  Perhaps separate tables for each of
Request, ResponseHeaders, ResponseBody (with a one-to-many relation
between ResponseHeader and ResponseBody that can be established as a
background task that just does a byte-for-byte comparison of
ResponseBody blobs ).  Note that ResponseBody blobs won't always contain
the entire requested object.  I would probably do any necessary assembly
of multiple ResponseBody blobs at the time they're needed by an
application, not try to do this while logging.

(This was actually very much on my TODO list for once I found funding
for my Internet HCI research.  But alas, it looks like the funding
situation that I'm currently pursuing mandates Java.)

I think you could also try using Apache in proxy mode, having it call
your Scheme code.  Offhand, I don't know the details of how to do this
in the current versions of Apache.

-- 
                                             http://www.neilvandyke.org/