[racket] SSL and db/postgresql performance

On 05/11/2013 05:20 PM, Neil Van Dyke wrote:
> Ryan Culpepper wrote at 05/10/2013 08:02 PM:
>> The problem might be in the way Racket's SSL ports do buffering. If I
>> recall a past experiment correctly, each write to an SSL port probably
>> creates a separate SSL "record", where each record is separately
>> MAC'd, padded, and encrypted. (The SSL port is still "buffered" in the
>> sense that the records are only sent over the underlying output port
>> when the SSL port is flushed, though.) My implementation of the
>> PostgreSQL wire protocol uses lots of small writes, so that could be
>> the source of the problem. I could be wrong; you should verify this
>> using ssldump or something similar.
>
> Looks like this is probably what's happening.  After the connection is
> negotiated, a query like "select * from mytable" causes a large
> sequences of client->server SSL records to be sent.  For example: 16
> C>S, 2 S>C, 34 C>S.
>
> For a test case that opens a connection and does 100 simple queries,
> using the "db" library with SSL took approx. 7 times as long as using a
> "pqsl" script to to do the same thing with SSL.  Seems plausible to me
> that the fragmentation of messages across so many SSL records could
> account for a 7x difference.
>
>> If that does turn out to be a problem, a quick fix would be to create
>> a helper port that does its own buffering and only writes to the SSL
>> port when the helper port is flushed. If that significantly improves
>> the performance, then we should probably fix the SSL library.
>> (Although there might be issues I'm not thinking of that make that
>> harder.)
>
> Do you mean that I would modify the "db" library source code?  (I don't
> see a public interface that would permit me to add a buffering port,
> such as supplying such a port to "postgresql-connect".)

Yes. I just pushed a workaround.

Ryan