[racket] SSL and db/postgresql performance

On 05/10/2013 04:28 PM, Neil Van Dyke wrote:
> Anyone run into any unusual performance problems when using SSL with a
> "db/postgresql" connection, or have ideas why that might happen?
>
> Reason I ask is that a particular large Racket installation has reported
> a problem, and I have only a few guesses as to why.  This installation
> has two libraries for accessing PostgreSQL: an old one using a C
> extension and "libpq", and a drop-in replacement that layers over the
> Racket "db" library.  When SSL is not used, the new library gives vastly
> better performance under load, perhaps due mostly to no C mutual
> exclusion bottleneck when handling concurrent connections.  However,
> when the connections are SSL'd, the relative performance difference
> becomes reversed, and the old library with "libpq" C extension library
> becomes much faster than the one using Racket "db".  The SSL protocol
> versions should be the same.
>
> Assuming that the performance difference is not due to a configuration
> difference or experimental error... my first guess would be a huge
> latency difference, such as if "libpq" were flushing I/O and padding
> encryption blocks much more aggressively.  I'm also suddenly wondering
> whether Racket calling out to OpenSSL re-creates the C mutual exclusion
> bottleneck that we went to some trouble to get rid of.

I believe that Racket uses OpenSSL in a way that the "mutual exclusion 
bottleneck", as you call it, should be minimal. That is, OpenSSL handles 
doing the encryption, but Racket handles the I/O, so you should get the 
concurrency of Racket's ports.

The problem might be in the way Racket's SSL ports do buffering. If I 
recall a past experiment correctly, each write to an SSL port probably 
creates a separate SSL "record", where each record is separately MAC'd, 
padded, and encrypted. (The SSL port is still "buffered" in the sense 
that the records are only sent over the underlying output port when the 
SSL port is flushed, though.) My implementation of the PostgreSQL wire 
protocol uses lots of small writes, so that could be the source of the 
problem. I could be wrong; you should verify this using ssldump or 
something similar.

If that does turn out to be a problem, a quick fix would be to create a 
helper port that does its own buffering and only writes to the SSL port 
when the helper port is flushed. If that significantly improves the 
performance, then we should probably fix the SSL library. (Although 
there might be issues I'm not thinking of that make that harder.)

Ryan


> As a backup option, I'm considering using SSL tunnels as separate Linux
> processes, but there are reasons we'd like to avoid that if there is a
> reasonable solution to the SSL performance in Racket.
>
> Neil V.
>
> ____________________
>   Racket Users list:
>   http://lists.racket-lang.org/users