[plt-scheme] Segfault in web server (due to missing library??)

From: Geoffrey S. Knauth (geoff at knauth.org)
Date: Fri Oct 30 21:47:04 EDT 2009

On Oct 30, 2009, at 20:51, David Storrs wrote:

> Ok, we've got a bit more information.  Here's a test file (we have  
> it saved as "test-hmac.c"):
> #include <openssl/hmac.h>
> #include <stdio.h>
> int main(int argc, char** argv) {
>   unsigned char foo[10] = "boo";
>   unsigned char* res = malloc(20);
>   printf("%s\n", SHA1(foo, 10, res));
>   free(res);
> }
> We built this on the Debian 5.0 system in question, using gcc - 
> lcrypto test-hmac.c.  If we run it as-is, it works.  If we change:
>   printf("%s\n", SHA1(foo, 10, res));
> to
>   printf("%s\n", SHA1(foo, 10, 0));
> ...it segfaults.  Here is the code from web-server/stuffers/hmac- 
> sha1.ss:
> (define HMAC-SHA1/raw
>   (if libcrypto
>       (get-ffi-obj 'HMAC libcrypto
>                    (_fun [EVP_MD : _fpointer = (EVP_SHA1)]
>                          [key : _bytes]
>                          [key_len : _int = (bytes-length key)]
>                          [data : _bytes]
>                          [data_len : _int = (bytes-length data)]
>                          [md : _int = 0]
>                          [md_len : _int = 0]     ;;  @@@@ Note the  
> "0" here.
>                          f->
>                          _pointer))
>       (lambda (key data) (error 'HMAC-SHA1/raw "libcrypto could not  
> load"))))
> The last argument to the SHA1() function is where to put the  
> result.  When passed a NULL, it allocates its own return space.  We  
> are guessing that, on the Debian box, NULL is defined to something  
> other than 0, so when it receives a literal 0, it tries to write to  
> 0x0 and segfaults.
> We're trying to track down where NULL is defined now.

I looked at the source to SHA1:

unsigned char *SHA1(const unsigned char *d, size_t n, unsigned char *md)
	static unsigned char m[SHA_DIGEST_LENGTH];

	if (md == NULL) md=m;
	if (!SHA1_Init(&c))
		return NULL;

If md is not specified, the pointer returned is a static buffer inside  
the function.  I don't know if that presents the FFI with a problem.   
That doesn't explain the crash in your pure C program.

On my Mac OS X 10.6 and Ubuntu 9.04 systems, I had to add #include  
<openssl/sha.h> to the C file, else the compiler thought SHA1 returned  
an int.  On both systems, both versions of your program ran fine and  
returned identical results.

If I were in your shoes, I'd compile my open libcrypto with debug  
symbols, and step through your test program with gdb to see where the  
SIGSEGV is coming from.

I also looked at the source to HMAC referenced in your FFI code:

unsigned char *HMAC(const EVP_MD *evp_md, const void *key, int key_len,
		    const unsigned char *d, size_t n, unsigned char *md,
		    unsigned int *md_len)
	static unsigned char m[EVP_MAX_MD_SIZE];

	if (md == NULL) md=m;

My only observation is that md is unsigned char *, but your FFI code  
calls it int.

Posted on the users mailing list.