[racket-dev] FFI: pointer to an array in a C struct type

On 12/03/2012 12:10 PM, Matthew Flatt wrote:
> At Mon, 03 Dec 2012 11:05:10 -0700, Neil Toronto wrote:
>> On 12/03/2012 07:31 AM, Matthew Flatt wrote:
>>> Neil, can you say more about how `_mpz' instances are used with foreign
>>> functions?
>>
>> They represent GMP's bignums, and they're used as both input and output
>> arguments.
>>
>> When used as input arguments, GMP's functions never mutate them. It
>> should be safe to pass an `mpz' that contains a pointer to memory that
>> Racket manages, as long as it doesn't get relocated during the function
>> call. In this case, I need to be able to set the `limbs' array's
>> elements in Racket code.
>>
>> When used as output arguments, GMP expects to be able to free the array
>> pointed at by the `limbs' field, allocate a new array for the result,
>> and set the `limbs' field to point to it. I use the GMP function
>> "mpz_init" to initialize the fields of an `mpz' instance before using it
>> as an output argument. In this case, I need to be able to read the
>> `limbs' array's elements in Racket code.
>
> Overall, it sounds to me like you should `malloc' a `limbs' array in
> 'raw mode, use `free' to free it, use `_pointer' as the ctype for the
> `limbs' field, and so on. If you need something like finalization, use
> `ffi/unsafe/alloc' to make sure that a freeing function for a `_mpz' is
> paired with an `_mpz' allocation.

That's what I've had in the past, except I used 'atomic-interior instead 
of 'raw. I also had the `_mpz' instances allocated using 
'atomic-interior. IIRC, the limbs didn't survive a GC. Now that I think 
about it, that was probably because the `_mpz' instances were allocated 
using "atomic"...

I've been avoiding 'raw generally, in case the foreign libraries have 
their own allocation schemes. MPFR has its own special malloc and free 
for character strings, for example. It's weird.

But I've just discovered mpz_init2, which allocates enough limbs to 
store an n-bit number. Between that and mpz_clear, I've basically got 
GMP's malloc and free. This is my code now:


(define-cstruct _mpz ([alloc _int] [size _int] [limbs _pointer]))

(define mpz-init
  (get-gmp-fun '__gmpz_init (_fun _mpz-pointer -> _void)))
(define mpz-init2
  (get-gmp-fun '__gmpz_init2 (_fun _mpz-pointer _ulong -> _void)))
(define mpz-clear
  (get-gmp-fun '__gmpz_clear (_fun _mpz-pointer -> _void)))

(define (new-mpz [bits 0])
   (define z (ptr-ref (malloc _mpz 'atomic-interior) _mpz))
   (if (= bits 0) (mpz-init z) (mpz-init2 z bits))
   (register-finalizer z mpz-clear)
   z)


I think that return values from `new-mpz' won't be relocated (interior) 
and won't be traced (atomic). I've verified that the limbs are freed 
when their `_mpz' instances are collected, by allocating one with 200MB 
of limbs, filling the limbs to page them in, getting the `_mpz' instance 
collected, and watching Racket's memory usage drop.

The downside to this is that Racket doesn't know about the memory GMP 
manages. The 200MB of limbs only showed up in my system monitor, not in 
DrRacket's memory indicator. I'm only using them for temporary values, 
though.

(FWIW, I've managed to get bigfloats' limbs managed by Racket, using 
(_gcable _cvector) and malloc with 'atomic-interior. I never need to set 
or ref those limbs in Racket code.)

> Just to me clear, even if it somehow worked to use `_gcpointer' as the
> ctype of a field in an `_mpz' that you allocate, that would be the
> wrong ctype for an `_mpz' that was filled in by `mpz_init' or updated
> as a result.

Gotcha. If I wanted to have Racket manage an `_mpz' instance's limbs, 
then, I'd need a different ctype for that.

Neil ⊥