[racket] FFI: problems using (_list i _string)

From: keydana at gmx.de (keydana at gmx.de)
Date: Mon Jun 6 16:55:54 EDT 2011

Hi Thomas,

thanks so much for your introduction to memory management in FFI programming, which was very informative, concise and helpful indeed!
It gives me a perfect starting point for exploring the topic.
For the beginning, I think I'd be well advised, then, to explicitly call the ...Free() functions provided by the c library, and later on,  I will experiment with the decorator strategy.
As regards the more complicated  cases you mention, I will no doubt stumble over them sooner or later...

For now, thanks a lot again for taking the time, and writing things up so well :-)

Am 06.06.2011 um 00:56 schrieb Thomas Chust:

> 2011/6/5 keydana at gmx.de <keydana at gmx.de>:
>> [...]
>> In some example c code available, the argument actually is a
>> two-dimensional array of chars, which then of course is passed as a
>> pointer to char, and I guess the separation of strings is achieved
>> by the null-termination of c strings then...
> Hello Sigrid,
> in C, strings are almost always zero terminated, yes. However, a two
> dimensional array that doesn't consist of string pointers but the
> string data itself will necessarily contain fixed length strings in C,
> possibly padded with zero bytes in case the actual payload for some
> items is smaller than the available space.
>> [...]
>> I wonder how I am going to do this in racket, should I append null
>> to every string, string-append the strings and then pass a single
>> string to the function?
>> [...]
> Yes, I would say this is a perfectly fine solution. To convert a list
> of strings to something that can be used as an array of fixed width
> items try something like this module:
> #lang racket/base
> (require
>  srfi/13
>  srfi/26)
> (define (string-list->fixed-width-array width items)
>   (string-join (map (cut string-pad-right <> width #\null) items)))
> (provide
>  (all-defined-out))
>> [...]
>> Regarding racket garbage collection, on the one hand, and c pointer
>> freeing, that is generally a topic I'm very unsure about and could
>> perhaps use some "basic" advice (more basic  than the FFI reference,
>> I mean).  For example, do I have to explicitly "destroy"  c pointers
>> somehow, and if so, when? In the case of my c library given, I
>> assume I have to implement every function like '<object>Free' and
>> call it when I'm done, but apart from that?
>> [...]
> Ok, I'll try to sum up a few basics: First it's important to note that
> values allocated by Racket and blocks allocated by C code live in
> different memory areas and are handled differently. While Racket
> values are reclaimed automatically by the garbage collector some time
> after they are no longer referenced, blocks allocated from C live
> forever unless explicitly destroyed. These two worlds know nothing
> about each other by default.
> The pointers your code handles never have to be destroyed explicitly,
> but the objects they point to may have to be destroyed. There is no
> simple general rule when and how this has to happen. It all depends on
> the way the C code is designed and what mechanisms it uses for memory
> management.
> In the most simple situation, you do not use any C functions that
> allocate and store or return (pointers to) blocks of memory, so you
> can just let all the data live in the memory area managed by Racket
> and let the garbage collector do its job without caring about manually
> freeing any blocks of memory. C structures can be allocated through
> Racket's garbage collector, too, and they will be reclaimed just like
> Racket values some time after all pointers referring to them go out of
> scope. Look at the documentation of malloc in the ffi/unsafe module
> for details about memory allocation from the Racket side.
> Another case encountered frequently is that there are C functions
> creating, operating on and destroying pointers to some opaque data
> structure. These structures do not live in Racket's managed memory, so
> calls to the creation functions have to be matched by calls to the
> destruction functions or the objects will stay in memory until the
> process terminates.
> In this situation you have two basic choices: You can either provide
> bindings to the functions that create and destroy objects and require
> any client code to handle memory management of those objects
> explicitly, manually destroying them when they are no longer
> needed. Or you can set up an automatism in Racket that ensures all
> those objects created by calls from Racket are freed some time after
> Racket code no longer holds any references to them. An easy way to
> achieve this is by decorating the bindings for the creation and
> destruction procedures using allocator and deallocator from the
> ffi/unsafe/alloc module. However, care has to be taken that no code
> outside Racket is still using the objects when they are reclaimed by
> the Racket garbage collector. Some C libraries use reference counting
> to better handle the situation that different pieces of code may hold
> references to an object for random periods of time; the
> ffi/unsafe/alloc module also has support for that situation.
> Things get progressively more complicated if the C structures put
> inside Racket's memory areas contain pointers themselves, if
> structures outside Racket's memory areas contain pointers to Racket
> values, or if callbacks from C into Racket are used.
>> [...]
>> I hope I didn't ask too many too basic questions now, but never
>> having done any c programming and just knowing the concepts
>> "theoretically", there a quite some things about using FFI I don't
>> automatically understand from the Reference...
>> [...]
> Among all the FFIs I have used, Racket's is one of the most
> comfortable and full featured ones. However, its features imply
> complexity, too, and I can well imagine that without any prior
> experience in low level programming and manual memory management, one
> can easily get lost. So questions have to be expected :-)
> Ciao,
> Thomas
> -- 
> When C++ is your hammer, every problem looks like your thumb.

Posted on the users mailing list.