[plt-scheme] Variable-length C structures in MzScheme

On Wed, 25 Aug 2004 14:52:13 -0600, Matthew Flatt <mflatt at cs.utah.edu> wrote:
> Sorry for the delay --- I didn't see this message until just now.

No problem.
 
> At Wed, 11 Aug 2004 16:06:37 +0900, Daniel Silva wrote:
> > You would insert objects with array_insert:
> >
> > Scheme_Object* array_insert(int argc, Scheme_Object* argv[])
> > {
> >   MySchemeArrayType* arr = (MySchemeArrayType*)
> >   arr->items = scheme_realloc(arr->items,
> >                               (arr->len + 1) * sizeof(Scheme_Object*));
> >   arr->items[arr->len] = argv[0];
> >   arr->len++;
> >   return scheme_void;
> > }
> >
> > The scheme_realloc function uses memcpy to fill in old values in the
> > new, larger items C array.
> >
> > I have a situation like this and after some time (and I guess a few GC
> > runs), the pointers in items[] point to invalid memory addresses.  Is
> > it that the GC thinks the array object is smaller than it really is,
> > and is then not seeing the pointers in items[]?
> 
> No, I don't think that could happen.
> 
> Is this still an issue? If so, I think I'd like to see the real code,
> including scheme_realloc(), because so many little things can go wrong
> in C.

I haven't changed anything, so it's still crashing.  The code I posted
was just a fake example (and yep, I meant argv[1]).  Here's the
scheme_realloc:

void* scheme_realloc(void* o, size_t new_size)
{
  int amt_to_copy;
  void *new_buffer;
  if ( new_size == 0 )
    new_size = 1;
  if ( !o )
   return scheme_malloc(new_size);

  // how can I get the old size? should be min(new_size, old_size)
  // dangerous if amt_to_copy is bigger than the old size...
  amt_to_copy = new_size;

  new_buffer = scheme_malloc(new_size);
  if ( amt_to_copy )
    memcpy(new_buffer, o, amt_to_copy);
  return new_buffer;
}

and the code that uses it:

// my code
#define PyMem_REALLOC scheme_realloc

// (define-struct py-object (ob_type))

#define PyObject_HEAD \
        Scheme_Type scheme_type; \
        MZ_HASH_KEY_EX \
        void* stype; \
        PYTYPEOBJECT * ob_type;

#define PyObject_VAR_HEAD \
                PyObject_HEAD \
                int ob_size;


// CPython code  (ins1 is their implementation of list_insert)

#define PyMem_RESIZE(p, type, n) \
        ( (p) = (type *) PyMem_REALLOC((p), (n) * sizeof(type)) )

void nresize(PyObject* var, int nitems)
{
        size_t _new_size = roundupsize(nitems);
        if (_new_size <= ((~(size_t)0) / sizeof(var)))
                PyMem_RESIZE(var, PyObject*, _new_size);
        else
                var = NULL;
}

typedef struct {
    PyObject_HEAD
}; PyObject;

typedef struct {
    PyObject_VAR_HEAD
    PyObject **ob_item;
} PyListObject;


static int
ins1(PyListObject *self, int where, PyObject *v)
{
  int i;
  PyObject **items;
  if (v == NULL) { PyErr_BadInternalCall();
    return -1;
  }
  if (self->ob_size == INT_MAX) { PyErr_OverFlowError();
    return -1;
  }
  items = self->ob_item;
  nresize(items, PyObject *, self->ob_size+1);
....
}

So far scheme_realloc is only being called to grow arrays, so I don't
think it's overwriting anything it shouldn't with that memcpy.

Daniel