[racket-dev] Memory grows without bound compiling tool-lib-extract.rkt

At Thu, 3 Jun 2010 13:00:45 -0400, Eli Barzilay wrote:
> On Jun  3, Sam Tobin-Hochstadt wrote:
> > 
> > A quick `git bisect' (a great tool, btw) suggests that commit
> > ac69f11 caused the problem.
> 
> That's unlikely to be the cause -- only exposing some other bug.

Right. Although `bisect' correctly pointed to the immediate cause,
it wasn't the real problem.

The most direct cause of the problem was the change in DrRacket's
implementation to make some unit signatures bigger and to build them
through a longer chain of `open's. The expansion of `unit' forms with
signatures that are created by chaining `open's turns out to be
especially verbose (lots of renamings that lead to other renamings),
but that's just the start.

The solution involved changes at several layers to reduce quadratic
behavior that feeds into other quadratic behavior --- much harder to
deal with than an exponential algorithm at a single point in the
sequence. Mostly, it took me a long time to work through the layers and
find the last problem layer (in the syntax-object representation),
which was arguably the main offender.

I've pushed a repair.