[racket-dev] RFC: Coding Guidelines

On Thu, Aug 19, 2010 at 9:01 AM, Sam Tobin-Hochstadt <samth at ccs.neu.edu> wrote:
> On Thu, Aug 19, 2010 at 9:48 AM, Robby Findler
> <robby at eecs.northwestern.edu> wrote:
>> On Thu, Aug 19, 2010 at 8:44 AM, Sam Tobin-Hochstadt <samth at ccs.neu.edu> wrote:
>>> On Tue, Aug 17, 2010 at 3:57 PM, Jay McCarthy <jay.mccarthy at gmail.com> wrote:
>>>> Please comment.
>>>
>>> I think that this:
>>>
>>> "Your first task when changing old code is to build an adequate test
>>> suite to ensure you do not introduce new mistakes as you attempt to
>>> improve it. Thank you for improving the world for future generations!"
>>>
>>> is too demanding.  There are enormous areas of our code that don't
>>> have a test suite.  How comprehensive a test suite do I need before
>>> changing slideshow?  Or scribble (which has a test suite for the
>>> syntax, but not the language)?
>>
>> You should make a reasonable effort to put in a test case or two for
>> the specific functionality that you're adding/modifying. If there is
>> no test suite and you aren't sure how to build one, then ask, see what
>> responses you get, and go from there.
>
> So, to make this concrete, I recently made some small changes to the
> scriblib/autobib library.  These included changing the regexp to allow
> hyphens in last names, as well as allowing multiple citations at once
> in a particular special case.  I tested these changes by running a
> not-then-published paper that I was working on, and seeing that it did
> the right thing.  Then I sent the patch to Matthew, and he said it
> looked good.  Then I committed it.
>
> There's two forms of automated test that I could have added.  I could
> have just added a file the used the functionality, and checked that it
> didn't error when run.  That would have been a minimal test, and
> wouldn't even have exercised the bug fix (it was just parsing my name
> wrong).   Or I could have written a test framework for Scribble PDF
> rendering.  I'm not sure how that would have worked, and it would have
> been a big programming investment.  I don't think I had time then (or
> now) to do that work.

This example sounds to me like you could have written a call to a
scriblib function and checked that there was no error (for the
second). For the first, you probably would have had to expose some
functionality to make the bib record give you back the sorted list of
names (or something like that) that it would have used when rendering.

In general, I'd say that a big piece of good test suite design is
figuring out how to add observations to some library you want to test
so that you can test it only and not test things that it depends on
(pdf rendering in this case).

Robby