[plt-dev] lessons learned

Sad fact: I agree with you and have brought up the idea in our world  
even before you moved to Google.

Sadder fact is that this bug would have been caught with an  
evaluation. With a require in a mred repl. Done. No tests required.  
NONE. (When I modify someone else's code, I make sure to at least  
evaluate the definition. Usually I also ask where the test suite is.)

Saddest fact is that I was stupid enough to think that our nightly  
tests actually executed each piece of code once. That would have  
revealed the code. I am just totally stupid and naive here.


On May 22, 2009, at 12:22 PM, Jacob Matthews wrote:

> I've said it before and I'll say it again: Set up a continuous
> integration engine. Run the PLT Scheme test suites _all the time_
> (every checkin if possible --- this lets you isolate bugs to
> individual svn submits or small ranges of them). Automatically measure
> code coverage when the tests run.
>
> Your tests can catch problems like the one you discovered when they're
> submitted rather than months later, but only if (1) you have tests,
> (2) they actually exercise your code, and (3) you run them often. In
> practice if you've got #3 you can measure #2, and measurement of #2
> drives you to do a better job of #1.
>
> You might want to look at tools like Hudson and CruiseControl for
> ideas, though honestly it's probably easier to write your own ---
> other tools would require you to write plugins in order to support PLT
> Scheme anyway, and the core functionality is dead simple: it's just a
> checkout-build-test loop.
>
> -jacob
>
>
> On Thu, May 21, 2009 at 10:43 AM, Matthias Felleisen
> <matthias at ccs.neu.edu> wrote:
>>
>> Today's chase for the source of
>>
>>   define-compound-unit/infer: untagged initialization dependent
>>   signature mred^ is supplied from a later unit with link temp4
>>
>> cost me an hour of my time when a few seconds of common sense  
>> would have
>> sufficed. I don't have such hours to waste.
>>
>> ;; ---
>>
>> LESSON 1: When I gave the POPL keynote a few years ago about our  
>> project,
>> every single part of the talk had a single lesson:
>>
>>      **************************
>>      * ERROR MESSAGES MATTER. *
>>      **************************
>>
>> I just can't see how the above error message would point anyone to  
>> this
>> specification
>>
>>> (define-values/invoke-unit/infer
>>>  (export graphics^)
>>>  (link standard-mred@ graphics-posn-less@))
>>
>> on line 15 of big-draw.ss. The name of the construct is different  
>> from that
>> in the error message. And the message itself is gobbldedygook.
>>
>> ;; ---
>>
>> LESSON 2: People recently (27 Mar 2009) changed (see svn log, svn  
>> blame)
>> these lines, and didn't even both to run the file as they did.
>>
>>      *********************************
>>      * WHEN YOU EDIT A FILE, RUN IT. *
>>      *********************************
>>
>> It takes a cmd-t or a click-run and it evaluates in 3 seconds.  
>> Doing so
>> would have revealed the bug. 3 SECONDS, 1 HOUR.
>>
>> ;; ---
>>
>> LESSON 3: When you do synthesize such language constructs, put the  
>> effort in
>> to get them right. A simple topological sort would probably avoid  
>> all this
>> nonsense.
>>
>> ;; ---
>>
>> Guys, this shop is not the federal government. We can't afford this
>> sloppiness -- Matthias
>>
>>
>>
>>
>>
>> _________________________________________________
>>  For list-related administrative tasks:
>>  http://list.cs.brown.edu/mailman/listinfo/plt-dev
>>
>>