[plt-dev] lessons learned

I will take charge of doing this if no-one else is dying to do it.

I'll plan on running all the code and the test cases. If there are
failures I'll send mail to the committer and the person listed on
plt:resposibile. I'll figure out some way for you to put in properties
or info files what shouldn't be run or what is a test cases, etc.

If you have any ideas / comments, let me know.

Jay

On Fri, May 22, 2009 at 12:32 PM, Matthias Felleisen
<matthias at ccs.neu.edu> wrote:
>
> Sad fact: I agree with you and have brought up the idea in our world even
> before you moved to Google.
>
> Sadder fact is that this bug would have been caught with an evaluation. With
> a require in a mred repl. Done. No tests required. NONE. (When I modify
> someone else's code, I make sure to at least evaluate the definition.
> Usually I also ask where the test suite is.)
>
> Saddest fact is that I was stupid enough to think that our nightly tests
> actually executed each piece of code once. That would have revealed the
> code. I am just totally stupid and naive here.
>
>
> On May 22, 2009, at 12:22 PM, Jacob Matthews wrote:
>
>> I've said it before and I'll say it again: Set up a continuous
>> integration engine. Run the PLT Scheme test suites _all the time_
>> (every checkin if possible --- this lets you isolate bugs to
>> individual svn submits or small ranges of them). Automatically measure
>> code coverage when the tests run.
>>
>> Your tests can catch problems like the one you discovered when they're
>> submitted rather than months later, but only if (1) you have tests,
>> (2) they actually exercise your code, and (3) you run them often. In
>> practice if you've got #3 you can measure #2, and measurement of #2
>> drives you to do a better job of #1.
>>
>> You might want to look at tools like Hudson and CruiseControl for
>> ideas, though honestly it's probably easier to write your own ---
>> other tools would require you to write plugins in order to support PLT
>> Scheme anyway, and the core functionality is dead simple: it's just a
>> checkout-build-test loop.
>>
>> -jacob
>>
>>
>> On Thu, May 21, 2009 at 10:43 AM, Matthias Felleisen
>> <matthias at ccs.neu.edu> wrote:
>>>
>>> Today's chase for the source of
>>>
>>>  define-compound-unit/infer: untagged initialization dependent
>>>  signature mred^ is supplied from a later unit with link temp4
>>>
>>> cost me an hour of my time when a few seconds of common sense would have
>>> sufficed. I don't have such hours to waste.
>>>
>>> ;; ---
>>>
>>> LESSON 1: When I gave the POPL keynote a few years ago about our project,
>>> every single part of the talk had a single lesson:
>>>
>>>     **************************
>>>     * ERROR MESSAGES MATTER. *
>>>     **************************
>>>
>>> I just can't see how the above error message would point anyone to this
>>> specification
>>>
>>>> (define-values/invoke-unit/infer
>>>>  (export graphics^)
>>>>  (link standard-mred@ graphics-posn-less@))
>>>
>>> on line 15 of big-draw.ss. The name of the construct is different from
>>> that
>>> in the error message. And the message itself is gobbldedygook.
>>>
>>> ;; ---
>>>
>>> LESSON 2: People recently (27 Mar 2009) changed (see svn log, svn blame)
>>> these lines, and didn't even both to run the file as they did.
>>>
>>>     *********************************
>>>     * WHEN YOU EDIT A FILE, RUN IT. *
>>>     *********************************
>>>
>>> It takes a cmd-t or a click-run and it evaluates in 3 seconds. Doing so
>>> would have revealed the bug. 3 SECONDS, 1 HOUR.
>>>
>>> ;; ---
>>>
>>> LESSON 3: When you do synthesize such language constructs, put the effort
>>> in
>>> to get them right. A simple topological sort would probably avoid all
>>> this
>>> nonsense.
>>>
>>> ;; ---
>>>
>>> Guys, this shop is not the federal government. We can't afford this
>>> sloppiness -- Matthias
>>>
>>>
>>>
>>>
>>>
>>> _________________________________________________
>>>  For list-related administrative tasks:
>>>  http://list.cs.brown.edu/mailman/listinfo/plt-dev
>>>
>>>
>
> _________________________________________________
>  For list-related administrative tasks:
>  http://list.cs.brown.edu/mailman/listinfo/plt-dev
>



-- 
Jay McCarthy <jay at cs.byu.edu>
Assistant Professor / Brigham Young University
http://teammccarthy.org/jay

"The glory of God is Intelligence" - D&C 93