[plt-dev] lessons learned

Jay, thanks for volunteering to do something. That was truly  
positive, and it made my day.

The one thing I'd like to add to Eli's nightly build is some way of  
just evaluating each file once. Period. If it doesn't load, send a  
message and nag. Just like for the test cases. (I do have a separate  
test suite for /htdp and /2htdp, but they aren't automated.)

Over the upcoming decades, I wouldn't mind seeing some way of testing  
more, running it continuously, etc. If someone really wants to do  
this, I will get a farm or two or three of machines.

-- Matthias



On May 22, 2009, at 12:47 PM, Jay McCarthy wrote:

> I will take charge of doing this if no-one else is dying to do it.
>
> I'll plan on running all the code and the test cases. If there are
> failures I'll send mail to the committer and the person listed on
> plt:resposibile. I'll figure out some way for you to put in properties
> or info files what shouldn't be run or what is a test cases, etc.
>
> If you have any ideas / comments, let me know.
>
> Jay
>
> On Fri, May 22, 2009 at 12:32 PM, Matthias Felleisen
> <matthias at ccs.neu.edu> wrote:
>>
>> Sad fact: I agree with you and have brought up the idea in our  
>> world even
>> before you moved to Google.
>>
>> Sadder fact is that this bug would have been caught with an  
>> evaluation. With
>> a require in a mred repl. Done. No tests required. NONE. (When I  
>> modify
>> someone else's code, I make sure to at least evaluate the definition.
>> Usually I also ask where the test suite is.)
>>
>> Saddest fact is that I was stupid enough to think that our nightly  
>> tests
>> actually executed each piece of code once. That would have  
>> revealed the
>> code. I am just totally stupid and naive here.
>>
>>
>> On May 22, 2009, at 12:22 PM, Jacob Matthews wrote:
>>
>>> I've said it before and I'll say it again: Set up a continuous
>>> integration engine. Run the PLT Scheme test suites _all the time_
>>> (every checkin if possible --- this lets you isolate bugs to
>>> individual svn submits or small ranges of them). Automatically  
>>> measure
>>> code coverage when the tests run.
>>>
>>> Your tests can catch problems like the one you discovered when  
>>> they're
>>> submitted rather than months later, but only if (1) you have tests,
>>> (2) they actually exercise your code, and (3) you run them often. In
>>> practice if you've got #3 you can measure #2, and measurement of #2
>>> drives you to do a better job of #1.
>>>
>>> You might want to look at tools like Hudson and CruiseControl for
>>> ideas, though honestly it's probably easier to write your own ---
>>> other tools would require you to write plugins in order to  
>>> support PLT
>>> Scheme anyway, and the core functionality is dead simple: it's  
>>> just a
>>> checkout-build-test loop.
>>>
>>> -jacob
>>>
>>>
>>> On Thu, May 21, 2009 at 10:43 AM, Matthias Felleisen
>>> <matthias at ccs.neu.edu> wrote:
>>>>
>>>> Today's chase for the source of
>>>>
>>>>  define-compound-unit/infer: untagged initialization dependent
>>>>  signature mred^ is supplied from a later unit with link temp4
>>>>
>>>> cost me an hour of my time when a few seconds of common sense  
>>>> would have
>>>> sufficed. I don't have such hours to waste.
>>>>
>>>> ;; ---
>>>>
>>>> LESSON 1: When I gave the POPL keynote a few years ago about our  
>>>> project,
>>>> every single part of the talk had a single lesson:
>>>>
>>>>     **************************
>>>>     * ERROR MESSAGES MATTER. *
>>>>     **************************
>>>>
>>>> I just can't see how the above error message would point anyone  
>>>> to this
>>>> specification
>>>>
>>>>> (define-values/invoke-unit/infer
>>>>>  (export graphics^)
>>>>>  (link standard-mred@ graphics-posn-less@))
>>>>
>>>> on line 15 of big-draw.ss. The name of the construct is  
>>>> different from
>>>> that
>>>> in the error message. And the message itself is gobbldedygook.
>>>>
>>>> ;; ---
>>>>
>>>> LESSON 2: People recently (27 Mar 2009) changed (see svn log,  
>>>> svn blame)
>>>> these lines, and didn't even both to run the file as they did.
>>>>
>>>>     *********************************
>>>>     * WHEN YOU EDIT A FILE, RUN IT. *
>>>>     *********************************
>>>>
>>>> It takes a cmd-t or a click-run and it evaluates in 3 seconds.  
>>>> Doing so
>>>> would have revealed the bug. 3 SECONDS, 1 HOUR.
>>>>
>>>> ;; ---
>>>>
>>>> LESSON 3: When you do synthesize such language constructs, put  
>>>> the effort
>>>> in
>>>> to get them right. A simple topological sort would probably  
>>>> avoid all
>>>> this
>>>> nonsense.
>>>>
>>>> ;; ---
>>>>
>>>> Guys, this shop is not the federal government. We can't afford this
>>>> sloppiness -- Matthias
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _________________________________________________
>>>>  For list-related administrative tasks:
>>>>  http://list.cs.brown.edu/mailman/listinfo/plt-dev
>>>>
>>>>
>>
>> _________________________________________________
>>  For list-related administrative tasks:
>>  http://list.cs.brown.edu/mailman/listinfo/plt-dev
>>
>
>
>
> -- 
> Jay McCarthy <jay at cs.byu.edu>
> Assistant Professor / Brigham Young University
> http://teammccarthy.org/jay
>
> "The glory of God is Intelligence" - D&C 93