[plt-dev] lessons learned

That sounds great! You will find that nearly all test suites print
something, tho. I'm sure it won't be hard to change them to be mroe
kind to a simple predicate like that, tho.

I like the idea of loading each file to see if it just runs without
input (esp. now that we have a real bug that it would have caught).
Are we worried about external effects here, tho?

Also, I recently sped up test-coverage in the teaching languages by a
large factor -- this speedup assumes that the code is only run in a
single thread, which seems like a fine approximation for this job;
perhaps something to think about eventually including?

Robby

On Fri, May 22, 2009 at 2:26 PM, Matthias Felleisen
<matthias at ccs.neu.edu> wrote:
>
> I like your idea a lot. (I am sure you realize that some files need to be
> loaded in mred.) Keep us posted about the progress -- Matthias
>
>
>
>
> On May 22, 2009, at 3:09 PM, Jay McCarthy wrote:
>
>> On Fri, May 22, 2009 at 2:02 PM, Matthias Felleisen
>> <matthias at ccs.neu.edu> wrote:
>>>
>>> The one thing I'd like to add to Eli's nightly build is some way of just
>>> evaluating each file once. Period. If it doesn't load, send a message and
>>> nag. Just like for the test cases. (I do have a separate test suite for
>>> /htdp and /2htdp, but they aren't automated.)
>>
>> It seems like Eli's build is doing two things: producing the binaries
>> for download and doing the automated tests linked in tests/. It makes
>> sense to the first only nightly, but the second every commit.
>>
>> I can buy a machine and have it do the testing part basically 24/7 at
>> BYU. I'll have it re-compile mzscheme only when the C code changes and
>> the byte-code always. Then I'll load each file (which will include the
>> tests, of course) and send blame to the most recent committer and the
>> owner (via plt:responsible) of the file.
>>
>> I'll have a web page that shows the success of each file's loading and
>> any messages spit out when it runs. At the beginning I can be testing
>> library agnostic if I can assume any output to stderr is an error.
>>
>> I think that will be a lot better than what we have now and then we
>> can experiment with better ways of communicating errors and coverage
>> to my setup. For example, Eli is referring to errortrace as if it is
>> the only option. I reckon if we just want to know whether some line of
>> a file has been run we could get by with less information than is
>> provided by errortrace.
>>
>> Anyways... that's my plan. Before I turn it live and you start getting
>> obnoxious emails, I'll get more permission.
>>
>> Jay
>>
>> --
>> Jay McCarthy <jay at cs.byu.edu>
>> Assistant Professor / Brigham Young University
>> http://teammccarthy.org/jay
>>
>> "The glory of God is Intelligence" - D&C 93
>
> _________________________________________________
>  For list-related administrative tasks:
>  http://list.cs.brown.edu/mailman/listinfo/plt-dev
>