[plt-dev] lessons learned

On Fri, May 22, 2009 at 2:02 PM, Matthias Felleisen
<matthias at ccs.neu.edu> wrote:
> The one thing I'd like to add to Eli's nightly build is some way of just
> evaluating each file once. Period. If it doesn't load, send a message and
> nag. Just like for the test cases. (I do have a separate test suite for
> /htdp and /2htdp, but they aren't automated.)

It seems like Eli's build is doing two things: producing the binaries
for download and doing the automated tests linked in tests/. It makes
sense to the first only nightly, but the second every commit.

I can buy a machine and have it do the testing part basically 24/7 at
BYU. I'll have it re-compile mzscheme only when the C code changes and
the byte-code always. Then I'll load each file (which will include the
tests, of course) and send blame to the most recent committer and the
owner (via plt:responsible) of the file.

I'll have a web page that shows the success of each file's loading and
any messages spit out when it runs. At the beginning I can be testing
library agnostic if I can assume any output to stderr is an error.

I think that will be a lot better than what we have now and then we
can experiment with better ways of communicating errors and coverage
to my setup. For example, Eli is referring to errortrace as if it is
the only option. I reckon if we just want to know whether some line of
a file has been run we could get by with less information than is
provided by errortrace.

Anyways... that's my plan. Before I turn it live and you start getting
obnoxious emails, I'll get more permission.

Jay

-- 
Jay McCarthy <jay at cs.byu.edu>
Assistant Professor / Brigham Young University
http://teammccarthy.org/jay

"The glory of God is Intelligence" - D&C 93