[racket] tests/eli-tester feedback (Was: Racket unit testing)

Eli Barzilay wrote at 03/03/2011 10:23 AM:
> 2. A unified foundation for building test suites on, which will make it possible to deal with testing independently of the actual test library you happen to use.  (There was less agreement on that.)  I plan to look into it after the above.
>   

Regarding unified foundation... I just converted one of my test suites 
to RackUnit, and the foundation it seems to use is different than I 
expected...

Could someone explain the advantages of having, say, the "test-suite" 
form produce a data structure to be supplied to test-runner code... 
rather than having evaluation of the "test-suite" form result in the 
tests actually being executed?

Regarding the latter alternative, I had in mind a simple model, closer 
to Racket language model, in which there is a *test run context*, 
represented as a Racket language parameter.  Evaluating a "check" form 
results i its test being executed, and it then simply calls back to code 
in the test run context to report the result of the test.

Contexts can be nested hierarchically.  By default, a "check" form, when 
there's no context (such as is the default when evaluating from the REPL 
as a one-liner) will use an effectivve one-off context to report the 
result.  A "test-suite" form, which creates its own context, if there is 
no parent context, will report the results of each test, along with a 
scorecard.  A context that does have a parent will report its results up 
to the parent.

Generally, each "test-suite" would get its own file, and could be run by 
itself, for unit testing while working on a module, or it could be run 
inside the dynamic context of a parent "test-suite", such as when 
testing all the module in a subsystem with one hierarchically-nested report.

Since information about results of each individual test is always 
propagated to the topmost context, running the tests from a GUI tool 
simply means establishing a test run context for the GUI before the 
topmost "test-suite" form is evaluated.

This alternative with dynamic contexts seems to me to be somewhat easier 
to work with than RackUnit's data structures approach.  What are the 
advantages of the RackUnit approach?

This question is important to me because, if the unified foundation for 
building test suites has a model similar to that of RackUnit, I'm not 
sure that's the desired model.

-- 
http://www.neilvandyke.org/