[racket] tests/eli-tester feedback (Was: Racket unit testing)

On Sun, 13 Feb 2011 12:16:59 -0500
Eli Barzilay <eli at barzilay.org> wrote:

> Not documented, and not distributed by default (since it still lives
> as just a file in the `tests' collection).  

Hidden like an easter egg :-)

> The second issue is that currently it catches almost *all* errors,
> including syntax errors.  A good example for a bogus result that Robby
> got once is:
> 
>   (test (+ x 1) => x)
> 
> where `x' is unbound -- it will catch the syntax error on both sides,
> and since the error messages match, it will conclude that the test is
> successful.  The solution here is to avoid making it catch errors
> unless you use an explicit `=error>' arrow, and even then catch only
> runtime error and add another arrow for syntax errors (perhaps
> `=syntax=error>').

Naively speaking (and without reading any docs ;-), I'd expect =>
to handle "normal" operations. Using error-arrows is a good idea,
as it makes it clearly visible that there's something going on here
without clobbering the test descriptions.

In Robby's case (+ x 1) would raise an error, which would _not_
be caught by => but instead show up as normal exception.

I'm not convinced that you'd need more than one type of error arrow,
though.

> b. Add some way to wrap all evaluations, so, for example, you can
>    create a sandbox and have all evaluations happen inside it, so you
>    can do things like:
>      (test (some-loop) =error> "out of memory")
>    which are only possible if the expressions are running inside a
>    sandbox.

hm. Personally, I'm looking for a "small and easy" tester that I can
use while teaching my son some basic programming skills. Between
(check-expect whatever) and (assert-what's-it-called-again something or-other) your
(test expr => result) is a very good fit for this task. I can live
without that wrapper very comfortably :-)

> In any case, any feedback on those questions will be good -- feel free
> to mail me directly how you'd prefer this to look.

Heh... there's a good chance I'm saying something stupid, so I prefer
to do this in public, as then it is bound to be spotted more quickly.

s.