[racket] Thoughts on Overeasy

Thanks for the comments, Noel.

Noel Welsh wrote at 08/27/2011 05:21 PM:
> I think the keyword argument method is a mistake. It seems you'd have
> to do a lot of checking in the test macro to make sure the user
> doesn't specify non-sensical parameters. E.g. if I specify #:exn and
> #:val what happens?

You're speaking of programming difficulty for implementing Overeasy 
itself?  "syntax-parse" should make that particular one (both "#:val" 
and "#:exn" present) a non-issue.  I suspect that rest of any 
nonsensical argument combinations can be checked easily.

> The test macro grabs whatever values value-expr produces, along with
> the output to current-output and current-error, and any exceptions and
> passes these values to
>
> match-expr, which is a predicate that produces a boolean value given
> the above information
>   

What I'd like to do as a requirements analysis exercise is to identify a 
representative set of side-effects that people would like to have 
tested, like "current-output-port" and "current-error-port" are now.  
If, in practice, there's only one or two more, we might just hardcode 
them into Overeasy for now.

However, in the future, I would like to treat the "#:val" and "#:exn" 
specially, but to make all the effects testing be plugins, for 
extensibility.  The "test" macro could then consult the test context to 
get the sequence of effects to be tested, and would use the that info 
for parsing unfamiliar keyword arguments and for evaluating the result.

There's no reason that people who want to, say, check side-effects in an 
RDBMS can't just hook that effects testing into their "#:val-check", but 
letting them extend the "test" syntax might be convenient.  (The "test" 
macro admittedly has some syntactic sugar going on already, and I think 
that making it extensible this way might be a good idea.)

> Since test must be a macro, it should capture location and report that
> when a test fails.
>   

Agreed.  I almost added this originally, and it is on my TODO list.  I 
would like one of the report handlers to make messages that Emacs 
"compile" will understand.

(Aside: "test" didn't *have* to be a macro, if we were willing to do an 
"eval" for "#:code".  I was tempted to do that for handling broken 
syntax expansions more robust by default.  For now, I'm trying to avoid 
"eval" in Overeasy, although I can add it in later, if necessary, and if 
I can make it work wrt the calling environment in the way the user expects.)

> "No special forms for setup and tear-down are required, nor are they
> provided."  I believe a test library should provide these conveniences
> around dynamic-wind.
>   

OK, I'd like to see examples of this.  Do you think it will always be 
things like setting up a test input file or database tables, and then 
deleting files and tables when done?  Also, does this include syntax 
that needs to be done ``around'' multiple tests, like the following, 
correct?

(with-something-used-by-bar

  (test (bar 42) 69)

  (test (bar 88) 1701))


-- 
http://www.neilvandyke.org/