[racket-dev] [racket] tests/eli-tester feedback (Was: Racket unit testing)

[Trimmed Neil from CCs]

On Tue, Feb 15, 2011 at 12:32 PM, Eli Barzilay <eli at barzilay.org> wrote:

> * It uses TR -- but it has to be very minimal.  (*Extremely* minimal,
>  since it's a candidate for inclusion in the `racket' language, which
>  is why I was thinking of not much more than agreed structs and
>  parameters.)

Not a problem. TR just keeps the types straight.

> * I don't see why some results are structs and some are exceptions.

Test failures should include continuation marks etc. Exceptions feel
like the right types for these.

> * I don't see the point of a `pending' struct -- if it's supposed to
>  represent a pending test, then it seems to me that the test itself
>  should be modified when it has been executed.

I don't understand this. Some Rackunit users want to write tests that
are have not yet been implemented, and want the UI to report how many
yet-to-be implemented tests they have. I think you interpreted pending
in the sense of concurrency. I mean it here in the sense of yet to be
written. It is useful to annotate tests you know you have to write but
haven't yet done so.

> * Slightly worse: how would pending tests be used with something like
>  my tester or the rackunit lightweight things where there is no way
>  to know the tests in advance?

See above, I think.

> * The test-listener thing is good (I missed it when I looked at the
>  code earlier).

Well, we agree on 1 thing. :)

> * I'm not sure about things around "test-name" -- it should be free
>  form information that the testing framework should render later on.
>  Perhaps with a minimal spec of source and name, where a name can be
>  #f.  (I'm thinking of these "anonymous" tests here.)  (This might be
>  some unification of your name and info things, which should really
>  always come together since you'd never modify one without the
>  other.)

Agreed info and name should be unified. In the current design a name
can always be null, indicating no name. It is a list to represent a
hierarchy of names.

> * The `with-*' macros should go away.  They're leaks of the higher
>  level code.
>
> * The same goes for `pass' and `fail'.  Even more importantly `lift',
>  which is dictating some treatment of errors etc.

No problem there.

N.