[racket] tests/eli-tester feedback (Was: Racket unit testing)

On Monday, February 14, 2011, Eli Barzilay <eli at barzilay.org> wrote:
> Yesterday, Neil Van Dyke wrote:
>> Eli Barzilay wrote at 02/13/2011 09:41 PM:
>> > It currently shoots for (and will continue in the future) a very
>> > low readability overhead -- that's the whole reason for the
>> > infixish `=>' syntax. [...] To put this differently, I view tests
>> > as an important thing that lives in the api neighborhood.  So
>> > anything that requires looking at the documentation for casual
>> > readers is as bad as writing the manual in hebrew and and handing
>> > out dictionaries.
>>
>> I'm not so sure about the requirement "readability by casual readers
>> of the source without requiring looking at the documentation".
>
> I'm tempted to ask what would you consider unclear about
>
>   (test (+ 1 2) => 3)
>
> but that's getting into subjectiveland.  In any case, I figured that
> a much better solution to avoid some new `/=>' is to have instead a
> new `true' so that (test E => true) works for any non-#f value.

That looks worse to me than (test (and X #true) => #true) because it
raises all the new keyword binding issues. FWIW.

Robby

>
>> But if we do have that requirement, to me, it implies:
>>
>> * Racket-idiomatic syntax (which usually means grouping parens, and
>>   no infix keywords); and
>>
>> * fairly self-explanatory human-readable identifiers (like using the
>>   words like "equals", "eq", "result", "returns", "yields",
>>   "expect", and "value", and not using more ambiguous or cryptic
>>   symbols like "=>").
>
> Obviously, there's additional directions that are important (IMO,
> possibly less for others).  Two of these are:
>
> 1. Needs to be very convenient to write.  IOW, I want to just add some
>    tests quickly, without typing a whole pile of boilerplate for
>    defining a test suite, etc.
>
> 2. Errors should be helpful in finding out what went wrong.
>
> Without these, you could reduce your tests to:
>
>   (unless (and (equal? E1 V1) ...) (error "failure"))
>
>
>> Going back to that root requirement... I think that unit testing
>> should be so central to contemporary programming that we should just
>> pick some syntax that makes sense for practical development both
>> large and small, use it everywhere, and simply expect people who are
>> looking at the source to know what Racket unit tests look like.  If
>> we can do this canonical test syntax sensibly, and the syntax is set
>> up so that we can plug in our own user interfaces for running the
>> tests, I will convert all my existing and new open source code to
>> use this canonical syntax.
>
> My own conclusion here is that there are a lot of variables in a test
> system -- even with something as minimal and small as my thing there
> are already a number of non-trivial choices to make, and possibly some
> alternatives that will be provided (like a `test*' thing that would do
> the same was what its `test' is doing now).
>
> So instead of trying to get a canonical syntax, it's better to get to
> a canonical support that all test frameworks will build on.  For
> example, they'd all run in a similar way, return some common structs
> for failures etc.  This way a tool like the rackunit gui becomes
> independent of what you choose to use.
>
> --
>           ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
>                     http://barzilay.org/                   Maze is Life!
> _________________________________________________
>   For list-related administrative tasks:
>   http://lists.racket-lang.org/listinfo/users
>