[racket] tests/eli-tester feedback

On Mon, Feb 14, 2011 at 7:39 PM, Eli Barzilay <eli at barzilay.org> wrote:
> 30 minutes ago, Robby Findler wrote:
>> On Mon, Feb 14, 2011 at 4:53 PM, Eli Barzilay <eli at barzilay.org> wrote:
>> > Ah -- I should have remembered that point.  Yes, I meant that it
>> > is bound.  I should have also said that `true' is a bad name for
>> > this reason, in the same way that `->' is (I'd prefer it over
>> > `=>').
>>
>> FWIW, existing convention in Scheme literature for "evaluates to" is
>> already "=>" so, in my mind, we lucked on out on this one. :)
>
> (I probably have baggage coming from a place where evaluation is
> almost always a single lined arrow.)
>
>
>> > BTW, this leads to another requirement -- the "namespace
>> > pollution" should be minimal since you don't want code to change
>> > (as in `true' getting some strange value for tests) because you
>> > use tests.
>>
>> Yes!
>>
>> > In any case, if this is a sane approach, then maybe it can be used
>> > for the multiple values and the exceptions case too.  Something
>> > like this:
>> >
>> >  (test E1 => true-result
>> >        E2 => (values-result 1 2)
>> >        E3 => (raised-result 666))
>>
>> I prefer to not have true-result and just deal with the (and E1 #t)
>> => #t or add an =/=> arrow.
>
> I *really* don't want to force the `(and E #t) => #t' thing.

Then, imo, the special arrow "evaluates to something other than" is
the best of the bad options.

>
>> As for multiple values, I think that => should deal with that
>> already.  Ie these should all be passing tests:
>>
>>   (test 1 => 1)
>>   (test (values 1 2 3) => (values 1 2 3))
>>
>> and this should be a failing test (not a runtime error (necc)):
>>
>>   (test (values 1 2) => (values 1 2 3))
>
> OK, I can buy that.
>
>
>> For raised-result, I think that's uncommon enough that writing
>>
>>   (test (with-handlers ((values values)) E3) => 666)
>>
>> is just fine.
>
> That's also an issue, since if you really want a complete version,
> then you should catch only non-exceptions, and avoid confusing results
> from raised results, so it's more like
>
>  (test (with-handlers ([(negate exn?) (lambda (x) (list x))]) E #f)
>        => (list 666))
>
> and IMO that's definitely enough non-trivial boilerplate that
> justifies some abstraction -- even if it's rare.
>
> So, maybe go back to what it does now, only use a different arrow for
> those cases?  Something like `=>' being the simple thing, `==>' (or
> `*=>') becomes the version that catches runtime errors and other
> raised values (on both sides)?  (Having a `===>' that also catches
> syntax errors doesn't feel right here.)

That would not be my first choice. If you really think the above is
common enough to warrant special support, why not make that special
support be something like:

(define-syntax-rule (raise-non-exn? e) (raise-non-exn?/proc (lambda () e)))
(define-struct ack-no-exn! ())
(define (raise-non-exn?/proc t)
  (with-handlers ([(negate exn?) (lambda (x) x)]) (t) (make-ack-no-exn!)))

and then make people write:

  (test (raise-non-exn? E) => 666)

or something?

>> I take my cue here from what I see as the essence of the design of
>> your test library, namely its minimality (I also really like this
>> idea).
>>
>> I think adding extra arrows (or having support for defining them) to
>> capture things like state initialization (aka "setup" and
>> "teardown") and catching IO is the right way to go. And if these
>> arrows always have the form "=something=>" (replacing something),
>> then they will be iconic and easy to look up if the name is not
>> enough to be able to guess.
>
> Yeah, different arrows are very likely to be in.  I already have that,
> with things like
>
>  (test E => 123 =outputs> "foo")
>
> IOW, the arrows can even compose.  But it's difficult to write new
> arrows, while write a new kind of a `*-result' would be generally
> easier.
>
> So I guess that this is a sign that more thinking is needed.

I think there's been enough lack of activity on this library to
justify the thought process that could have produced a giant computer
capable of computing 42.

> (BTW, none of this is doing any kind of setup/teardown -- that's left
> for plain racket code.)

Great!

Robby