[racket-dev] A standard for testing Racket's libraries?

On Wed, Feb 27, 2013 at 2:34 PM, Danny Yoo <dyoo at hashcollision.org> wrote:
> On Wed, Feb 27, 2013 at 1:36 PM, Robby Findler
> <robby at eecs.northwestern.edu> wrote:
>> I think that currently you can look at what drdr does. That's the best we
>> have.
>
> I'll assume that this is the drdr collection in meta,
>
>     http://git.racket-lang.org/plt/tree/HEAD:/collects/meta/drdr
>
> and not the drdr2 directory, right?
>

No, I think Robby means (because it is what I would suggest) just go
to drdr.racket-lang.org and see what gets run from the tests directory
and the collection source directory.

>
>> Meanwhile, I think that if you are willing to live a bit on the edge, you
>> can try to run some reasonable looking tests (break something intentionally
>> and make sure they complain, say) and then go ahead and push and let drdr
>> sort things out more. Worst comes to worst, you just revert a commit and try
>> again tomorrow, no?
>
> I am somewhat risk averse.  "Skittish" is probably apt.  I'd prefer to
> rerun re-run a collection-wide test immediately after a bug fix, while
> the change is still fresh in my mind.  Some collections are harder to
> test this way, so DrDr plays its role, but for algorithmic or
> data-structure stuff, I should be able to easily run the tests that I
> can execute locally, before committing a patch to the central
> repository.

I think it would be good to be able to say "raco test -c collection
tests/collection" does everything. You've found a case where that
fails, which while not exactly an error is non-robust code that could
change.

> I can already do this, by spending time reading through the collection
> tree till I find the modules that drive the unit tests.  Sometimes I
> do a manual "grep -r" for the word 'test' and hope for the best.  I
> just wish I didn't have to do that manual search.  And I sometimes get
> it wrong; e.g. the datalog thing from a day ago: I was testing a
> change to parser-tools, but couldn't find any tests for the
> parser-tools collection.  The next best thing I could do was run tests
> on the collections that _use_ parser-tools, and that's how this topic
> got started.

However, you'll never be able to cleanly specify what other
collections/modules you should look at it. Maybe when the majority of
the core are packages that express their dependencies you could run
the things that depend on what you are changing, but even then I doubt
you should be willing to do that.

Jay

-- 
Jay McCarthy <jay at cs.byu.edu>
Assistant Professor / Brigham Young University
http://faculty.cs.byu.edu/~jay

"The glory of God is Intelligence" - D&C 93