[racket-dev] A standard for testing Racket's libraries?

FWIW, I wouldn't mind if someone wanted to add a flag to "raco test" that
took a collection (or subcollection) and followed some standardized way to
test it. I think that would help us bring the collections in line to test
them that way.

It would probably work best if we had someone that was actually willing to
do the work to get a large number of collections to follow the plan,
whatever "raco test --collection" ends up doing.

Robby


On Thu, Feb 28, 2013 at 6:42 AM, Jay McCarthy <jay.mccarthy at gmail.com>wrote:

> On Wed, Feb 27, 2013 at 2:34 PM, Danny Yoo <dyoo at hashcollision.org> wrote:
> > On Wed, Feb 27, 2013 at 1:36 PM, Robby Findler
> > <robby at eecs.northwestern.edu> wrote:
> >> I think that currently you can look at what drdr does. That's the best
> we
> >> have.
> >
> > I'll assume that this is the drdr collection in meta,
> >
> >     http://git.racket-lang.org/plt/tree/HEAD:/collects/meta/drdr
> >
> > and not the drdr2 directory, right?
> >
>
> No, I think Robby means (because it is what I would suggest) just go
> to drdr.racket-lang.org and see what gets run from the tests directory
> and the collection source directory.
>
> >
> >> Meanwhile, I think that if you are willing to live a bit on the edge,
> you
> >> can try to run some reasonable looking tests (break something
> intentionally
> >> and make sure they complain, say) and then go ahead and push and let
> drdr
> >> sort things out more. Worst comes to worst, you just revert a commit
> and try
> >> again tomorrow, no?
> >
> > I am somewhat risk averse.  "Skittish" is probably apt.  I'd prefer to
> > rerun re-run a collection-wide test immediately after a bug fix, while
> > the change is still fresh in my mind.  Some collections are harder to
> > test this way, so DrDr plays its role, but for algorithmic or
> > data-structure stuff, I should be able to easily run the tests that I
> > can execute locally, before committing a patch to the central
> > repository.
>
> I think it would be good to be able to say "raco test -c collection
> tests/collection" does everything. You've found a case where that
> fails, which while not exactly an error is non-robust code that could
> change.
>
> > I can already do this, by spending time reading through the collection
> > tree till I find the modules that drive the unit tests.  Sometimes I
> > do a manual "grep -r" for the word 'test' and hope for the best.  I
> > just wish I didn't have to do that manual search.  And I sometimes get
> > it wrong; e.g. the datalog thing from a day ago: I was testing a
> > change to parser-tools, but couldn't find any tests for the
> > parser-tools collection.  The next best thing I could do was run tests
> > on the collections that _use_ parser-tools, and that's how this topic
> > got started.
>
> However, you'll never be able to cleanly specify what other
> collections/modules you should look at it. Maybe when the majority of
> the core are packages that express their dependencies you could run
> the things that depend on what you are changing, but even then I doubt
> you should be willing to do that.
>
> Jay
>
> --
> Jay McCarthy <jay at cs.byu.edu>
> Assistant Professor / Brigham Young University
> http://faculty.cs.byu.edu/~jay
>
> "The glory of God is Intelligence" - D&C 93
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.racket-lang.org/dev/archive/attachments/20130228/c41971ca/attachment.html>