[racket-dev] Test results are slightly incomprehensible...

From: Paulo J. Matos (paulo at matos-sorge.com)
Date: Wed Sep 28 16:02:14 EDT 2011

...if you're not used to them, that is.

Hi,

I am running the racket tests to know if something went wrong with my
fork of racket. The fork was done at tag 2ef9f5ae.

I run the tests with from the install directory with:
,----
| racket$ bin/racket -r ./collects/tests/racket/quiet.rktl
`----

Attached, you can find the output I obtain for both my repository and
the main repository. The diff looks like:
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test_results_main.txt
URL: <http://lists.racket-lang.org/dev/archive/attachments/20110928/9dd8b146/attachment.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test_results_pmatos.txt
URL: <http://lists.racket-lang.org/dev/archive/attachments/20110928/9dd8b146/attachment-0001.txt>
-------------- next part --------------
,----
| --- /home/pmatos/repositories/racket/test_results.txt   2011-09-28 20:44:41.000000000 +0100
| +++ test_results.txt    2011-09-28 20:51:12.000000000 +0100
| @@ -90,8 +90,8 @@
|  Section(resource)
|  Section(syntax/....)
|  
| -Performed 152701 expression tests (139300 value expressions, 13401 exn expressions)
| -and 26923 exception field tests.
| +Performed 152694 expression tests (139295 value expressions, 13399 exn expressions)
| +and 26919 exception field tests.
|  
|  Errors were:
|  (Section (got expected (call)))
| @@ -105,8 +105,8 @@
|  Section(cm)
|  Section(module-reader)
|  
| -Performed 153275 expression tests (139865 value expressions, 13410 exn expressions)
| -and 26941 exception field tests.
| +Performed 153268 expression tests (139860 value expressions, 13408 exn expressions)
| +and 26937 exception field tests.
|  
|  Errors were:
|  (Section (got expected (call)))
| @@ -118,8 +118,8 @@
|  Section(foreign)
|  Section(uni-norm)
|  
| -Performed 505944 expression tests (492534 value expressions, 13410 exn expressions)
| -and 26941 exception field tests.
| +Performed 505937 expression tests (492529 value expressions, 13408 exn expressions)
| +and 26937 exception field tests.
|  
|  Errors were:
|  (Section (got expected (call)))
`----

Some of the interesting things are:
- My fork seems to, after `Section(syntax/....)', perform less
expression tests and less exception field tests. Why is this?
- Same applies for all the other performed tests that were reported.

Now, what I don't understand is that I was expecting something like:
,----
| 10 tests PASS, 5 tests FAIL, 2 tests CHECK, ...
| <list of failed tests>
`----

How do I go about checking what failed or not and where is the code that
failed?

Cheers,

-- 
PMatos

Posted on the dev mailing list.