[racket-dev] how to test rackunit
What is the correct way to run the tests which test rackunit?
I am running the following after building with a clean tree:
$ racket/bin/raco test pkgs/rackunit-pkgs/rackunit-test/tests/rackunit
This seems to show several "failures", though since it is testing rackunit
I'm pretty sure most if not all are intentional (to verify that things
which should fail do fail.) The full output is below [*].
What is not clear is how do people run this and determine whether they
have any unintentional failures? Is there a separate command I should
be using? Perhaps people just diff the output from a clean tree vs
the output with their changes?
BTW, I have read:
http://blog.racket-lang.org/2012/11/tutorial-contributing-to-racket.html
which has been very helpful since Joe's example was a change to
rackunit, but I didn't see anything addressing this problem.
For additional context: my motivation for running these tests is to improve
check-exn so it no longer passes when given a non-thunk. I think it
should raise an error instead. I already have a patch for this but
running the tests has me thinking I'm not using the correct command to
test rackunit.
David
[*] Here is the output from the above 'raco test ...' command, with stdout
and stderr redirected to a file. Note the redirection changes the
ordering of some lines compared to the terminal output.
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/all-rackunit-tests.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/base-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/check-info-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/check-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/counter-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/format-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/hash-monad-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/location-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/log.rkt"
--------------------
FAILURE
name: check-true
location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/log.rkt> 38 2 1185 15)
expression: (check-true #f)
params: (#f)
Check failure
--------------------
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/monad-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/pr10950.rkt"
1 test--------------------
ERROR
Outta here!
context...:
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/check.rkt:133:29
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/check.rkt:58:0: top-level-check-around
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/check.rkt:119:21: check-pred492750
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-check-test.rkt: [running body]
f88
loop
loop
loop
f88
/home/dtp/src/dtp-racket/pkgs/compiler-pkgs/compiler-lib/compiler/commands/test.rkt: [running body]
/home/dtp/src/dtp-racket/racket/collects/raco/raco.rkt: [running body]
/home/dtp/src/dtp-racket/racket/collects/raco/main.rkt: [running body]
--------------------
--------------------
FAILURE
name: check
location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-check-test.rkt> 44 0 1344 17)
expression: (check = 1 2)
params: (#<procedure:=> 1 2)
message: 0.0
Check failure
--------------------
--------------------
ERROR
First Outta here!
context...:
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-test-case-test.rkt: [running body]
f88
loop
loop
loop
f88
/home/dtp/src/dtp-racket/pkgs/compiler-pkgs/compiler-lib/compiler/commands/test.rkt: [running body]
/home/dtp/src/dtp-racket/racket/collects/raco/raco.rkt: [running body]
/home/dtp/src/dtp-racket/racket/collects/raco/main.rkt: [running body]
--------------------
--------------------
error
ERROR
Second Outta here!
context...:
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-test-case-test.rkt: [running body]
f88
loop
loop
loop
f88
/home/dtp/src/dtp-racket/pkgs/compiler-pkgs/compiler-lib/compiler/commands/test.rkt: [running body]
/home/dtp/src/dtp-racket/racket/collects/raco/raco.rkt: [running body]
/home/dtp/src/dtp-racket/racket/collects/raco/main.rkt: [running body]
--------------------
--------------------
FAILURE
actual: 1
expected: 2
name: check-eq?
location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-test-case-test.rkt> 19 12 520 15)
expression: (check-eq? 1 2)
Check failure
--------------------
--------------------
failure
FAILURE
actual: 1
expected: 2
name: check-eq?
location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-test-case-test.rkt> 20 21 558 15)
expression: (check-eq? 1 2)
Check failure
--------------------
--------------------
FAILURE
actual: #""
expected: #"Oh HAI!\nI didn't run\n"
name: check-equal?
location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone.rkt> 29 4 956 23)
expression: (check-equal? cout #"Oh HAI!\nI didn't run\n")
Check failure
--------------------
--------------------
FAILURE
actual: #""
expected: #"--------------------\nERROR\nOutta here!\n\n--------------------\n--------------------\nFAILURE\nname: check\nlocation: (#<path:PLTHOME/collects/tests/rackunit/standalone-check-test.rkt> 44 0 1344 17)\nexpression: (check = 1 2)\nparams: (#<procedure:=> 1 2)\nmessage: 0.0\n\nCheck failure\n--------------------\n"
name: check-equal?
location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone.rkt> 30 4 984 23)
expression: (check-equal? cerr #"--------------------\nERROR\nOutta here!\n\n--------------------\n--------------------\nFAILURE\nname: check\nlocation: (#<path:PLTHOME/collects/tests/rackunit/standalone-check-test.rkt> 44 0 1344 17)\nexpression: (check = 1 2)\nparams: (#<procedure:=> 1 2)\nmessage: 0.0\n\nCheck failure\n--------------------\n")
Check failure
--------------------
--------------------
FAILURE
actual: #""
expected: #"--------------------\nERROR\nFirst Outta here!\n\n--------------------\n--------------------\nerror\nERROR\nSecond Outta here!\n\n--------------------\n--------------------\nFAILURE\nactual: 1\nexpected: 2\nname: check-eq?\nlocation: (#<path:PLTHOME/collects/tests/rackunit/standalone-test-case-test.rkt> 19 12 520 15)\nexpression: (check-eq? 1 2)\n\nCheck failure\n--------------------\n--------------------\nfailure\nFAILURE\nactual: 1\nexpected: 2\nname: check-eq?\nlocation: (#<path:PLTHOME/collects/tests/rackunit/standalone-test-case-test.rkt> 20 21 558 15)\nexpression: (check-eq? 1 2)\n\nCheck failure\n--------------------\n"
name: check-equal?
location: (#<path:/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone.rkt> 30 4 984 23)
expression: (check-equal? cerr #"--------------------\nERROR\nFirst Outta here!\n\n--------------------\n--------------------\nerror\nERROR\nSecond Outta here!\n\n--------------------\n--------------------\nFAILURE\nactual: 1\nexpected: 2\nname: check-eq?\nlocation: (#<path:PLTHOME/collects/tests/rackunit/standalone-test-case-test.rkt> 19 12 520 15)\nexpression: (check-eq? 1 2)\n\nCheck failure\n--------------------\n--------------------\nfailure\nFAILURE\nactual: 1\nexpected: 2\nname: check-eq?\nlocation: (#<path:PLTHOME/collects/tests/rackunit/standalone-test-case-test.rkt> 20 21 558 15)\nexpression: (check-eq? 1 2)\n\nCheck failure\n--------------------\n")
Check failure
--------------------
passed
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/result-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/run-tests.rkt"
127 success(es) 0 failure(s) 0 error(s) 127 test(s) run
0
--------------------
Failures > Intended to fail
Intended to fail
FAILURE
name: fail
location: pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/all-rackunit-tests.rkt:47:33
params: Check failure
--------------------
--------------------
Failures > Also intended to fail
Also intended to fail
FAILURE
name: check-eq?
location: pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/all-rackunit-tests.rkt:48:38
actual: 'apples
expected: 'orange
Check failure
--------------------
--------------------
Failures > Yet again intended to fail
Yet again intended to fail
FAILURE
name: check-equal?
location: pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/all-rackunit-tests.rkt:49:3
actual: "apples"
expected: "oranges"
Check failure
--------------------
--------------------
Failures > Intended to throw error
Intended to throw error
ERROR
testing: <<This is an error message>>
context...:
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/result.rkt:99:3
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/test-suite.rkt:28:2
the-tests
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/test-suite.rkt:60:0: apply-test-suite
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/text-ui.rkt:238:0: run-tests14
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/run-tests.rkt: [running body]
f88
loop
loop
loop
f88
/home/dtp/src/dtp-racket/pkgs/compiler-pkgs/compiler-lib/compiler/commands/test.rkt: [running body]
/home/dtp/src/dtp-racket/racket/collects/raco/raco.rkt: [running body]
/home/dtp/src/dtp-racket/racket/collects/raco/main.rkt: [running body]
--------------------
--------------------
Failures > Error within a check
Error within a check
ERROR
name: check
location: pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/all-rackunit-tests.rkt:51:37
params: #<procedure:error>
'foo
'bar
error: contract violation
expected: string?
given: 'bar
argument position: 2nd
other arguments...:
'foo
context...:
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/check.rkt:133:29
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/check.rkt:119:21: check491437
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/result.rkt:99:3
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/test-suite.rkt:28:2
the-tests
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/private/test-suite.rkt:60:0: apply-test-suite
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-lib/rackunit/text-ui.rkt:238:0: run-tests14
/home/dtp/src/dtp-racket/pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/run-tests.rkt: [running body]
f88
loop
loop
loop
f88
/home/dtp/src/dtp-racket/pkgs/compiler-pkgs/compiler-lib/compiler/commands/test.rkt: [running body]
/home/dtp/src/dtp-racket/racket/collects/raco/raco.rkt: [running body]
/home/dtp/src/dtp-racket/racket/collects/raco/main.rkt: [running body]
--------------------
0 success(es) 3 failure(s) 2 error(s) 5 test(s) run
5
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-check-test.rkt"
Oh HAI!
I didn't run
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone-test-case-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/standalone.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/test-case-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/test-suite-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/test-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/text-ui-test.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/text-ui-util-test.rkt"
55/340 test failures
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/tl.rkt"
raco test: "pkgs/rackunit-pkgs/rackunit-test/tests/rackunit/util-test.rkt"
![[logo]](/logo.png){kind=logo}