test-manager/ - An automatic unit-testing framework for MIT Scheme or Guile
(load "test-manager/load.scm")
(in-test-group simple-stuff (define-test (easy) (assert-= 4 (+ 2 2) "Two and two should make four.") (assert-= 6 (+ 2 2 2))) (define-test (harder) (assert-= 2147483648 (+ 2147483647 1) "Addition shouldn't overflow.")) (define-test (cons) (assert-equal '(1 2 3) (cons 1 '(2 3)))))
(run-registered-tests)
This test framework defines a language for specifying test suites and a simple set of commands for running them. A test suite is a collection of individual tests grouped into a hierarchy of test groups. The test group hierarchy serves to semantically aggregate the tests, allowing the definition of shared code for set up, tear down, and surround, and also partition the test namespace to avoid collisions.
The individual tests are ordinary procedures, with some associated bookkeeping. A test is considered to pass if it returns normally, and to fail if it raises some condition that it does not handle (tests escaping into continuations leads to unspecified behavior). The framework provides a library of assertions that can be invoked in tests and have the desired behavior of raising an appropriate condition if they fail.
All tests are grouped into a hierarchy of test groups. At any point in the definition of a test suite, there is an implicit ``current test group'', into which tests and subgroups can be added. By default, the current test group is the top-level test group, which is the root of the test group hierarchy.
name
that consists of the given expressions, and
add it to the current test group. When the test is run, the
expressions will be executed in order, just like the body of any
procedure. If the test raises any condition that it does not handle,
it is considered to have failed. If it returns normally, it is
considered to have passed. Usually, tests will contain assertions
from the list below, which raise appropriate conditions when they
fail.
name
in the current test
group. Then temporarily make this subgroup the current test group,
and execute the expressions in the body of in-test-group
. This
groups any tests and further subgroups defined by those expressions
into this test group. Test groups can nest arbitrarily deep. Test
groups serve to disambiguate the names of tests, and to group them
semantically. In particular, should a test fail, the names of the
stack of groups it's in will be displayed along with the test name
itself.
define-surround
, the identifier
run-test
is bound to a nullary procedure that actually runs the
test. Clobbers any previously defined surround for this group.
define-group-surround
, the identifier run-test
is bound to a
nullary procedure that actually runs the tests in this group.
Clobbers any previously defined group surround for this group.
The following procedures are provided for running test suites:
(run-test '(simple-stuff harder))
would run the second
test defined in the example at the top of this page.
(run-test '())
.
The following assertions are provided for writing tests:
message
to appear in
the failure report. This is a primitive assertion in whose terms
other assertions are defined.
message
argument is included in the failure report if the
assertion fails.
message
argument is included in the failure report if the
assertion fails.
actual
value is equivalent, according to the
corresponding predicate, to the expected
value. Produces a
reasonably helpful message on failure, and includes the optional
message
argument in it if present. When in doubt, use
assert-equal
to compare most things; use assert-=
to compare
exact numbers like integers; and use assert-in-delta
, below, for
inexact numbers like floating points.
assert-equal
and company could have been defined in terms of
assert-equivalent
as, for example, (define assert-equal
(assert-equivalent equal? "equal?"))
.
message
argument is included in the failure report if the
assertion fails.
actual
value differs, in absolute value, from
the given expected
value by no more than delta
. Use this in
preference to assert-=
to check sameness of inexact numerical
values. The optional message
argument is included in the failure
report if the assertion fails.
This unit testing framework is a work in progress. The assertion library is quite impoverished, the test groups do not support as much shared set up code among their tests as I would like, and the language for explicit test group handling is ill-specified and undocumented (peruse test-group.scm if interested). Suggestions are welcome.
Alexey Radul, axch@mit.edu