[1663] in Kerberos_V5_Development
Re: testing
daemon@ATHENA.MIT.EDU (Barry Jaspan)
Thu Aug 29 11:03:22 1996
Date: Thu, 29 Aug 1996 11:02:56 -0400
From: "Barry Jaspan" <bjaspan@MIT.EDU>
To: raeburn@cygnus.com
Cc: tytso@MIT.EDU, krbdev@MIT.EDU
In-Reply-To: <tx1afvfnepg.fsf@cygnus.com> (message from Ken Raeburn on 28 Aug
1996 21:47:55 -0400)
Right now, if a db test fails, the "make check" run stops. No
applications get tested. No other unit tests that would be completely
independent of db get tested. This sort of thing has been wasting my
time recently.
Okay, now I understand your suggestion.
I disagree. The behavior you want can be accomplished with make -k.
I think that lumping all the tests into one run of dejagnu will make
it too hard to run sub-groups of tests together; for example, the
kadm5 api tests consist of many separate .exp files, and it would be
very cumbersome to specify them all. Furthermore, running all the
tests under one dejagnu presupposed that all the tests operate in the
same environment. In the kadm5 system, each makefile in a unit-test
subdirectory has three targets, unit-test-setup, unit-test-body, and
unit-test-cleanup. setup typically runs the standard "create database
and start servers" script, but then in some directories also does some
other things (creates specific principals for that directory's tests,
or creates a srvtab needed by v4passwd, or whatever). Some sets of
tests may need to run in conflicting environments such that it is
impossible for them to be set up at the same time. And so forth.
The ability to tell if all the tests pass without scanning through a
huge log file can be accomplished with a simple perl script. I wrote
such a script while at OV, and I could probably even get them to mail
it to me (I'm sure they don't even know it exists and aren't using
it). It read the make -k output checking for each of an enumerated
set of tests, printed out a "passed" or "failed: <info>" line for
each, and then ended with a simple PASSED or FAILED, so you could just
run it and look at the last line of output.
Barry
