Re: RFC: KFAIL DejaGnu patch

[Rob Savoye <rob@welcomehome.org>]

On Tue, Apr 09, 2002 at 11:13:18AM -0500, Michael Elizabeth Chastain wrote:

> I'm not looking for any changes in 1.4.3.  Think of this more as one
> user's feedback.  If you get any good ideas from it, that's fine.
 
  I'm always looking for constructive comments from end users.

> Now that I know Dejagnu is actively maintained, maybe I'll pick this
> one up myself.  I would have to learn more TCL but Dejagnu is not
> that large.

  DejaGnu has been mostly actively maintained except for a period 5-6 years
ago where I needed a break... plus occasionally income generating projects
get in the way... Gotta pay the bills...

> the documentation is that the human is deciding whether the test passed
> or not.  Actually a human is needed here to fix the tool, or perhaps fix
> the testing environment.  I think TIMEOUT would be more accurate than
> using UNRESOLVED for this.
 
  According to POSIX though, UNRESOLVED is the test state, if GDB crashed
while executing a test case. If GDB crashed between test cases, then TIMEOUT,
would be appropriate. Maybe between tests, GDB should be checked to see if
it's crashed...

> Right now we use FAIL "... (timeout)" for this, so in principle, we're
> getting the information we need.  So I can live without this if we really

  In a sense FAIL is appropriate, since a test case fails if GDB crashes. :-)
the FAIL should automatically become an UNRESOLVED, unless somewhere in the
GDB tests the threshold is changed to turn off this feature. Mind you, there
is no real need to have the GDB test suite be POSIX conforming in behavior, so
whatever you do that works is fine.

> Another way to look at it: as projects get larger, the roles separate.
> I'm planning to publish my scripts so that lots of people can run gdb
> tests and mail in useful information.  My end goal is an Internet scale

  Maybe I should include your scripts in dejagnu/contrib. I used to have the
ancient ones we used at Cygnus there till they've kindof fallen out of use...

  Just as a note, DejaGnu now has some support for unit testing. You basically
include a dejagnu.h file, and your test case (typically on an embedded system)
spits out standard test states, and DejaGnu then parses those to produce the
final results. This is probably more useful for GCC or libstdc++ testing, but
I've been using this feature alot lately for customers. (which is why I wrote it)
I rarely write any new code these days without an associated test case.

	- rob -