Re: RFC: KFAIL DejaGnu patch

[Rob Savoye <rob@welcomehome.org>]

On Mon, Apr 08, 2002 at 12:33:54PM -0500, Michael Elizabeth Chastain wrote:

> Now there is an open ended question.  I'll ramble on it for a while.
 
  Really, I might have stepped in a deep one there... :-) I was thinking of
changes the XML format, but these are mostly good suggestions.

> ERRORs and WARNINGs: it would help if all ERRORs and WARNINGs went
> through report_test and got treated like the other results, including
> printing the name of the current test script.  Right now I pick this up
> when I am lexing the gdb.sum file.

  This is probably a good idea.

> Duplicate test names: we have as many as 30 tests in the same file
> with the same name (often named "continue").  Right now I add sequence
> numbers to this when I am lexing the gdb.sum file:
> It would be nice if DejaGnu did this automatically.
 
  Um, I don't think it's the responsibility of DejaGnu to work around a
poor programming practice.

> TIMEOUT: I would like to have a new test result of TIMEOUT for a test
> that times out.  A TIMEOUT often indicates that gdb has crashed, and is
> much more serious than an ordinary FAIL.
 
  If GDB has crashed, this is supposed to be UNRESOLVED, according to POSIX.
UNRESOLVED is a test case that can't be finished, and needs a human to look
at it. UNTESTED is when there isn't support in the underlying OS or
hardware to run a test case.
  
> Split gdb.log file: when I look at gdb.log, I am usually interested in
> just one test script.  So I'd like to have a directory of gdb.log files,
> one per each test script.  I don't need the giant log.

  Hum... I might try that. Mind you, the log files were more for debugging
purposes, than analysing them for test results.

> features to add more stuff to the log file (like a uuid, the email
> address of the person who ran the test, the version of binutils and
> gcc used when testing gdb, and lots of stuff like that).

  Most of this is GDB testing specific, but I'll see what I can do. I can
probably add an "output_hook", that lets test suites add their own info
to the log.

> All of this stuff is just me; the gdb group hasn't talked this over
> and reached any kind of consensus yet.

  Any changes I make have to work generically, since the gdb team is a 
only part of the DejaGnu user community.

	- rob -