Re: [testsuite] add gdb.cp/gdb1355.exp

[David Carlton <carlton@kealia.com>]

On Wed, 17 Sep 2003 22:01:43 -0400, Michael Elizabeth Chastain <mec@shout.net> said:

> As far as making it a FAIL goes, I think David's argument comes down
> to: people pay attention to FAIL, but not to XFAIL or KFAIL.

More or less, yes.  Let me rephrase this in a different way (and I'll
discuss the issue in terms of KFAIL, but the same goes for XFAIL).  We
have three possible states for a bug:

1) The bug is completely undiagnosed.
2) The bug is diagnosed but known not to be fixed.
3) The bug was diagnosed, thought to be fixed, but not actually fixed.

We have three states here, and only two bug labels (and I will scream
if anybody tries to add "ONCEFAILED" to dejagnu).  So, no matter what,
we're going to lose information.  In that regard, I don't think this
statement of yours is the correct way to phrase the question:

> I see the question as: whether to fight the situation and use more
> accurate XFAIL/KFAIL rather than generic FAIL, or to acquiesce to
> situation and give people what they expect to see.

In your proposal, KFAIL is less precise (since it encompasses cases 2
and 3) while FAIL is more precise (it encompasses only case 1); in my
proposal, FAIL is less precise (cases 1,3) while KFAIL is more precise
(only case 2).  One question, then, is: which state do we want to be
more precise?  I would answer KFAIL.

That's the static way of looking at the situation; another is a
dynamic way, namely how will the output of the tests change between
runs.  In both of our proposals, if a regression pops up, we'll see a
state transition: either PASS=>FAIL or PASS=>KFAIL.  But my proposal
also guarantees that there will always be a state transition after a
bug is (thought to be) fixed: either KFAIL=>PASS or KFAIL=>FAIL.  In
your proposal, the latter scenario won't lead to a state transition:
it will remain KFAIL.

I guess that's enough babbling from me; I'll try to be quiet for a
little while now. :-)

David Carlton
carlton@kealia.com