XFAIL vs. KFAIL

XFAIL vs. KFAIL

[Vladimir Prus]

Hi,
can somebody explain the difference between XFAILing and
KFAILing a test in the GDB testsuite? I've looked at DejaGNU manual,
I haven't found the answer.

Thanks,
Volodya

Re: XFAIL vs. KFAIL

[Joel Brobecker]

> can somebody explain the difference between XFAILing and KFAILing a
> test in the GDB testsuite? I've looked at DejaGNU manual, I haven't
> found the answer.

KFAIL is when we know of a failure caused by a bug in GDB.

XFAIL is when a problem outside of GDB's control is causing the test
to fail. For instance, a kernel issue, or some wrong debugging info
generated by the compiler, etc.

-- 
Joel

Re: XFAIL vs. KFAIL

[Vladimir Prus]

Joel Brobecker wrote:

>> can somebody explain the difference between XFAILing and KFAILing a
>> test in the GDB testsuite? I've looked at DejaGNU manual, I haven't
>> found the answer.
> 
> KFAIL is when we know of a failure caused by a bug in GDB.
> 
> XFAIL is when a problem outside of GDB's control is causing the test
> to fail. For instance, a kernel issue, or some wrong debugging info
> generated by the compiler, etc.

Hmm, it appears that at least MI testsuite routinely uses XFAIL for what is
a GDB issue/limitation.

Is the distinction really useful? Both seem to be a mechanism to "hide"
failures that are known to be immediately fixeable, and exact description
of the problem belongs to a comment, anyway.

- Volodya

Re: XFAIL vs. KFAIL

[Daniel Jacobowitz]

On Sat, Apr 05, 2008 at 08:55:55PM +0400, Vladimir Prus wrote:
> Hmm, it appears that at least MI testsuite routinely uses XFAIL for what is
> a GDB issue/limitation.

KFAIL is relatively new.  What I mean by that is it's about five years
old, but the MI testsuite is older.

Most of the XFAILs in the testsuite are bogus at this point.

> Is the distinction really useful? Both seem to be a mechanism to "hide"
> failures that are known to be immediately fixeable, and exact description
> of the problem belongs to a comment, anyway.

No, that's exactly what XFAIL isn't for.  We use XFAIL for problems
that we can't fix, like known bugs in versions of GCC or Linux or HP-UX.

-- 
Daniel Jacobowitz
CodeSourcery

Re: XFAIL vs. KFAIL

[Michael Snyder]

On Sat, 2008-04-05 at 20:55 +0400, Vladimir Prus wrote:
> Joel Brobecker wrote:
> 
> >> can somebody explain the difference between XFAILing and KFAILing a
> >> test in the GDB testsuite? I've looked at DejaGNU manual, I haven't
> >> found the answer.
> > 
> > KFAIL is when we know of a failure caused by a bug in GDB.
> > 
> > XFAIL is when a problem outside of GDB's control is causing the test
> > to fail. For instance, a kernel issue, or some wrong debugging info
> > generated by the compiler, etc.
> 
> Hmm, it appears that at least MI testsuite routinely uses XFAIL for what is
> a GDB issue/limitation.

Hey, you're welcome to judiciously update them!   ;-)


> Is the distinction really useful? Both seem to be a mechanism to "hide"
> failures that are known to be immediately fixeable, and exact description
> of the problem belongs to a comment, anyway.

We added KFAIL because the distinction seemed to be useful.
It tells us that this is something we ought to fix some day, 
as opposed to something we have little or no control over.