Re: [patch/rfc] Remove all setup_xfail's from testsuite/gdb.mi/

Andrew Cagney
2002-10-24T21:22:47Z

> On Thu, Oct 24, 2002 at 03:29:56PM -0400, Andrew Cagney wrote:
>
>> >On Thu, Oct 24, 2002 at 02:41:05PM -0400, Andrew Cagney wrote:
>> >
>
>> >>Hello,
>> >>
>> >>GDB's testsuite is known to be full of xfails that are really kfails or
>> >>testsuite bugs. Rather than try to audit each xfail in turn, the
>> >>proposal as been to rip out all the xfails (creating a clean slate) and
>> >>start marking up the tests from scratch - two steps forward but first
>> >>one step back.
>> >>
>> >>I figure I might as well try to get the ball rolling on this and find
>> >>out just how much real resistance there is going to be to a change like
>> >>this. To that end, this removes all xfail's from the gdb.mi testsuite.
>> >> Similar tests, for the other directories, would follow.
>
>> >
>> >
>> >I'm known to be a testsuite nazi - I really, really dislike the current
>> >failure levels, and people aren't doing much about it. I'm all in
>> >favor of getting the ball rolling. But are you planning to do the
>> >marking promptly, or just make us stare at even more MI failures for a
>> >while? I've been staring at the mi-console one for a year...
>
>>
>> I believe that you're free to start kfailing things :-)
>>
>> As for me doing the kfail's, the chances of me getting back to that
>> short term are, lets say, pretty remote. I'm just trying to get the
>> process started - eliminate the task that's going to cop the most flack :-)

Like I said, the task that would cop the most flack ... :-)

> In that case, I'd ask you not to commit this. There's no point in
> adding to the FAILs if it doesn't gain us anything.

I think the patch, regardless of KFAIL, is still technically correct. It
fixes a bug: the XFAILs are all wrong so removing them changes the
testsuite so that the numbers it reports better reflect reality. It's
just unfortunate that part of the reality is a jump in testsuite
failures. Remember, the XFAILs were originally added to artifically
deflate the test failure rate.

> Would it be
> hard to file PRs for all the failures you see and mark them KFAIL?

I think that would be a step backwards as all it would do is fill the
bug database with reports like ``test failed''.

At least this does move things forward - it puts the tesuite in a state
where everyone and everyone can incrementally do the marking.

Andrew