Clean up gdb.c++ tests for dwarf 1

Clean up gdb.c++ tests for dwarf 1

[Michael Elizabeth Chastain]

gdb HEAD%20030205 has loads of this stuff in the C++ test suite:

  setup_xfail_format "DWARF 1"

There are 81 instances of this.

Some test scripts have them, and some don't.  I'd like to get rid
of them and replace them with something more coherent -- or nothing.

I see four choices.

(1) Just remove these calls to setup_xfail_format.  If someone runs the
    gdb test suite with DWARF 1, the test suite will do its job and give
    FAIL results for all the C++ tests that do not work with DWARF 1.

(2) Change the tests so that each C++ test script reports a single
    UNSUPPORTED for the whole script (if the debug format is DWARF 1).
    
    This requires about five lines of code per script.  This cannot be
    centralized in 'skip_cplus_tests', because the check for the debug
    format has to happen after the test script has compiled the test
    program, started gdb, and proceeded to main, so that 'info source'
    works.

(3) Leave the mess alone.

(4) Start fixing the DWARF 1 support.  I'll start testing DWARF 1 and
    filing bug reports for the things that don't work.

My preference order is (1) - (2) - (3) - (4).

What do you think?

Michael C

Re: Clean up gdb.c++ tests for dwarf 1

[David Carlton]

On Wed, 5 Feb 2003 15:30:17 -0600, Michael Elizabeth Chastain
<mec@shout.net> said:

> My preference order is (1) - (2) - (3) - (4).

I agree.  There's no reason to fix C++ DWARF 1 support that I can see,
so (4) is bad.  We're trying to clean up xfails; I don't think those
xfails give us any useful information, so we might as well get rid of
them, so (3) is bad.  (1) and (2) seem approximately equally desirable
to me; if (1) is easier, then we should just go with that.  There's no
reason to spend any more time on this issue than is absolutely
necessary.

David Carlton
carlton@math.stanford.edu

Re: Clean up gdb.c++ tests for dwarf 1

[Jim Blandy]

Michael Elizabeth Chastain <mec@shout.net> writes:
> gdb HEAD%20030205 has loads of this stuff in the C++ test suite:
> 
>   setup_xfail_format "DWARF 1"
> 
> There are 81 instances of this.
> 
> Some test scripts have them, and some don't.  I'd like to get rid
> of them and replace them with something more coherent -- or nothing.
> 
> I see four choices.
> 
> (1) Just remove these calls to setup_xfail_format.  If someone runs the
>     gdb test suite with DWARF 1, the test suite will do its job and give
>     FAIL results for all the C++ tests that do not work with DWARF 1.

Your rationale here is that, since we don't really know which of these
failures are genuine, can't-be-done-with-Dwarf-1 expected failures,
and which are GDB bugs, you want to dump them all into the "genuine
bug" category and start re-categorizing, using our modern
interpretation of XFAIL and KFAIL?

Re: Clean up gdb.c++ tests for dwarf 1

[Daniel Jacobowitz]

On Wed, Feb 05, 2003 at 04:29:14PM -0500, Jim Blandy wrote:
> 
> Michael Elizabeth Chastain <mec@shout.net> writes:
> > gdb HEAD%20030205 has loads of this stuff in the C++ test suite:
> > 
> >   setup_xfail_format "DWARF 1"
> > 
> > There are 81 instances of this.
> > 
> > Some test scripts have them, and some don't.  I'd like to get rid
> > of them and replace them with something more coherent -- or nothing.
> > 
> > I see four choices.
> > 
> > (1) Just remove these calls to setup_xfail_format.  If someone runs the
> >     gdb test suite with DWARF 1, the test suite will do its job and give
> >     FAIL results for all the C++ tests that do not work with DWARF 1.
> 
> Your rationale here is that, since we don't really know which of these
> failures are genuine, can't-be-done-with-Dwarf-1 expected failures,
> and which are GDB bugs, you want to dump them all into the "genuine
> bug" category and start re-categorizing, using our modern
> interpretation of XFAIL and KFAIL?

I think his rationale here is, we don't know which of these failures
are which, and we don't care...

I certainly don't care either.  I like (1).

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Clean up gdb.c++ tests for dwarf 1

[David Carlton]

On 05 Feb 2003 16:29:14 -0500, Jim Blandy <jimb@redhat.com> said:
> Michael Elizabeth Chastain <mec@shout.net> writes:

>> (1) Just remove these calls to setup_xfail_format.  If someone runs the
>> gdb test suite with DWARF 1, the test suite will do its job and
>> give FAIL results for all the C++ tests that do not work with DWARF
>> 1.

> Your rationale here is that, since we don't really know which of
> these failures are genuine, can't-be-done-with-Dwarf-1 expected
> failures, and which are GDB bugs, you want to dump them all into the
> "genuine bug" category and start re-categorizing, using our modern
> interpretation of XFAIL and KFAIL?

I won't speak for Michael, but my rationale is that, based on
discussions over the last few weeks and based on the patch to
gdb.texinfo that I committed on Monday, we currently don't support C++
on DWARF 1.  The testsuite is cluttered with XFAILs that we have no
reason to believe are either correct or comprehensive; the two obvious
choices, then, are to either throw them out entirely or to XFAIL
everything, which are Michael's options 1 and 2.

If at some point in the future somebody decides to start working on
improving C++ DWARF 1 support in GDB (and, presumably, in GCC), then
that person can take the time to sort through what works, what doesn't
work because of failures in GDB, what doesn't work because of failures
in GCC, and what doesn't work because of limitations in the debug
format.  I doubt anybody will be motivated to do so; getting rid of
the current XFAIL's won't hurt such an effort, however.

David Carlton
carlton@math.stanford.edu

Re: Clean up gdb.c++ tests for dwarf 1

[Elena Zannoni]

Michael Elizabeth Chastain writes:
 > gdb HEAD%20030205 has loads of this stuff in the C++ test suite:
 > 
 >   setup_xfail_format "DWARF 1"
 > 
 > There are 81 instances of this.
 > 
 > Some test scripts have them, and some don't.  I'd like to get rid
 > of them and replace them with something more coherent -- or nothing.
 > 
 > I see four choices.
 > 
 > (1) Just remove these calls to setup_xfail_format.  If someone runs the
 >     gdb test suite with DWARF 1, the test suite will do its job and give
 >     FAIL results for all the C++ tests that do not work with DWARF 1.
 > 
 > (2) Change the tests so that each C++ test script reports a single
 >     UNSUPPORTED for the whole script (if the debug format is DWARF 1).
 >     
 >     This requires about five lines of code per script.  This cannot be
 >     centralized in 'skip_cplus_tests', because the check for the debug
 >     format has to happen after the test script has compiled the test
 >     program, started gdb, and proceeded to main, so that 'info source'
 >     works.
 > 
 > (3) Leave the mess alone.
 > 
 > (4) Start fixing the DWARF 1 support.  I'll start testing DWARF 1 and
 >     filing bug reports for the things that don't work.
 > 
 > My preference order is (1) - (2) - (3) - (4).
 > 
 > What do you think?
 > 
 > Michael C


My preferences are (1) (1) (1) (1) (1) (1).....

elena

Re: Clean up gdb.c++ tests for dwarf 1

[Michael Elizabeth Chastain]

I'm hearing a lot of support for option (1) so I will prepare a patch
and back it up with some test runs.  I did test DWARF 1 a year or two
ago and I'm kinda curious what it looks like now.

This will be an "everything on the Internet takes a week" patch because
someone might pop up with a DWARF 1 story that foils everything.

Michael C

Re: Clean up gdb.c++ tests for dwarf 1

[Michael Elizabeth Chastain]

Jim B posits:
> Your rationale here is that, since we don't really know which of these
> failures are genuine, can't-be-done-with-Dwarf-1 expected failures,
> and which are GDB bugs, you want to dump them all into the "genuine
> bug" category and start re-categorizing, using our modern
> interpretation of XFAIL and KFAIL?

You could interpret plan (1) that way.  Consider the scenario where
someone is still testing gdb with DWARF 1.  They are getting a mixture
of FAIL's and XFAIL's now.  After plan (1), they will get a lot more
FAIL's and a lot less XFAIL's.

There is a lot of bit rot and lies in the test suite.  If my task were
to evaluate c++ support with DWARF 1, then I would *start* by assuming
that all the XFAIL's are lying about the "X" part, and I would treat
them the same as FAIL's anyways.  So that corresponds to what you said.

Of course I am hoping that after I dump them all into the 'genuine bug'
category, that they will sit there until DWARF 1 is obsolete and
removed, and no one will ever have to 'start re-categorizing'.
But someone can do that if they need to.

Michael C