Re: Running the test suite

[Daniel Jacobowitz <drow@mvista.com>]

On Fri, Sep 28, 2001 at 01:55:22PM -0600, Tom Tromey wrote:
> I ran the gdb test suite today.  I get more than a few errors (211
> FAIL).  I'd like to know whether these errors come from my hacks or
> are just expected.
> 
> Is there some "standard" number of fails I should expect?
> 
> Soon I'll back out my patches, run the test suite, and then compare
> results.  That's a pain though.

In general, the testsuite is exceedingly sensitive to compiler changes;
hopefully someday we'll put together XFAILs in such a way that we can
have no failures, or at least document a few platforms that pass.

For comparison, here's a clean testsuite run on i386-linux, with gcc
2.95.3 (+ Debian/unstable patches):

# of expected passes            7560
# of unexpected failures        143
# of unexpected successes       32
# of expected failures          192
# of unresolved testcases       104
# of untested testcases         6

And with a couple of testsuite patches I've posted today, and one
mildly destabilizing patch that I'm working on:
# of expected passes            7570
# of unexpected failures        134
# of unexpected successes       34
# of expected failures          190
# of unresolved testcases       104
# of untested testcases         6


The best you can do right now is to run the testsuite before and after
patches.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer