Re: relying on testsuite results

[Mark Kettenis <mark.kettenis@xs4all.nl>]

> From: Tom Tromey <tromey@redhat.com>
> Reply-To: tromey@redhat.com
> 
> I generally do full regression tests for each patch.  ("Generally"
> because I occasionally forget, or something goes wrong and I don't
> notice.)  This is nice, although slow, and unfortunately also open to
> some kinds of major failure (e.g., a change of compiler negatively
> affecting baseline results).

And there you touch upon an important reason why it is so hard to make
all tests PASS.  In many cases our testsuite isn't just testing GDB in
isolation, but also the compiler, kernel or threads library.
Unfortunately the developers of those components don't run the GDB
testsuite as part of their regression testing.  So things are
sometimes broken and take a while to get fixed again.

That said, on my preferred platform (OpenBSD) the testsuite produces
fairly consistent results.  We use a stable toolchain (GCC 3.3.5,
binutils 2.15) which helps.  And a fair number of the threads-related
tests FAIL because of some Linuxisms that crept, which means they
often dom't even get to the racy bits.  But it is also because in the
past I spent time on actually fixing broken tests.