Re: relying on testsuite results

[Tom Tromey <tromey@redhat.com>]

>>>>> "Thiago" == Thiago Jung Bauermann <bauerman@br.ibm.com> writes:

Thiago> So the most use I get from the testsuite is to run regression
Thiago> tests on each patch I submit, and tediously eyeball the diff
Thiago> looking to see if any of the PASS<->FAIL flips actuallly mean
Thiago> something.

Yeah, me too.

These non-deterministic tests may add something, in one scenario: if
they start failing consistently, and somebody notices that, then they
may provide information about a real regression.

I'm sure I won't notice if they start failing consistently, though.
I already ignore them.

I generally do full regression tests for each patch.  ("Generally"
because I occasionally forget, or something goes wrong and I don't
notice.)  This is nice, although slow, and unfortunately also open to
some kinds of major failure (e.g., a change of compiler negatively
affecting baseline results).

Tom