Re: relying on testsuite results

Tom Tromey
2009-04-06T19:51:09Z

>>>>> "Thiago" == Thiago Jung Bauermann <bauerman@br.ibm.com> writes:

Thiago> So the most use I get from the testsuite is to run regression
Thiago> tests on each patch I submit, and tediously eyeball the diff
Thiago> looking to see if any of the PASS<->FAIL flips actuallly mean
Thiago> something.

Yeah, me too.

These non-deterministic tests may add something, in one scenario: if
they start failing consistently, and somebody notices that, then they
may provide information about a real regression.

I'm sure I won't notice if they start failing consistently, though.
I already ignore them.

I generally do full regression tests for each patch. ("Generally"
because I occasionally forget, or something goes wrong and I don't
notice.) This is nice, although slow, and unfortunately also open to
some kinds of major failure (e.g., a change of compiler negatively
affecting baseline results).

Tom