Re: Unreliable test suites?

[<Paul_Koning@Dell.com>]

> On Oct 26, 2015, at 1:36 AM, Doug Evans <xdje42@gmail.com> wrote:
> 
> On Wed, Oct 14, 2015 at 12:58 PM,  <Paul_Koning@dell.com> wrote:
>> I'm doing some work on gdb and want to run the testsuites to confirm that I didn't mess it up.
>> 
>> The problem is that a number of them seem to be quite unreliable.  I've seen test runs where gdb.btrace/step.exp and/or stepi.exp have a pile of failures, but then when I rerun either just those tests, or the whole suite, they pass.
>> 
>> Since I haven't a clue how the reverse execution stuff works, I don't know if this is expected.  It seems strange.  I also don't know what to do about it if it's not supposed to be like that. For now, I'm just running things a couple of times, and if they pass once, I call it good enough.
> 
> Those particular tests don't fail for me, even with check-parallel,
> but then it could depend on the target.
> OTOH several tests *are* flaky, especially under load.
> 
> What I normally do is for the failures,
> run just those tests one at a time (to reduce load induced failures).
> And do that in the before and after trees.
> If they fail in the before tree too, then don't worry about it.

Thanks.  I've been doing this sort of thing as well, and that helps.  Running just one test as a confirmation seems to be particularly effective.

	paul