Re: [PATCH v2] Improve analysis of racy testcases

[Sergio Durigan Junior <sergiodj@redhat.com>]

On Tuesday, March 01 2016, Pedro Alves wrote:

> On 03/01/2016 05:51 AM, Sergio Durigan Junior wrote:
>
>>   # Note that we must resort to a recursive make invocation here,
>> @@ -190,6 +197,26 @@ DO_RUNTEST = \
>>   check-single:
>>   	$(DO_RUNTEST) $(RUNTESTFLAGS) $(expanded_tests_or_none)
>>
>> +check-single-racy:
>
> (...)
>
>> +	$(srcdir)/analyze-racy-logs.py \
>> +	  `ls racy_outputs/gdb*.sum` > racy.sum; \
>
> This ls invocation needs updating.

Fixed.

>> +	sed -n '/=== gdb Summary ===/,$$ p' racy.sum
>> +
>
>
>> diff --git a/gdb/testsuite/README b/gdb/testsuite/README
>> index 6b59027..043a8bd 100644
>> --- a/gdb/testsuite/README
>> +++ b/gdb/testsuite/README
>> @@ -50,6 +50,51 @@ to any non-empty value:
>>   If you want to use runtest directly instead of using the Makefile, see
>>   the description of GDB_PARALLEL below.
>>
>> +Racy testcases
>> +**************
>> +
>> +Sometimes, new testcases are added to the testsuite that are not
>> +entirely deterministic, and can randomly pass or fail.  We call them
>> +"racy testcases", and they can be bothersome when one is comparing
>> +different testsuite runs.  In order to help identifying them, it is
>> +possible to run the tests several times in a row and ask the testsuite
>> +machinery to analyze the results.  To do that, you need to specify the
>> +RACY_ITER environment variable to make:
>
> Nicer, thanks.  :-)
>
>
>> +
>> +    # Print the header.
>> +    print "\t\t=== gdb tests ===\n"
>
> "gdb racy tests" ?

Fixed.

>> +
>> +    # Print each test.
>> +    for line in sorted (racy_tests):
>> +        print line
>> +
>> +    # Print the summary.
>> +    print "\n"
>> +    print "\t\t=== gdb Summary ===\n"
>> +    print "# of racy testcases:\t\t%d" % len (racy_tests)
>
> AFAICS, it's # of racy tests, not # of racy testcases, right?

Yeah, fixed.

> Feel free to push with the above addressed.

Thanks, pushed.

  fb6a751f5f1fe7912e84dd90d06395c357da47c2

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/