From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 22519 invoked by alias); 1 Mar 2016 12:41:40 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Received: (qmail 22508 invoked by uid 89); 1 Mar 2016 12:41:40 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD,SPF_HELO_PASS autolearn=ham version=3.3.2 spammy= X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-GCM-SHA384 encrypted) ESMTPS; Tue, 01 Mar 2016 12:41:39 +0000 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id F3D038E3CF for ; Tue, 1 Mar 2016 12:41:37 +0000 (UTC) Received: from [127.0.0.1] (ovpn01.gateway.prod.ext.phx2.redhat.com [10.5.9.1]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u21Cfamg019835; Tue, 1 Mar 2016 07:41:37 -0500 Subject: Re: [PATCH v2] Improve analysis of racy testcases To: Sergio Durigan Junior , GDB Patches References: <87r3gcgm91.fsf@redhat.com> <87d1rewxst.fsf@redhat.com> From: Pedro Alves Message-ID: <56D58E00.2030503@redhat.com> Date: Tue, 01 Mar 2016 12:41:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: <87d1rewxst.fsf@redhat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-SW-Source: 2016-03/txt/msg00006.txt.bz2 On 03/01/2016 05:51 AM, Sergio Durigan Junior wrote: > # Note that we must resort to a recursive make invocation here, > @@ -190,6 +197,26 @@ DO_RUNTEST = \ > check-single: > $(DO_RUNTEST) $(RUNTESTFLAGS) $(expanded_tests_or_none) > > +check-single-racy: (...) > + $(srcdir)/analyze-racy-logs.py \ > + `ls racy_outputs/gdb*.sum` > racy.sum; \ This ls invocation needs updating. > + sed -n '/=== gdb Summary ===/,$$ p' racy.sum > + > diff --git a/gdb/testsuite/README b/gdb/testsuite/README > index 6b59027..043a8bd 100644 > --- a/gdb/testsuite/README > +++ b/gdb/testsuite/README > @@ -50,6 +50,51 @@ to any non-empty value: > If you want to use runtest directly instead of using the Makefile, see > the description of GDB_PARALLEL below. > > +Racy testcases > +************** > + > +Sometimes, new testcases are added to the testsuite that are not > +entirely deterministic, and can randomly pass or fail. We call them > +"racy testcases", and they can be bothersome when one is comparing > +different testsuite runs. In order to help identifying them, it is > +possible to run the tests several times in a row and ask the testsuite > +machinery to analyze the results. To do that, you need to specify the > +RACY_ITER environment variable to make: Nicer, thanks. :-) > + > + # Print the header. > + print "\t\t=== gdb tests ===\n" "gdb racy tests" ? > + > + # Print each test. > + for line in sorted (racy_tests): > + print line > + > + # Print the summary. > + print "\n" > + print "\t\t=== gdb Summary ===\n" > + print "# of racy testcases:\t\t%d" % len (racy_tests) AFAICS, it's # of racy tests, not # of racy testcases, right? Feel free to push with the above addressed. Thanks, Pedro Alves