From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 129028 invoked by alias); 6 Mar 2016 01:44:45 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Received: (qmail 128726 invoked by uid 89); 6 Mar 2016 01:44:44 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD,SPF_HELO_PASS autolearn=ham version=3.3.2 spammy=Sometimes, identifying X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-GCM-SHA384 encrypted) ESMTPS; Sun, 06 Mar 2016 01:44:42 +0000 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (Postfix) with ESMTPS id C57BA486AA for ; Sun, 6 Mar 2016 01:44:41 +0000 (UTC) Received: from localhost (unused-10-15-17-51.yyz.redhat.com [10.15.17.51]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u261ifKS004163 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 5 Mar 2016 20:44:41 -0500 From: Sergio Durigan Junior To: Pedro Alves Cc: GDB Patches Subject: Re: [PATCH v2] Improve analysis of racy testcases References: <87r3gcgm91.fsf@redhat.com> <87d1rewxst.fsf@redhat.com> <56D58E00.2030503@redhat.com> X-URL: http://blog.sergiodj.net Date: Sun, 06 Mar 2016 01:44:00 -0000 In-Reply-To: <56D58E00.2030503@redhat.com> (Pedro Alves's message of "Tue, 1 Mar 2016 12:41:36 +0000") Message-ID: <87ziucv0qf.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-IsSubscribed: yes X-SW-Source: 2016-03/txt/msg00100.txt.bz2 On Tuesday, March 01 2016, Pedro Alves wrote: > On 03/01/2016 05:51 AM, Sergio Durigan Junior wrote: > >> # Note that we must resort to a recursive make invocation here, >> @@ -190,6 +197,26 @@ DO_RUNTEST = \ >> check-single: >> $(DO_RUNTEST) $(RUNTESTFLAGS) $(expanded_tests_or_none) >> >> +check-single-racy: > > (...) > >> + $(srcdir)/analyze-racy-logs.py \ >> + `ls racy_outputs/gdb*.sum` > racy.sum; \ > > This ls invocation needs updating. Fixed. >> + sed -n '/=== gdb Summary ===/,$$ p' racy.sum >> + > > >> diff --git a/gdb/testsuite/README b/gdb/testsuite/README >> index 6b59027..043a8bd 100644 >> --- a/gdb/testsuite/README >> +++ b/gdb/testsuite/README >> @@ -50,6 +50,51 @@ to any non-empty value: >> If you want to use runtest directly instead of using the Makefile, see >> the description of GDB_PARALLEL below. >> >> +Racy testcases >> +************** >> + >> +Sometimes, new testcases are added to the testsuite that are not >> +entirely deterministic, and can randomly pass or fail. We call them >> +"racy testcases", and they can be bothersome when one is comparing >> +different testsuite runs. In order to help identifying them, it is >> +possible to run the tests several times in a row and ask the testsuite >> +machinery to analyze the results. To do that, you need to specify the >> +RACY_ITER environment variable to make: > > Nicer, thanks. :-) > > >> + >> + # Print the header. >> + print "\t\t=== gdb tests ===\n" > > "gdb racy tests" ? Fixed. >> + >> + # Print each test. >> + for line in sorted (racy_tests): >> + print line >> + >> + # Print the summary. >> + print "\n" >> + print "\t\t=== gdb Summary ===\n" >> + print "# of racy testcases:\t\t%d" % len (racy_tests) > > AFAICS, it's # of racy tests, not # of racy testcases, right? Yeah, fixed. > Feel free to push with the above addressed. Thanks, pushed. fb6a751f5f1fe7912e84dd90d06395c357da47c2 -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/