From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 5584 invoked by alias); 14 Mar 2016 12:32:05 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Received: (qmail 5563 invoked by uid 89); 14 Mar 2016 12:32:04 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_PASS autolearn=ham version=3.3.2 spammy=noise, H*r:Symantec, Thanks!, identifying X-HELO: usplmg21.ericsson.net Received: from usplmg21.ericsson.net (HELO usplmg21.ericsson.net) (198.24.6.65) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-SHA encrypted) ESMTPS; Mon, 14 Mar 2016 12:32:00 +0000 Received: from EUSAAHC008.ericsson.se (Unknown_Domain [147.117.188.96]) by usplmg21.ericsson.net (Symantec Mail Security) with SMTP id D2.71.32102.E1FA6E65; Mon, 14 Mar 2016 13:31:26 +0100 (CET) Received: from elxa4wqvvz1 (147.117.188.8) by smtps-am.internal.ericsson.com (147.117.188.96) with Microsoft SMTP Server (TLS) id 14.3.248.2; Mon, 14 Mar 2016 08:31:57 -0400 References: <87r3gcgm91.fsf@redhat.com> <878u22wxgk.fsf@redhat.com> User-agent: mu4e 0.9.17; emacs 24.4.1 From: Antoine Tremblay To: Sergio Durigan Junior CC: Antoine Tremblay , GDB Patches Subject: Re: [PATCH] Improve analysis of racy testcases In-Reply-To: <878u22wxgk.fsf@redhat.com> Date: Mon, 14 Mar 2016 12:32:00 -0000 Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-IsSubscribed: yes X-SW-Source: 2016-03/txt/msg00208.txt.bz2 Sergio Durigan Junior writes: > On Thursday, February 25 2016, Antoine Tremblay wrote: > >> Sergio Durigan Junior writes: >> >>> This patch is a proposal to introduce some mechanisms to identify racy >>> testcases present in our testsuite. As can be seen in previous >>> discussions, racy tests are really bothersome and cause our BuildBot to >>> pollute the gdb-testers mailing list with hundreds of false-positives >>> messages every month. Hopefully, by identifying these racy tests in >>> advance (and automatically) will contribute to the reduction of noise >>> traffic to gdb-testers, maybe to the point where we will be able to send >>> the failure messages directly to the authors of the commits. >>> [...] >> Thanks for this ! This was quite a problem for me while testing on arm. >> I'm testing it now... > > Thanks! Please let me know if you find anything wrong with the script. Just to let know you that the script worked as expected during my testing and it's quite nice to see. The tests however were more flacky then expected so I can't use the output directly guess I'll need a never ignore this test list... (like base/break.exp). Still it's very helpful. > >> One note maybe it would be nice output the list of unracy tests too to >> be able to auto-build a list of tests to run out of this since I'm not >> sure you can set an exclusion list ? > > Hm, it would be possible to output the non-racy tests, but only to a > different file (instead of outputting to stdout, as I'm doing with the > racy tests). Perhaps this could be a separate option to the script? > I'm not sure if the users would always want this information... > > As for the exclusion list you mentioned, DejaGNU's runtest allows one to > specify a --ignore flag with the names of the tests you don't want to > run. It should be possible to pass this via RUNTESTFLAGS, but I haven't > tried. I'll give it a go tomorrow. Great, I did not know that, I'll give it a try and that works no need to output the non-racy tests... Thanks, Antoine