From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 13269 invoked by alias); 10 Jul 2015 14:33:47 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Received: (qmail 13260 invoked by uid 89); 10 Jul 2015 14:33:46 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.7 required=5.0 tests=AWL,BAYES_00,KAM_LAZY_DOMAIN_SECURITY,RP_MATCHES_RCVD,SPF_HELO_PASS autolearn=no version=3.3.2 X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-GCM-SHA384 encrypted) ESMTPS; Fri, 10 Jul 2015 14:33:45 +0000 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 7A9722E6DC0; Fri, 10 Jul 2015 14:33:44 +0000 (UTC) Received: from [127.0.0.1] (ovpn01.gateway.prod.ext.ams2.redhat.com [10.39.146.11]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t6AEXgsR026401; Fri, 10 Jul 2015 10:33:43 -0400 Message-ID: <559FD7C6.2060502@redhat.com> Date: Fri, 10 Jul 2015 14:33:00 -0000 From: Pedro Alves User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: David Edelsohn , Joel Brobecker CC: GDB Patches Subject: Re: GDB 7.9.90 available for testing References: <20150709232141.GA7406@adacore.com> <20150710034255.GB7406@adacore.com> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-SW-Source: 2015-07/txt/msg00325.txt.bz2 On 07/10/2015 03:04 PM, David Edelsohn wrote: > On Thu, Jul 9, 2015 at 11:42 PM, Joel Brobecker wrote: >>> I'm not certain if the baselines truly are accurate for all >>> buildslaves, but it seems strange to create a release when the >>> buildbot testsuite results show patches causing new failures. >> >> To me, you are saying the same thing, and I don't disagree with you. >> I said I didn't know that the buildBots were showing regressions. >> Of course I would have held the creation of the branch if I had >> known about this. But I didn't, and so here we are. Now we all know, >> and the only way forward is to look at those regressions, and decide >> what to do. We can and will delay the release if we have to. > > Joel, > > We are agreeing. I was trying to provide some additional information > about interpretation of the buildbot status. > > I am note two things about the buildbots: > > 1) Their color-coded "regression status" apparently is a comparison of > the testsuite between a "base" run and the current run. This is due to > few or no targets have completely clean testsuite runs to consider > "green". Because there has been some adjustment and tweaking while > buildbots were added, the first run was not necessarily the ideal one > to choose as the "base" run, i.e., "regressions" may be due to changes > in the measurements after the first "base" run, not new failing tests. There's no single "base" run, actually. The baseline is dynamically adjusted at each build; it's a moving baseline, and it's per test (single PASS/FAIL, not file). As soon as a test PASSes, it's added to the baseline. That means that if some test is racy, it'll sometimes FAIL, and then a few builds later it'll PASS, at which point the PASS is recorded in the baseline, and then a few builds again later, the test FAIL again, and so the buildbot email report mentions the regression against the baseline. In sum, if a test goes FAIL -> PASS -> FAIL -> PASS on and on over builds, you'll constantly get reports of regressions against the baseline for that racy test. For each build, you can find the baseline file in the corresponding git commit pointed at in the email report. E.g., see the "baseline" file here: http://gdb-build.sergiodj.net/cgit/AIX-POWER7-plain/.git/tree/?h=master&id=42b08c842d422ae995d244efeb1a85aa8a082e7b The gdb.thread/ FAILs you see on AIX seem to fall in that category. >From the results, it looks to me that those are caused by the AIX port not implementing schedlock correctly. Is anyone from IBM available to look at these? The gdb.cp/var-tag.exp FAILs currently reported on AIX are not really regressions, but new FAILs. And they are really a test problem, not a GDB bug. They actually depend on compiler or debug info format used, not system. > > 2) Separate from the "regression" status, a quick inspection of some > testsuite output in the buildbots show the introduction of new errors > with recent commits. Even if the overall regression status is not an > accurate measure of the state of GDB on those targets, the change in > regression status that is not monotonic reduction in regressions in > preparation for a release is disappointing. > > I'm not demanding STOP SHIP. GDB may not necessarily be in a bad > state for a release. I don't know how this compares to the regression > status of previous releases. > > I hope that GDB developers will become more aware of the effects of > their patches on multiple targets. This is just a misunderstanding. Thanks, Pedro Alves