From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 2923 invoked by alias); 23 Sep 2014 15:16:12 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Received: (qmail 2909 invoked by uid 89); 23 Sep 2014 15:16:09 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.3 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 X-HELO: usevmg21.ericsson.net Received: from usevmg21.ericsson.net (HELO usevmg21.ericsson.net) (198.24.6.65) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-SHA encrypted) ESMTPS; Tue, 23 Sep 2014 15:16:07 +0000 Received: from EUSAAHC004.ericsson.se (Unknown_Domain [147.117.188.84]) by usevmg21.ericsson.net (Symantec Mail Security) with SMTP id F1.29.25146.32631245; Tue, 23 Sep 2014 10:58:11 +0200 (CEST) Received: from [142.133.110.254] (147.117.188.8) by smtps-am.internal.ericsson.com (147.117.188.84) with Microsoft SMTP Server (TLS) id 14.3.174.1; Tue, 23 Sep 2014 11:16:05 -0400 Message-ID: <54218EB4.4080906@ericsson.com> Date: Tue, 23 Sep 2014 15:16:00 -0000 From: Simon Marchi User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.1.1 MIME-Version: 1.0 To: Subject: Re: automated testing comment [Re: time to workaround libc/13097 in fsf gdb?] References: <5411CFAE.7040805@redhat.com> <20140912115452.GA5626@host2.jankratochvil.net> <5412E3AC.80203@redhat.com> <20140912123320.GA8704@host2.jankratochvil.net> <5412EB1F.40309@redhat.com> <20140917201049.GA22880@host2.jankratochvil.net> <541C3FCF.4000400@redhat.com> <541C409E.6010408@redhat.com> <20140920213033.GA6255@host2.jankratochvil.net> <541F2311.1040404@redhat.com> <20140923105855.GA10164@host2.jankratochvil.net> <54216860.7060008@redhat.com> In-Reply-To: <54216860.7060008@redhat.com> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-IsSubscribed: yes X-SW-Source: 2014-09/txt/msg00696.txt.bz2 On 2014-09-23 08:32 AM, Pedro Alves wrote: >> And sure deploying automated testing with the current GDB testsuite as is >> would not work now automatically as the testsuite has fuzzy results. > > We should be able to filter those out though. Of course ideally we'd > just fix them to not be fuzzy... If we adopt automated testing as part of the process and those fuzzy tests are in the way, it will help identify them and it will give some a motivation to fix them. What is bugging me is also the failing test cases in master. In order to make Jenkins (or similar) useful, you need a baseline where everything is green. Otherwise all patches that will get tested will fail and it will be difficult to weed out the actually failing patches from the good ones. Unless there is actually a way to make a pass/fail/unresolved "diff" that compares the patch's results to those of the latest master commit's results. Last time I checked, it didn't seem obvious. Simon