From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <gdb-patches-return-116101-listarch-gdb-patches=sources.redhat.com@sourceware.org>
Received: (qmail 6573 invoked by alias); 23 Sep 2014 14:48:49 -0000
Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm
Precedence: bulk
List-Id: <gdb-patches.sourceware.org>
List-Subscribe: <mailto:gdb-patches-subscribe@sourceware.org>
List-Archive: <http://sourceware.org/ml/gdb-patches/>
List-Post: <mailto:gdb-patches@sourceware.org>
List-Help: <mailto:gdb-patches-help@sourceware.org>, <http://sourceware.org/ml/#faqs>
Sender: gdb-patches-owner@sourceware.org
Received: (qmail 6562 invoked by uid 89); 23 Sep 2014 14:48:48 -0000
Authentication-Results: sourceware.org; auth=none
X-Virus-Found: No
X-Spam-SWARE-Status: No, score=-2.1 required=5.0 tests=AWL,BAYES_00,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2
X-HELO: mail-wi0-f182.google.com
Received: from mail-wi0-f182.google.com (HELO mail-wi0-f182.google.com) (209.85.212.182) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-SHA encrypted) ESMTPS; Tue, 23 Sep 2014 14:48:47 +0000
Received: by mail-wi0-f182.google.com with SMTP id d1so5253009wiv.3        for <gdb-patches@sourceware.org>; Tue, 23 Sep 2014 07:48:44 -0700 (PDT)
MIME-Version: 1.0
X-Received: by 10.180.108.176 with SMTP id hl16mr23125305wib.4.1411483724812; Tue, 23 Sep 2014 07:48:44 -0700 (PDT)
Received: by 10.27.202.84 with HTTP; Tue, 23 Sep 2014 07:48:44 -0700 (PDT)
In-Reply-To: <20140923105855.GA10164@host2.jankratochvil.net>
References: <5411CFAE.7040805@redhat.com>	<20140912115452.GA5626@host2.jankratochvil.net>	<5412E3AC.80203@redhat.com>	<20140912123320.GA8704@host2.jankratochvil.net>	<5412EB1F.40309@redhat.com>	<20140917201049.GA22880@host2.jankratochvil.net>	<541C3FCF.4000400@redhat.com>	<541C409E.6010408@redhat.com>	<20140920213033.GA6255@host2.jankratochvil.net>	<541F2311.1040404@redhat.com>	<20140923105855.GA10164@host2.jankratochvil.net>
Date: Tue, 23 Sep 2014 14:48:00 -0000
Message-ID: <CAP9bCMS1R1S36oh77wOU6FgphGY4+Q2t-Pg20ANi2JWWYe1xYA@mail.gmail.com>
Subject: Re: automated testing comment [Re: time to workaround libc/13097 in fsf gdb?]
From: Doug Evans <xdje42@gmail.com>
To: Jan Kratochvil <jan.kratochvil@redhat.com>
Cc: Pedro Alves <palves@redhat.com>, 	"gdb-patches@sourceware.org" <gdb-patches@sourceware.org>
Content-Type: text/plain; charset=UTF-8
X-IsSubscribed: yes
X-SW-Source: 2014-09/txt/msg00692.txt.bz2

On Tue, Sep 23, 2014 at 3:58 AM, Jan Kratochvil
<jan.kratochvil@redhat.com> wrote:
> And sure deploying automated testing with the current GDB testsuite as is
> would not work now automatically as the testsuite has fuzzy results.  Although
> at least running new testcases (from the patch under review) would work which
> would be sufficient in this case (but not in other cases - regression cases).

Sometimes tests are flaky and one isn't aware of it, and not
everything gets caught by patch review.

IWBN if automated testing reran everything failing test N times (10?)
and reported those results as something like "fail" (always failed) or
"flaky" (inconsistent).