From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 72361 invoked by alias); 14 Aug 2018 20:17:05 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Received: (qmail 72305 invoked by uid 89); 14 Aug 2018 20:17:01 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-3.0 required=5.0 tests=AWL,BAYES_00,SPF_HELO_PASS autolearn=ham version=3.3.2 spammy= X-HELO: mx1.redhat.com Received: from mx3-rdu2.redhat.com (HELO mx1.redhat.com) (66.187.233.73) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 14 Aug 2018 20:16:59 +0000 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1D932401C96A; Tue, 14 Aug 2018 20:16:58 +0000 (UTC) Received: from localhost (unused-10-15-17-196.yyz.redhat.com [10.15.17.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id F36E910075FF; Tue, 14 Aug 2018 20:16:57 +0000 (UTC) From: Sergio Durigan Junior To: Philippe Waroquiers Cc: Pedro Alves , GDB Patches , Joel Brobecker Subject: Re: [RFC/PATCH] Don't disable selftests in a non-development build References: <20180814054221.13061-1-sergiodj@redhat.com> <367277ad-d735-0854-7aca-1df9a1927115@redhat.com> <1534272169.15655.6.camel@skynet.be> Date: Tue, 14 Aug 2018 20:17:00 -0000 In-Reply-To: <1534272169.15655.6.camel@skynet.be> (Philippe Waroquiers's message of "Tue, 14 Aug 2018 20:42:49 +0200") Message-ID: <877eksln46.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-IsSubscribed: yes X-SW-Source: 2018-08/txt/msg00372.txt.bz2 On Tuesday, August 14 2018, Philippe Waroquiers wrote: > On Tue, 2018-08-14 at 19:08 +0100, Pedro Alves wrote: >> Another approach to addressing this issue here: >> >> > Due to the many racy testcases and random failures we see when running >> > the GDB testsuite, it is unfortunately not possible to perform a full >> > test when one is building a downstream package. As the Fedora GDB >> > maintainer and one of the Debian GDB uploaders, I feel like this >> > situation could be improved by, at least, executing our selftests >> > after the package has been built. However, we currently (for some >> > reason that is not clear by reading the archives, but see more below) >> > disable selftests on non-development builds. Therefore, this patch >> > aims to leave them enabled all the time, for everyone (including the >> > end users). >> >> ... is to come up with some small set of stable testcases that >> are considered the "smoke tests" and add a mechanism to run them. >> Could be just a list of testcases in a file that is passed to >> make check TESTS="list of basic tests here" >> or some make target like "make check-smoke", or something >> else even. > > Instead of (or in addition to) some list of tests that are known to be > non-racy/alwayd ok, > maybe it would be nice to have a list of tests that are known to > be racy/sometimes wrong ? > We e.g. could mark these racy tests as part of the test itself, > with some pseudo dejagnu code like : > > if [do_not_run_racy_or_sometimes_wrong_tests] { > return > } > > # or, if the test is only racy on some platform > if [do_not_run_racy_or_sometimes_wrong_tests && istarget x86_64-*-*] { > return > } Yes, this is a good idea. Pedro suggested something very similar (if not equal) a while ago, when I was doing some work on detecting racy tests for our BuildBot. The idea is to mark the known racy tests using something a la "setup_kfail". I haven't gotten the time to work on this yet, but it's on my TODO list. > The above would also help the gdb developers to interpret > the results of a test run. > I am always wondering if a failure I see can (or cannot) be > explained by the change I just did. It's a PITA, indeed. If it helps, the BuildBot automatically refreshes the list of known racy tests every week (or every day, for x86* builders). You can have a look at the lists here: https://git.sergiodj.net/gdb-xfails.git/tree/xfails So, for a list of known racy tests for Fedora-x86-64-m64: https://git.sergiodj.net/gdb-xfails.git/tree/xfails/Fedora-x86_64-m64/xfails/master/xfail.table Thanks, -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/