From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 81440 invoked by alias); 12 Jan 2018 10:23:14 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Received: (qmail 81423 invoked by uid 89); 12 Jan 2018 10:23:13 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-4.0 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_NONE,SPF_PASS autolearn=ham version=3.3.2 spammy=fear, legitimately, hundred X-HELO: mail-wm0-f44.google.com Received: from mail-wm0-f44.google.com (HELO mail-wm0-f44.google.com) (74.125.82.44) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 12 Jan 2018 10:23:12 +0000 Received: by mail-wm0-f44.google.com with SMTP id i11so10722058wmf.4 for ; Fri, 12 Jan 2018 02:23:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=ENXruir0HWn/2qbXlWeWdFss2MEuCdNSjXCWWyJ5a/k=; b=eAIbc2P7fhhCIcPcvF+8SofbB8zpukaxw5MKhRCXtAeacFhK3dox2lbRzzffm5Kp37 3TUm77pjoKUVz6voyEoXzoTE81Kk+ijKYvDK90c0bmdGD7Xj8tACsxhYDPDKQCmh/uLS df2WDvV0nDVH1eW0lzqUdlyM0pQuE/7zuf8J+5zrDCavR5aWpDY0bVueHHotxVk4iQx4 f1MTHerLCpKIjS0ssqm0nXXPXNa5THGS1HVE/rc9df2cYpHgtmYRY/UHwx43HWMUeQt4 ZaOPBjWLnDDVpFZnL/F/7WocrdlnX+Y9Y8UlGBvsGjGCjbDYlZ+grCPfCszchy4QW7iY RiZg== X-Gm-Message-State: AKwxytdWYthpC7GeVUWH3a2A+881b0zA3LDfrxYCL7vHcpWmYNuRVzsQ XHwoaG1oLa0Q++0CzIfbLD48bA== X-Google-Smtp-Source: ACJfBou8/I/fFJo81zblwWF80WKu1oPMlPrfdEL1WQpo3kGiphQMxjAcGy5o5O+4eL5QNYnGicSIag== X-Received: by 10.28.112.13 with SMTP id l13mr3690650wmc.16.1515752589766; Fri, 12 Jan 2018 02:23:09 -0800 (PST) Received: from localhost ([81.141.199.69]) by smtp.gmail.com with ESMTPSA id 186sm3007779wmm.10.2018.01.12.02.23.08 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Jan 2018 02:23:09 -0800 (PST) Date: Fri, 12 Jan 2018 10:23:00 -0000 From: Andrew Burgess To: Simon Marchi Cc: gdb-patches@sourceware.org Subject: Re: [PATCH] gdb/testsuite: Don't attempt tests if they fail to compile Message-ID: <20180112102308.GN3026@embecosm.com> References: <20180111190055.4875-1-andrew.burgess@embecosm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Editor: GNU Emacs [ http://www.gnu.org/software/emacs ] User-Agent: Mutt/1.9.1 (2017-09-22) X-IsSubscribed: yes X-SW-Source: 2018-01/txt/msg00241.txt.bz2 * Simon Marchi [2018-01-11 17:03:51 -0500]: > On 2018-01-11 02:00 PM, Andrew Burgess wrote: > > In the gdb.base/whatis-ptype-typedefs.exp test, if the test program > > fails to compile, don't run the tests. > > Hi Andrew, > > That would make the test similar to other test, in that if we fail to > build the test program it's not a failure (it shows as UNTESTED, doesn't > make the test run fail). I find it's a strange behavior though. If a > test program starts not building for some reason, I'd certainly like to > know (e.g. it could be UNRESOLVED), instead of it silently failing. > > Any other opinion? If the test fails to compile we don't get a silent failure, as you mention, we get the UNTESTED. Changing this to something stronger, like UNRESOLVED, would I fear make cases where we legitimately can't compile a test program seem worse than they really are. The concern about missing the case where a test program goes from compiling to not compiling is fair, however, I don't think that it's something we need to worry about. My understanding of the "normal" testing flow for GDB is to compare against a baseline set of results, a few hundred tests disappearing should raise a red flag, and once the developer has realised that this particular test script has something weird going on, the extra UNTESTED should guide them to the cause of the problem. The failed to prepare leading to skipping the tests seems like the "standard" pattern within the GDB testsuite, so, if you agree, I think having this test fall in line with that is probably a good thing. That doesn't mean we can't change the standard pattern in the future if we can come up with a better model (though I don't have any good suggestions). Thanks, Andrew