From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 3872 invoked by alias); 18 Sep 2003 02:01:46 -0000 Mailing-List: contact gdb-patches-help@sources.redhat.com; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sources.redhat.com Received: (qmail 3865 invoked from network); 18 Sep 2003 02:01:45 -0000 Received: from unknown (HELO concert.shout.net) (204.253.184.25) by sources.redhat.com with SMTP; 18 Sep 2003 02:01:45 -0000 Received: from duracef.shout.net (duracef.shout.net [204.253.184.12]) by concert.shout.net (8.12.9/8.12.9) with ESMTP id h8I21h1W001570; Wed, 17 Sep 2003 21:01:43 -0500 Received: from duracef.shout.net (localhost [127.0.0.1]) by duracef.shout.net (8.12.9/8.12.9) with ESMTP id h8I21hHK013788; Wed, 17 Sep 2003 21:01:43 -0500 Received: (from mec@localhost) by duracef.shout.net (8.12.9/8.12.9/Submit) id h8I21hCr013787; Wed, 17 Sep 2003 22:01:43 -0400 Date: Thu, 18 Sep 2003 02:01:00 -0000 From: Michael Elizabeth Chastain Message-Id: <200309180201.h8I21hCr013787@duracef.shout.net> To: carlton@kealia.com, gdb-patches@sources.redhat.com Subject: Re: [testsuite] add gdb.cp/gdb1355.exp X-SW-Source: 2003-09/txt/msg00392.txt.bz2 Okay, I'm willing to make it an XFAIL rather than a KFAIL. I remember those threads. The XFAIL feels odd but it does stand for "external" and this bug is indeed a textbook external bug. As far as making it a FAIL goes, I think David's argument comes down to: people pay attention to FAIL, but not to XFAIL or KFAIL. I really don't like that situation. I see the question as: whether to fight the situation and use more accurate XFAIL/KFAIL rather than generic FAIL, or to acquiesce to situation and give people what they expect to see. I favor the former. > Scenario 1: The bug is fixed. A year later, the bug in question > regresses. But, if you just run 'make check', this isn't so obvious: > no message gets printed in the output, and while the numbers at the > end of the output will change, those numbers fluctuate for lots of > reasons. So it will only be noticed by somebody who actually looks at > gdb.sum (or does the moral equivalent); ideally that will happen, but > I'd rather not count on it (especially if the bug is on an obscure > platform). I'm willing to count on it. Because if they just grep for '^FAIL' then they will miss any tests that barf out completely with an ERROR at the beginning and abort. I have little sympathy for people who analyze gdb.sum that way. dc> Scenario 2: The fix works on platform A, but not on platform B. And dc> nobody using platform B has been paying close enough attention to the dc> discussion of the bug to realize that the test is supposed to start dc> passing. Then, nobody might ever realize that something's wrong: dc> platform A users think that everything is fine (which is the case for dc> them), and platform B have no indication from the testsuite that dc> something is wrong and that the patch didn't work as indicated. Ah, this is the old scenario "change the KFAIL/XFAIL to a FAIL" makes people on platform B aware that something has regressed (I always investigate transitions from KFAIL -> FAIL). I have some sympathy for this. dc> Which is why, in the situation at question, I like it that you have the dc> explicit failure branches, containing an informative comment as to when dc> the bug occurred - I just want the letter 'k' removed in a few places. dc> :-) So we have two proposals here: I'd like to make this an XFAIL, and David would like to make it a FAIL. Daniel, or anyone else, do you have a preference? Michael C