From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 31322 invoked by alias); 18 Sep 2003 15:38:27 -0000 Mailing-List: contact gdb-patches-help@sources.redhat.com; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sources.redhat.com Received: (qmail 31309 invoked from network); 18 Sep 2003 15:38:27 -0000 Received: from unknown (HELO hawaii.kealia.com) (209.3.10.89) by sources.redhat.com with SMTP; 18 Sep 2003 15:38:27 -0000 Received: by hawaii.kealia.com (Postfix, from userid 2049) id CE0E3CB26; Thu, 18 Sep 2003 08:38:26 -0700 (PDT) To: Michael Elizabeth Chastain Cc: gdb-patches@sources.redhat.com Subject: Re: [testsuite] add gdb.cp/gdb1355.exp References: <200309180201.h8I21hCr013787@duracef.shout.net> From: David Carlton Date: Thu, 18 Sep 2003 15:38:00 -0000 In-Reply-To: <200309180201.h8I21hCr013787@duracef.shout.net> (Michael Elizabeth Chastain's message of "Wed, 17 Sep 2003 22:01:43 -0400") Message-ID: User-Agent: Gnus/5.1002 (Gnus v5.10.2) XEmacs/21.4 (Rational FORTRAN, linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-SW-Source: 2003-09/txt/msg00396.txt.bz2 On Wed, 17 Sep 2003 22:01:43 -0400, Michael Elizabeth Chastain said: > As far as making it a FAIL goes, I think David's argument comes down > to: people pay attention to FAIL, but not to XFAIL or KFAIL. More or less, yes. Let me rephrase this in a different way (and I'll discuss the issue in terms of KFAIL, but the same goes for XFAIL). We have three possible states for a bug: 1) The bug is completely undiagnosed. 2) The bug is diagnosed but known not to be fixed. 3) The bug was diagnosed, thought to be fixed, but not actually fixed. We have three states here, and only two bug labels (and I will scream if anybody tries to add "ONCEFAILED" to dejagnu). So, no matter what, we're going to lose information. In that regard, I don't think this statement of yours is the correct way to phrase the question: > I see the question as: whether to fight the situation and use more > accurate XFAIL/KFAIL rather than generic FAIL, or to acquiesce to > situation and give people what they expect to see. In your proposal, KFAIL is less precise (since it encompasses cases 2 and 3) while FAIL is more precise (it encompasses only case 1); in my proposal, FAIL is less precise (cases 1,3) while KFAIL is more precise (only case 2). One question, then, is: which state do we want to be more precise? I would answer KFAIL. That's the static way of looking at the situation; another is a dynamic way, namely how will the output of the tests change between runs. In both of our proposals, if a regression pops up, we'll see a state transition: either PASS=>FAIL or PASS=>KFAIL. But my proposal also guarantees that there will always be a state transition after a bug is (thought to be) fixed: either KFAIL=>PASS or KFAIL=>FAIL. In your proposal, the latter scenario won't lead to a state transition: it will remain KFAIL. I guess that's enough babbling from me; I'll try to be quiet for a little while now. :-) David Carlton carlton@kealia.com