From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 4228 invoked by alias); 20 Feb 2002 18:07:52 -0000 Mailing-List: contact gdb-patches-help@sources.redhat.com; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sources.redhat.com Received: (qmail 4141 invoked from network); 20 Feb 2002 18:07:46 -0000 Received: from unknown (HELO fw-cam.cambridge.arm.com) (193.131.176.3) by sources.redhat.com with SMTP; 20 Feb 2002 18:07:46 -0000 Received: by fw-cam.cambridge.arm.com; id SAA05651; Wed, 20 Feb 2002 18:07:44 GMT Received: from unknown(172.16.1.2) by fw-cam.cambridge.arm.com via smap (V5.5) id xma005219; Wed, 20 Feb 02 18:06:55 GMT Received: from cam-mail2.cambridge.arm.com (localhost [127.0.0.1]) by cam-admin0.cambridge.arm.com (8.9.3/8.9.3) with ESMTP id SAA21403 for ; Wed, 20 Feb 2002 18:06:54 GMT Received: from sun18.cambridge.arm.com (sun18.cambridge.arm.com [172.16.2.18]) by cam-mail2.cambridge.arm.com (8.9.3/8.9.3) with ESMTP id SAA16821; Wed, 20 Feb 2002 18:06:52 GMT Message-Id: <200202201806.SAA16821@cam-mail2.cambridge.arm.com> X-Mailer: exmh version 2.0.2 2/24/98 To: gdb-patches@sources.redhat.com Cc: Richard.Earnshaw@arm.com Reply-To: Richard.Earnshaw@arm.com Organization: ARM Ltd. X-Telephone: +44 1223 400569 (direct+voicemail), +44 1223 400400 (switchbd) X-Fax: +44 1223 400410 X-Address: ARM Ltd., 110 Fulbourn Road, Cherry Hinton, Cambridge CB1 9NJ. X-Url: http://www.arm.com/ Subject: Re: [rfa:testsuite} Overhaul sizeof.exp In-reply-to: Your message of "Wed, 20 Feb 2002 11:49:47 CST." <200202201749.g1KHnlv04009@duracef.shout.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 20 Feb 2002 10:07:00 -0000 From: Richard Earnshaw X-SW-Source: 2002-02/txt/msg00554.txt.bz2 mec@shout.net said: > It sounds like we understand the alternatives and everyone's got > opinions about them. > [0] Status quo > [1] FAIL the test > [2] XFAIL the test > [3] KFAIL the test Well, speaking personally, I don't particularly like XFAIL because it hides failures from me. Since I rarely see a run with zero FAILs the only thing that is interesting is a diff between this and a previous run, to tell me if an existing test has started failing, or whether a new test has been added. About the only reasonable use of XFAIL that I can think of is when we fail because there was a bug in an external tool (such as the compiler) which means we are unable to successfully run the test (but haven't failed so badly as to get an UNRESOLVED -- we know the outcome we know its wrong, but we know its because of an external problem). Everything else should be a FAIL, because its our fault and we *should* fix it. R.