From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 17791 invoked by alias); 10 Nov 2004 16:40:05 -0000 Mailing-List: contact gdb-patches-help@sources.redhat.com; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sources.redhat.com Received: (qmail 17747 invoked from network); 10 Nov 2004 16:39:56 -0000 Received: from unknown (HELO mx1.redhat.com) (66.187.233.31) by sourceware.org with SMTP; 10 Nov 2004 16:39:56 -0000 Received: from int-mx1.corp.redhat.com (int-mx1.corp.redhat.com [172.16.52.254]) by mx1.redhat.com (8.12.11/8.12.11) with ESMTP id iAAGdoLJ018010 for ; Wed, 10 Nov 2004 11:39:56 -0500 Received: from localhost.redhat.com (to-dhcp51.toronto.redhat.com [172.16.14.151]) by int-mx1.corp.redhat.com (8.11.6/8.11.6) with ESMTP id iAAGdor18619; Wed, 10 Nov 2004 11:39:50 -0500 Received: from [127.0.0.1] (localhost.localdomain [127.0.0.1]) by localhost.redhat.com (Postfix) with ESMTP id 0EA7A129D8C; Wed, 10 Nov 2004 11:38:48 -0500 (EST) Message-ID: <41924417.8060202@gnu.org> Date: Wed, 10 Nov 2004 16:40:00 -0000 From: Andrew Cagney User-Agent: Mozilla Thunderbird 0.8 (X11/20041020) MIME-Version: 1.0 To: Hans-Peter Nilsson Cc: gdb-patches@sources.redhat.com Subject: Re: [RFA:] sim-defs.exp: support xfail References: <200411101455.iAAEtbqh014407@ignucius.se.axis.com> In-Reply-To: <200411101455.iAAEtbqh014407@ignucius.se.axis.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-SW-Source: 2004-11/txt/msg00197.txt.bz2 Hans-Peter Nilsson wrote: >>Date: Wed, 10 Nov 2004 09:01:56 -0500 >>From: Andrew Cagney > > >>Hans-Peter Nilsson wrote: >> >>>Directly taken from the same option in the ld run_dump_test. >>>Ok to commit? >>>2004-11-10 Hans-Peter Nilsson >>> >>> * lib/sim-defs.exp (run_sim_test): Support "xfail" option. >> >>What do you mean by "xfail"? (Hint, check a current dejagnu document >>where it describes kfail :-) > > > I know about kfail, but I'm agnostic. I just mean what happens > for the ld run_dump_test "xfail" option; a simple setup_xfail. > The purpose is to be able to keep failing tests around but not > get the total results marked as partially failing. Reasons for > the failure would be unspecified, matching both xfail and kfail > usage. > > To me, it doesn't really matter whether it's kfail or xfail > (vivid descriptions of kfail vs. xfail to /dev/null). In the > sim testsuite I believe it'll be used very sparingly, perhaps > not even for things that are checked in. But it'd be Nice to > Have. > > Is the change acceptable by naming the option "kfail" and > calling "setup_kfail"? Or do you want both? > > brgds, H-P > PS. BTW, the dejagnu documentation on > is not > current; KFAIL isn't mentioned there. Ok, really weird, the date on that page says 2004 but the manual is missing (from dejagnu 1.4.4): @item KFAIL A test is known to fail in some environment(s) due to a known bug in the tool being tested (identified by a bug id string). This exists so that, after a bug is identified and properly registered in a bug tracking database (Gnats, for instance), the count of failures can be kept as zero. Having zero as a baseline in all platforms allow the tool developers to immediately detect regressions caused by changes (which may affect some platforms and not others). The connection with a bug tracking database allows for automatic generation of the BUGS section of man pages or Release Notes, as well as a "Bugs Fixed this Release" section (by comparing to a previous release set of known failures). The procedure @code{setup_kfail} is used to indicate a failure is known to exist. @cindex KFAIL, avoiding for POSIX As with @code{XFAIL}, @sc{posix} tests must return @code{FAIL} rather than @code{KFAIL} even if a failure was due to a known bug. I can't think of a single reason for a simulator test that failing not being for a known bug. However, for consistency with dejagnu, we might as well support both. Feel free to add the necessary glue to cover all of xfail and kfail and kpass. Andrew