From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 10862 invoked by alias); 26 Nov 2001 04:59:27 -0000 Mailing-List: contact gdb-help@sourceware.cygnus.com; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-owner@sources.redhat.com Received: (qmail 10841 invoked from network); 26 Nov 2001 04:59:25 -0000 Received: from unknown (HELO handhelds.org) (204.123.13.90) by sourceware.cygnus.com with SMTP; 26 Nov 2001 04:59:25 -0000 Received: from shadowfax.middleearth (IDENT:root@localhost [127.0.0.1]) by handhelds.org (8.9.3/8.9.3) with SMTP id UAA08484; Sun, 25 Nov 2001 20:59:22 -0800 From: George France To: Daniel Jacobowitz Subject: Re: exit status of 'make check' Date: Tue, 13 Nov 2001 10:26:00 -0000 X-Mailer: KMail [version 1.1.99] Content-Type: text/plain; charset="us-ascii" Cc: gdb@sources.redhat.com, clp@iol.unh.edu References: <200111212321.RAA09463@duracef.shout.net> <01112513400901.05740@shadowfax.middleearth> <20011125135448.A16810@nevyn.them.org> In-Reply-To: <20011125135448.A16810@nevyn.them.org> MIME-Version: 1.0 Message-Id: <01112523550702.05740@shadowfax.middleearth> Content-Transfer-Encoding: 8bit X-SW-Source: 2001-11/txt/msg00142.txt.bz2 On Sunday 25 November 2001 13:54, Daniel Jacobowitz wrote: > On Sun, Nov 25, 2001 at 01:40:09PM -0500, George France wrote: > > We are not discussing, the results of 'runtest'. We are discussing the > > results of 'make check'. Here is another simple example, On 2001-11-22, > > I built the gcc_3.0.2 HEAD branch. 'make check' returned a zero for an > > exit code, but many of the testsuites FAILed. > > GCC does this by completely ignoring the exit code of runtest; in other > words, make check will always succeed. I don't think that's an > acceptable step. Do you? > I looked at the Makefiles (gdb & gcc). You are correct. It is equally unacceptable to always succeed. > We are discussing the result of runtest, because the DejaGNU > maintainers are the appropriate people to set policy for this sort of > thing. I am disappointed that there is not a more consistent GNU policy in regards to the meaning of exit code for 'make check'. For the auto-build system that I use, it makes much more sense to be able to separate errors from 'make check' into two categories: A) errors in the check system - Example, 'runtest' not being available. B) testsuite results either PASS / FAIL - Which are the result of a successful run of the check system. I can see that this is not possible under the current policy. I will have to research this matter. Any further suggestions would be greatly appreciated. Thank you for your help. Best Regards, --George From mboxrd@z Thu Jan 1 00:00:00 1970 From: George France To: Daniel Jacobowitz Cc: gdb@sources.redhat.com, clp@iol.unh.edu Subject: Re: exit status of 'make check' Date: Sun, 25 Nov 2001 20:59:00 -0000 Message-ID: <01112523550702.05740@shadowfax.middleearth> References: <200111212321.RAA09463@duracef.shout.net> <01112513400901.05740@shadowfax.middleearth> <20011125135448.A16810@nevyn.them.org> X-SW-Source: 2001-11/msg00249.html Message-ID: <20011125205900.sdW4j1JZWyiERtvFR_9dBjo6s7MeOjQLm3RAfNfx1f8@z> On Sunday 25 November 2001 13:54, Daniel Jacobowitz wrote: > On Sun, Nov 25, 2001 at 01:40:09PM -0500, George France wrote: > > We are not discussing, the results of 'runtest'. We are discussing the > > results of 'make check'. Here is another simple example, On 2001-11-22, > > I built the gcc_3.0.2 HEAD branch. 'make check' returned a zero for an > > exit code, but many of the testsuites FAILed. > > GCC does this by completely ignoring the exit code of runtest; in other > words, make check will always succeed. I don't think that's an > acceptable step. Do you? > I looked at the Makefiles (gdb & gcc). You are correct. It is equally unacceptable to always succeed. > We are discussing the result of runtest, because the DejaGNU > maintainers are the appropriate people to set policy for this sort of > thing. I am disappointed that there is not a more consistent GNU policy in regards to the meaning of exit code for 'make check'. For the auto-build system that I use, it makes much more sense to be able to separate errors from 'make check' into two categories: A) errors in the check system - Example, 'runtest' not being available. B) testsuite results either PASS / FAIL - Which are the result of a successful run of the check system. I can see that this is not possible under the current policy. I will have to research this matter. Any further suggestions would be greatly appreciated. Thank you for your help. Best Regards, --George