From mboxrd@z Thu Jan 1 00:00:00 1970 From: George France To: Daniel Jacobowitz Cc: gdb@sources.redhat.com, clp@iol.unh.edu Subject: Re: exit status of 'make check' Date: Sun, 25 Nov 2001 20:59:00 -0000 Message-ID: <01112523550702.05740@shadowfax.middleearth> References: <200111212321.RAA09463@duracef.shout.net> <01112513400901.05740@shadowfax.middleearth> <20011125135448.A16810@nevyn.them.org> X-SW-Source: 2001-11/msg00249.html Message-ID: <20011125205900.sdW4j1JZWyiERtvFR_9dBjo6s7MeOjQLm3RAfNfx1f8@z> On Sunday 25 November 2001 13:54, Daniel Jacobowitz wrote: > On Sun, Nov 25, 2001 at 01:40:09PM -0500, George France wrote: > > We are not discussing, the results of 'runtest'. We are discussing the > > results of 'make check'. Here is another simple example, On 2001-11-22, > > I built the gcc_3.0.2 HEAD branch. 'make check' returned a zero for an > > exit code, but many of the testsuites FAILed. > > GCC does this by completely ignoring the exit code of runtest; in other > words, make check will always succeed. I don't think that's an > acceptable step. Do you? > I looked at the Makefiles (gdb & gcc). You are correct. It is equally unacceptable to always succeed. > We are discussing the result of runtest, because the DejaGNU > maintainers are the appropriate people to set policy for this sort of > thing. I am disappointed that there is not a more consistent GNU policy in regards to the meaning of exit code for 'make check'. For the auto-build system that I use, it makes much more sense to be able to separate errors from 'make check' into two categories: A) errors in the check system - Example, 'runtest' not being available. B) testsuite results either PASS / FAIL - Which are the result of a successful run of the check system. I can see that this is not possible under the current policy. I will have to research this matter. Any further suggestions would be greatly appreciated. Thank you for your help. Best Regards, --George