Re: exit status of 'make check'

[Daniel Jacobowitz <drow@mvista.com>]

On Sun, Nov 25, 2001 at 12:37:42PM -0500, George France wrote:
> > That pretty much means that it did.  You can grep for ^FAIL: to see how
> > many there are.  I'm hoping to see the GDB testsuite pass entirely for
> > some target, someday...
> 
> Greetings,
> 
> Even if all the testsuites have a FAILed status, 'make check' should not 
> return a non-zero exit code unless there is a failure in the 'make check' 
> programs. The programs that comprise 'make check' should succeed.  
> 
> For example, if the "gdb.base/annota1.exp: breakpoint info" testsuite fails, 
> it is a failure of that testsuite, not 'make check'.  Not finding 'runtest' 
> would be an example of a failure in 'make check', that should return a 
> non-zero exit code.
> 
> Currently 'make check' always returns a non-zero exit code indicating the 
> testsuite results are unreliable due to a failure in 'make check'. 
> 
> I hope this makes things clearer.

I really can't agree.  I don't care (in an idealized world in which
we've finished fixing the GDB testsuite, which we're working on...)
what went wrong.  I'd consider make check to have failed if a test
failed - for use in, say, unattended builds.

From runtest's man page:

EXIT CODES
       runtest sets the exit code to 1 if any of the tests failed, or
       sets it to 0 if all the
       tests passed.

DejaGNU seems to agree with me on this one.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer