Re: exit status of 'make check'

Daniel Jacobowitz
2001-11-25T17:47:26Z

On Sun, Nov 25, 2001 at 12:37:42PM -0500, George France wrote:
> > That pretty much means that it did. You can grep for ^FAIL: to see how
> > many there are. I'm hoping to see the GDB testsuite pass entirely for
> > some target, someday...
>
> Greetings,
>
> Even if all the testsuites have a FAILed status, 'make check' should not
> return a non-zero exit code unless there is a failure in the 'make check'
> programs. The programs that comprise 'make check' should succeed.
>
> For example, if the "gdb.base/annota1.exp: breakpoint info" testsuite fails,
> it is a failure of that testsuite, not 'make check'. Not finding 'runtest'
> would be an example of a failure in 'make check', that should return a
> non-zero exit code.
>
> Currently 'make check' always returns a non-zero exit code indicating the
> testsuite results are unreliable due to a failure in 'make check'.
>
> I hope this makes things clearer.

I really can't agree. I don't care (in an idealized world in which
we've finished fixing the GDB testsuite, which we're working on...)
what went wrong. I'd consider make check to have failed if a test
failed - for use in, say, unattended builds.

From runtest's man page:

EXIT CODES
runtest sets the exit code to 1 if any of the tests failed, or
sets it to 0 if all the
tests passed.

DejaGNU seems to agree with me on this one.

--
Daniel Jacobowitz Carnegie Mellon University
MontaVista Software Debian GNU/Linux Developer