Re: exit status of 'make check'

George France
2001-11-25T20:59:00Z

On Sunday 25 November 2001 13:54, Daniel Jacobowitz wrote:
> On Sun, Nov 25, 2001 at 01:40:09PM -0500, George France wrote:
> > We are not discussing, the results of 'runtest'. We are discussing the
> > results of 'make check'. Here is another simple example, On 2001-11-22,
> > I built the gcc_3.0.2 HEAD branch. 'make check' returned a zero for an
> > exit code, but many of the testsuites FAILed.
>
> GCC does this by completely ignoring the exit code of runtest; in other
> words, make check will always succeed. I don't think that's an
> acceptable step. Do you?
>

I looked at the Makefiles (gdb & gcc). You are correct. It is equally
unacceptable to always succeed. <sigh>

> We are discussing the result of runtest, because the DejaGNU
> maintainers are the appropriate people to set policy for this sort of
> thing.

I am disappointed that there is not a more consistent GNU policy in regards
to the meaning of exit code for 'make check'. For the auto-build system that
I use, it makes much more sense to be able to separate errors from 'make
check' into two categories:

A) errors in the check system - Example, 'runtest' not being available.
B) testsuite results either PASS / FAIL - Which are the result of a
successful run of the check system.

I can see that this is not possible under the current policy. I will have to
research this matter. Any further suggestions would be greatly appreciated.

Thank you for your help.

Best Regards,

--George