[PATCH] testsuite: Make check-parallel return non-zero if a test failed

[PATCH] testsuite: Make check-parallel return non-zero if a test failed

[Simon Marchi]

When using the check-parallel target, the return code of make is always 0,
regardless of test results.  This patch makes it return the same code as
the "make do-check-parallel" sub-command.  So if there is a FAIL somewhere,
non-zero will be returned by make.

For the sake of example, I introduced a failure in gdb.base/break.exp.

  $ make check-single TESTS="gdb.base/break.exp gdb.python/py-value.exp" && echo 'Success :D' || echo 'Fail :('
  ...
  FAIL: gdb.base/break.exp: allo
  ...
  Fail :(

I think the parallel run should do the same.  Currently:

  $ make check-parallel TESTS="gdb.base/break.exp gdb.python/py-value.exp" && echo 'Success :D' || echo 'Fail :('
  ...
  FAIL: gdb.base/break.exp: allo
  ...
  Success :D

And with the patch (no big surprises there):

  $ make check-parallel TESTS="gdb.base/break.exp gdb.python/py-value.exp" && echo 'Success :D' || echo 'Fail :('
  ...
  FAIL: gdb.base/break.exp: allo
  ...
  Fail :(

What do you think?

gdb/testsuite/ChangeLog:

	* Makefile.in (check-parallel): Propagate return code from make
	do-check-parallel.
---
 gdb/testsuite/Makefile.in | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/gdb/testsuite/Makefile.in b/gdb/testsuite/Makefile.in
index ae7fa7b..50edf8a 100644
--- a/gdb/testsuite/Makefile.in
+++ b/gdb/testsuite/Makefile.in
@@ -198,11 +198,13 @@ check-single:
 check-parallel:
 	-rm -rf cache outputs temp
 	$(MAKE) -k do-check-parallel; \
+	result=$$?; \
 	$(SHELL) $(srcdir)/dg-extract-results.sh \
 	  `find outputs -name gdb.sum -print` > gdb.sum; \
 	$(SHELL) $(srcdir)/dg-extract-results.sh -L \
-	  `find outputs -name gdb.log -print` > gdb.log
-	@sed -n '/=== gdb Summary ===/,$$ p' gdb.sum
+	  `find outputs -name gdb.log -print` > gdb.log; \
+	sed -n '/=== gdb Summary ===/,$$ p' gdb.sum; \
+	exit $$result
 
 # Turn a list of .exp files into "check/" targets.  Only examine .exp
 # files appearing in a gdb.* directory -- we don't want to pick up
-- 
2.5.1

Re: [PATCH] testsuite: Make check-parallel return non-zero if a test failed

[Pedro Alves]

On 01/18/2016 10:48 PM, Simon Marchi wrote:
> When using the check-parallel target, the return code of make is always 0,
> regardless of test results.  This patch makes it return the same code as
> the "make do-check-parallel" sub-command.  So if there is a FAIL somewhere,
> non-zero will be returned by make.
> 
> For the sake of example, I introduced a failure in gdb.base/break.exp.
> 
>   $ make check-single TESTS="gdb.base/break.exp gdb.python/py-value.exp" && echo 'Success :D' || echo 'Fail :('
>   ...
>   FAIL: gdb.base/break.exp: allo
>   ...
>   Fail :(
> 
> I think the parallel run should do the same.  Currently:
> 
>   $ make check-parallel TESTS="gdb.base/break.exp gdb.python/py-value.exp" && echo 'Success :D' || echo 'Fail :('
>   ...
>   FAIL: gdb.base/break.exp: allo
>   ...
>   Success :D
> 
> And with the patch (no big surprises there):
> 
>   $ make check-parallel TESTS="gdb.base/break.exp gdb.python/py-value.exp" && echo 'Success :D' || echo 'Fail :('
>   ...
>   FAIL: gdb.base/break.exp: allo
>   ...
>   Fail :(
> 
> What do you think?

This has occasionally annoyed me too.  Thanks for fixing this!

LGTM.

Thanks,
Pedro Alves

Re: [PATCH] testsuite: Make check-parallel return non-zero if a test failed

[Simon Marchi]

On 16-01-19 06:15 AM, Pedro Alves wrote:
> This has occasionally annoyed me too.  Thanks for fixing this!

Thanks, pushed.