Re: [RFA] Add more parallelization to gdb.base testing

Re: [RFA] Add more parallelization to gdb.base testing

[Doug Evans]

I wrote:
> This patch speeds up gdb testing at -j6 on my machine from 7mins to 4mins.
> I didn't want to get too carried away here, this felt like a good compromise.
> I split the range into four roughly equal partitions alphabet wise,
> then noticed that [a-g] has far more files than the others so made the split
> more refined.  I'm happy to use whatever partitioning folks want, I just
> want the 4mins (or better).
> 
> Ok to check in?
> 
> btw, it's possible to do better.
> The ideal is (effectively) to be able to run all the .exp's at once.
> But that's a more work and this is trivial with a reasonable win.

Blech.  Bad cut-n-paste.  Here is the correct ChangeLog entry.

2012-06-15  Doug Evans  <dje@google.com>

	* Makefile.in (TEST_DIRS): Add gdb.base3, gdb.base4.
	(BASE1_FILES, BASE2_FILES): Change to handle a-d and e-k respectively.
	(BASE3_FILES, BASE4_FILES): New, handles l-r and s-z respectively.

[RFA] Add more parallelization to gdb.base testing

[Doug Evans]

Hi.

This patch speeds up gdb testing at -j6 on my machine from 7mins to 4mins.
I didn't want to get too carried away here, this felt like a good compromise.
I split the range into four roughly equal partitions alphabet wise,
then noticed that [a-g] has far more files than the others so made the split
more refined.  I'm happy to use whatever partitioning folks want, I just
want the 4mins (or better).

Ok to check in?

btw, it's possible to do better.
The ideal is (effectively) to be able to run all the .exp's at once.
But that's a more work and this is trivial with a reasonable win.

2012-06-15  Doug Evans  <dje@google.com>

	* Makefile.in (TEST_DIRS): Add gdb.base3, gdb.base4.
	(BASE1_FILES, BASE2_FILES): Change to handle a-e and f-m respectively.
	(BASE3_FILES, BASE4_FILES): New, handles n-s and t-z respectively.

diff --git a/gdb/testsuite/Makefile.in b/gdb/testsuite/Makefile.in
index fab124e..ca5ca55 100644
--- a/gdb/testsuite/Makefile.in
+++ b/gdb/testsuite/Makefile.in
@@ -158,7 +158,7 @@ check-single: all $(abs_builddir)/site.exp
 # A list of all directories named "gdb.*" which also hold a .exp file.
 # We filter out gdb.base and add fake entries, because that directory
 # takes the longest to process, and so we split it in half.
-TEST_DIRS = gdb.base1 gdb.base2 $(filter-out gdb.base,$(sort $(notdir $(patsubst %/,%,$(dir $(wildcard $(srcdir)/gdb.*/*.exp))))))
+TEST_DIRS = gdb.base1 gdb.base2 gdb.base3 gdb.base4 $(filter-out gdb.base,$(sort $(notdir $(patsubst %/,%,$(dir $(wildcard $(srcdir)/gdb.*/*.exp))))))
 
 TEST_TARGETS = $(addprefix check-,$(TEST_DIRS))
 
@@ -179,11 +179,15 @@ check-parallel:
 @GMAKE_TRUE@	@if test ! -d gdb.$*; then mkdir gdb.$*; fi
 @GMAKE_TRUE@	$(DO_RUNTEST) --directory=gdb.$* --outdir=gdb.$* $(RUNTESTFLAGS)
 
-# Each half (roughly) of the .exp files from gdb.base.
-BASE1_FILES = $(patsubst $(srcdir)/%,%,$(wildcard $(srcdir)/gdb.base/[a-m]*.exp))
-BASE2_FILES = $(patsubst $(srcdir)/%,%,$(wildcard $(srcdir)/gdb.base/[n-z]*.exp))
+# A simple partitioning of the .exp files from gdb.base.
+# The partitioning here is a modest attempt at having four equal sized pieces,
+# without trying to be too clever.
+BASE1_FILES = $(patsubst $(srcdir)/%,%,$(wildcard $(srcdir)/gdb.base/[a-d]*.exp))
+BASE2_FILES = $(patsubst $(srcdir)/%,%,$(wildcard $(srcdir)/gdb.base/[e-k]*.exp))
+BASE3_FILES = $(patsubst $(srcdir)/%,%,$(wildcard $(srcdir)/gdb.base/[l-r]*.exp))
+BASE4_FILES = $(patsubst $(srcdir)/%,%,$(wildcard $(srcdir)/gdb.base/[s-z]*.exp))
 
-# Handle each half of gdb.base.
+# Handle each partition of gdb.base.
 check-gdb.base%: all $(abs_builddir)/site.exp
 	@if test ! -d gdb.base$*; then mkdir gdb.base$*; fi
 	$(DO_RUNTEST) $(BASE$*_FILES) --outdir gdb.base$* $(RUNTESTFLAGS)

Re: [RFA] Add more parallelization to gdb.base testing

[Joel Brobecker]

> This patch speeds up gdb testing at -j6 on my machine from 7mins to 4mins.
> I didn't want to get too carried away here, this felt like a good compromise.

Love the idea of adding more parallelization. FWIW, I run the AdaCore
testsuite at -j24 on my 3.5 year old quad-core. I start getting
diminishing benefits after -j20, but even at -j24, the machine is
still completely responsive. When doing testing with gdbserver or Qemu,
I usually use -j16 to -j20 to avoid some spurious timeouts.  So I think
there is still room for more parallelization if we want to.

-- 
Joel

Re: [RFA] Add more parallelization to gdb.base testing

[Tom Tromey]

>>>>> "Doug" == Doug Evans <dje@google.com> writes:

Doug> This patch speeds up gdb testing at -j6 on my machine from 7mins
Doug> to 4mins.  I didn't want to get too carried away here, this felt
Doug> like a good compromise.  I split the range into four roughly equal
Doug> partitions alphabet wise, then noticed that [a-g] has far more
Doug> files than the others so made the split more refined.  I'm happy
Doug> to use whatever partitioning folks want, I just want the 4mins (or
Doug> better).

Doug> Ok to check in?

We discussed this on irc a bit, but for the record, I think there are
latent bugs that this patch will expose.

For example, several tests in gdb.base use the same executable name:

    barimba. egrep 'set testfile.* "break"$' *.exp
    completion.exp:set testfile "break"
    condbreak.exp:set testfile "break"
    define.exp:set testfile "break"
    ena-dis-br.exp:set testfile "break"
    environ.exp:set testfile "break"
    info-proc.exp:set testfile "break"
    maint.exp:set testfile "break"

Jan went through and fixed bugs like this for the current split.

Tom