[RFC] Framework for selectively omitting individual testsuite tests or directories for local use only

[Fred Fish <fnf@ninemoons.com>]

Very often when I'm working with a gdb that has very poor testsuite
results, I want to set things up so that I can selectively omit
individual troublesome tests or even entire directories of tests.

That way the I can get a completely clean gdb testsuite run, and then
start tackling the troublesome tests one a time, without having to
filter out the hundreds of other failures I haven't yet gotten around
to fixing.  Also, a clean testsuite runs *MUCH* faster than one that
has hundreds of timeouts or other such issues.  For example, the
current sh-elf gdb seems to hang forever if you don't avoid running
the gdbtk tests.

Dejagnu supports "ignoretests" and "ignoredirs" lists of specific
individual tests or directories to ignore.  Below is a patch that I
frequently apply locally to any gdb tree I'm working on.

I'd like to propose that we add it, or something like it, to the
testsuite strictly for use as a debugging aid.  It's not intended to
become a permanent way for various configurations to hide their more
problematic test cases, just a framework for developers to use
locally.

Comments?

-Fred


Index: Makefile.in
===================================================================
RCS file: /cvs/cvsfiles/devo/gdb/testsuite/Makefile.in,v
retrieving revision 1.66
diff -c -p -r1.66 Makefile.in
*** Makefile.in	9 Sep 2003 21:03:53 -0000	1.66
--- Makefile.in	2 Feb 2004 23:57:50 -0000
*************** host_canonical = @host@
*** 31,36 ****
--- 31,39 ----
  target_canonical = @target@
  target_cpu = @gdb_target_cpu@
  
+ ignoredirs = @ignoredirs@
+ ignoretests = @ignoretests@
+ 
  SHELL = @SHELL@
  EXEEXT = @EXEEXT@
  SUBDIRS = @subdirs@
*************** site.exp: ./config.status Makefile
*** 111,116 ****
--- 114,121 ----
  	@echo "set build_triplet ${build_canonical}" >> ./tmp0
  	@echo "set srcdir ${srcdir}" >> ./tmp0
  	@echo "set tool gdb" >> ./tmp0
+ 	@echo "lappend ignoredirs ${ignoredirs}" >>./tmp0
+ 	@echo "lappend ignoretests ${ignoretests}" >>./tmp0
  	@echo "## All variables above are generated by configure. Do Not Edit ##" >> ./tmp0
  		@cat ./tmp0 > site.exp
  	@cat site.bak | sed \
Index: configure.in
===================================================================
RCS file: /cvs/cvsfiles/devo/gdb/testsuite/configure.in,v
retrieving revision 1.59.6.1
diff -c -p -r1.59.6.1 configure.in
*** configure.in	22 Oct 2003 00:34:16 -0000	1.59.6.1
--- configure.in	2 Feb 2004 23:57:50 -0000
*************** esac
*** 77,84 ****
--- 77,102 ----
  # Add gdbtk tests when appropriate.
  if test $enable_gdbtk = yes; then
    configdirs="$configdirs gdb.gdbtk"
+ else
+   ignoredirs="${ignoredirs} gdb.gdbtk"
  fi
  
+ # Suppress some individual tests or groups of tests for specific targets.
+ 
+ ignoredirs=""
+ case "${target}" in
+  xxx-yyy-zzz)
+ 	ignoredirs="${ignoredirs} gdb.cp"
+ 	;;
+ esac
+ 
+ ignoretests=""
+ case "${target}" in
+  xxx-yyy-zzz)
+ 	ignoretests="${ignoretests} fileio.exp"
+ 	;;
+ esac
+ 
  # Enable shared libraries.
  AC_ARG_ENABLE(shared,
  [  --enable-shared         build shared libraries [deault=yes]],,
*************** AC_CHECK_HEADERS(pthread.h)
*** 112,117 ****
--- 130,137 ----
  AC_EXEEXT
  
  AC_CONFIG_SUBDIRS($configdirs)
+ AC_SUBST(ignoredirs)
+ AC_SUBST(ignoretests)
  AC_OUTPUT([Makefile gdb.arch/Makefile gdb.asm/Makefile gdb.base/Makefile \
    gdb.cp/Makefile gdb.disasm/Makefile gdb.java/Makefile gdb.mi/Makefile \
    gdb.objc/Makefile gdb.threads/Makefile gdb.trace/Makefile])