[RFC] Framework for selectively omitting individual testsuite tests or directories for local use only

Fred Fish
2004-02-03T00:13:50Z

Very often when I'm working with a gdb that has very poor testsuite
results, I want to set things up so that I can selectively omit
individual troublesome tests or even entire directories of tests.

That way the I can get a completely clean gdb testsuite run, and then
start tackling the troublesome tests one a time, without having to
filter out the hundreds of other failures I haven't yet gotten around
to fixing. Also, a clean testsuite runs *MUCH* faster than one that
has hundreds of timeouts or other such issues. For example, the
current sh-elf gdb seems to hang forever if you don't avoid running
the gdbtk tests.

Dejagnu supports "ignoretests" and "ignoredirs" lists of specific
individual tests or directories to ignore. Below is a patch that I
frequently apply locally to any gdb tree I'm working on.

I'd like to propose that we add it, or something like it, to the
testsuite strictly for use as a debugging aid. It's not intended to
become a permanent way for various configurations to hide their more
problematic test cases, just a framework for developers to use
locally.

Comments?

-Fred

Index: Makefile.in
===================================================================
RCS file: /cvs/cvsfiles/devo/gdb/testsuite/Makefile.in,v
retrieving revision 1.66
diff -c -p -r1.66 Makefile.in
*** Makefile.in 9 Sep 2003 21:03:53 -0000 1.66
--- Makefile.in 2 Feb 2004 23:57:50 -0000
*************** host_canonical = @host@
*** 31,36 ****
--- 31,39 ----
target_canonical = @target@
target_cpu = @gdb_target_cpu@

+ ignoredirs = @ignoredirs@
+ ignoretests = @ignoretests@
+
SHELL = @SHELL@
EXEEXT = @EXEEXT@
SUBDIRS = @subdirs@
*************** site.exp: ./config.status Makefile
*** 111,116 ****
--- 114,121 ----
@echo "set build_triplet ${build_canonical}" >> ./tmp0
@echo "set srcdir ${srcdir}" >> ./tmp0
@echo "set tool gdb" >> ./tmp0
+ @echo "lappend ignoredirs ${ignoredirs}" >>./tmp0
+ @echo "lappend ignoretests ${ignoretests}" >>./tmp0
@echo "## All variables above are generated by configure. Do Not Edit ##" >> ./tmp0
@cat ./tmp0 > site.exp
@cat site.bak | sed \
Index: configure.in
===================================================================
RCS file: /cvs/cvsfiles/devo/gdb/testsuite/configure.in,v
retrieving revision 1.59.6.1
diff -c -p -r1.59.6.1 configure.in
*** configure.in 22 Oct 2003 00:34:16 -0000 1.59.6.1
--- configure.in 2 Feb 2004 23:57:50 -0000
*************** esac
*** 77,84 ****
--- 77,102 ----
# Add gdbtk tests when appropriate.
if test $enable_gdbtk = yes; then
configdirs="$configdirs gdb.gdbtk"
+ else
+ ignoredirs="${ignoredirs} gdb.gdbtk"
fi

+ # Suppress some individual tests or groups of tests for specific targets.
+
+ ignoredirs=""
+ case "${target}" in
+ xxx-yyy-zzz)
+ ignoredirs="${ignoredirs} gdb.cp"
+ ;;
+ esac
+
+ ignoretests=""
+ case "${target}" in
+ xxx-yyy-zzz)
+ ignoretests="${ignoretests} fileio.exp"
+ ;;
+ esac
+
# Enable shared libraries.
AC_ARG_ENABLE(shared,
[ --enable-shared build shared libraries [deault=yes]],,
*************** AC_CHECK_HEADERS(pthread.h)
*** 112,117 ****
--- 130,137 ----
AC_EXEEXT

AC_CONFIG_SUBDIRS($configdirs)
+ AC_SUBST(ignoredirs)
+ AC_SUBST(ignoretests)
AC_OUTPUT([Makefile gdb.arch/Makefile gdb.asm/Makefile gdb.base/Makefile \
gdb.cp/Makefile gdb.disasm/Makefile gdb.java/Makefile gdb.mi/Makefile \
gdb.objc/Makefile gdb.threads/Makefile gdb.trace/Makefile])