Hi Tom, getting back to making gcc's mail-report.log target work... >> Rainer> * One could also copy over gcc's contrib/test_summary, used by the >> Rainer> toplevel make mail-report.log to provide a nice summary of test >> Rainer> results. However, this is currently hampered by the fact that for >> Rainer> parallel make check the gdb.sum and gdb.log files are left in >> Rainer> outputs/*/*/gdb.{sum,log} after dg-extract-results.sh has run >> Rainer> instead >> Rainer> of moving them to *.sep like gcc's gcc/Makefile.in does, so >> Rainer> mail-report.log lists every failure twice. >> >> I don't understand the "*.sep" comment - would you mind spelling it out? > > The test_summary scripts works by searching for *.sum and *.log files in > the whole tree (given that those live at different levels in the build > tree and cannot easily be found with a glob pattern). > > Currently, once dg-extract-results.sh has summarized the individual > gdb.sum and gdb.log files in outputs, we have both the individual > per-subdir files in place and the summarized one in gdb/testsuite. When > test_summary runs, it find all of of those and lists every non-PASS > result twice in its output, which isn't particularly helpful. > > To avoid this, the gcc testsuite moves the subdir .sum/.log files for > parallelized testsuites to .sum.sep/.log.sep before passing them to > dg-extract-results.sh. This way, we get one summary per testsuite > (e.g. gcc or g++, or gdb in the case at hand), and test_summary won't > pick them up twice. > >> Anyway, if this script is useful to you, it's fine with me if you want >> to find a way to make it work. I think the outputs/** stuff can be >> moved around or messed with pretty freely, though of course it is best >> not to outright lose things. > > Absolutely: as I said, the individual files are just moved aside not to > interfere with the likes of test_summary, but still left in place since > dg-extract-results.* isn't always perfect in merging them. > > I'll go ahead and prepare a patch then. Here's the mechanical part of such a patch, tested on amd64-pc-solaris2.11 and making sure that both gdb make check and make check-parallel-racy produce identical gdb.sum and racy.sum files (modulo racy tests, of course). However, looking at the resulting output, there are some issues to be decided before the result can be fully useful. I'm attaching an example output file for reference, from a Solaris 11/amd64 build with --disable-binutils --disable-gas --disable-ld --disable-sim. * Right now, the output talks of Compiler version: 8.2.50.20180803-git ... even when this is for gdb (or binutils) test results. * The resulting script would mail testsuite results to gcc-testresults@gcc.gnu.org; clearly not useful for gdb or binutils. While we could use gdb-testers@sourceware.org for gdb (where Andreas Krebbel already posts very similarly formatted results for Linux/s390x) and there's even an (apparently unused) gdb-testresults@sourceware.org list as I just happened to find at https://sourceware.org/lists.html#gdb, there seems to be no equivalent for binutils test results. I think the contrib/test_summary script should provide decent defaults depending on which tree it is run in. * There are 3 scenarios, I believe: gcc tree, already handled binutils-gdb tree, binutils results binutils-gdb tree, gdb results I believe the result in the binutils-gdb tree should be the same as what would happen in a binutils or gdb release tarball, i.e. the same results summarized in each case. This could be done using the same lists of subdirs used by src-release.sh to create the respective tarballs. In theory, there's still the combined tree case, although I have no idea if it is still common these days. My current thinking is that the toplevel mail-report.log target would create all summaries suitable for the current tree, either by passing corresponding -t [gcc|binutils|gdb] options to test_summary or letting the script figure that out on its own, creating * mail-report.log for gcc, keeping the name for backward compatibility, * binutils-report.log and/or gdb-report.log if either or both binutils/gas/ld and gdb/sim builds are present in the tree. Thoughts? Rainer -- ----------------------------------------------------------------------------- Rainer Orth, Center for Biotechnology, Bielefeld University 2018-08-03 Rainer Orth gdb/testsuite: * Makefile.in (check-parallel): Move gdb.sum, gdb.log to gdb.sum.sep, gdb.log.sep before summarizing. (check-parallel-racy): Likewise. contrib: * test_summary: New file from gcc repo.