From: Andrew Burgess <andrew.burgess@embecosm.com>
To: gdb-patches@sourceware.org
Subject: [PATCHv3 3/3] contrib: Handle GDB specific test result types
Date: Thu, 30 Apr 2020 12:20:11 +0100 [thread overview]
Message-ID: <e3ae04c67f73ec87f8ff4ef665d7c777fa8e7082.1588245462.git.andrew.burgess@embecosm.com> (raw)
In-Reply-To: <cover.1588245462.git.andrew.burgess@embecosm.com>
NOTE: I know that this patch needs to go through the GCC list and then
be back-ported to the binutils-gdb repository. I'm posting it here
for completeness while the other patches in this series are
discussed. If the general idea is approved for GDB then I'll get this
merged through the GCC route.
When running Dejagnu on GDB we can now (sometimes) see two additional
test result types, these are '# of paths in test names' and '# of
duplicate test names'.
If the test is run in parallel mode (make -j...) then these extra test
results will appear in the individual test summary files, but are not
merged into the final summary file.
Additionally, two new types of test summary line can be printed, these
are 'PATH: ...' and 'DUPLICATE: ....' these lines allow users to
quickly search the test summary to track down where the offending test
names are. These lines are similarly not merged into the unified
gdb.sum file after a parallel test run.
This commit adds support to the merge scripts to carry over these
extra result types, and the corresponding test status lines.
contrib/ChangeLog:
* dg-extract-results.py: Handle GDB specific test types.
* dg-extract-results.sh: Likewise.
---
contrib/ChangeLog | 5 +++++
| 6 ++++--
| 12 +++++++++++-
3 files changed, 20 insertions(+), 3 deletions(-)
--git a/contrib/dg-extract-results.py b/contrib/dg-extract-results.py
index 7100794d42a..30aa68771d4 100644
--- a/contrib/dg-extract-results.py
+++ b/contrib/dg-extract-results.py
@@ -117,7 +117,7 @@ class Prog:
self.tool_re = re.compile (r'^\t\t=== (.*) tests ===$')
self.result_re = re.compile (r'^(PASS|XPASS|FAIL|XFAIL|UNRESOLVED'
r'|WARNING|ERROR|UNSUPPORTED|UNTESTED'
- r'|KFAIL|KPASS):\s*(.+)')
+ r'|KFAIL|KPASS|PATH|DUPLICATE):\s*(.+)')
self.completed_re = re.compile (r'.* completed at (.*)')
# Pieces of text to write at the head of the output.
# start_line is a pair in which the first element is a datetime
@@ -143,7 +143,9 @@ class Prog:
'# of known failures\t\t',
'# of untested testcases\t\t',
'# of unresolved testcases\t',
- '# of unsupported tests\t\t'
+ '# of unsupported tests\t\t',
+ '# of paths in test names\t',
+ '# of duplicate test names\t'
]
self.runs = dict()
--git a/contrib/dg-extract-results.sh b/contrib/dg-extract-results.sh
index f948088370e..ff6c50d029c 100755
--- a/contrib/dg-extract-results.sh
+++ b/contrib/dg-extract-results.sh
@@ -326,7 +326,7 @@ BEGIN {
}
}
/^\t\t=== .* ===$/ { curvar = ""; next }
-/^(PASS|XPASS|FAIL|XFAIL|UNRESOLVED|WARNING|ERROR|UNSUPPORTED|UNTESTED|KFAIL|KPASS):/ {
+/^(PASS|XPASS|FAIL|XFAIL|UNRESOLVED|WARNING|ERROR|UNSUPPORTED|UNTESTED|KFAIL|KPASS|PATH|DUPLICATE):/ {
testname=\$2
# Ugly hack for gfortran.dg/dg.exp
if ("$TOOL" == "gfortran" && testname ~ /^gfortran.dg\/g77\//)
@@ -400,6 +400,7 @@ BEGIN {
variant="$VAR"
tool="$TOOL"
passcnt=0; failcnt=0; untstcnt=0; xpasscnt=0; xfailcnt=0; kpasscnt=0; kfailcnt=0; unsupcnt=0; unrescnt=0; dgerrorcnt=0;
+ pathcnt=0; dupcnt=0
curvar=""; insummary=0
}
/^Running target / { curvar = \$3; next }
@@ -414,6 +415,8 @@ BEGIN {
/^# of untested testcases/ { if (insummary == 1) untstcnt += \$5; next; }
/^# of unresolved testcases/ { if (insummary == 1) unrescnt += \$5; next; }
/^# of unsupported tests/ { if (insummary == 1) unsupcnt += \$5; next; }
+/^# of paths in test names/ { if (insummary == 1) pathcnt += \$7; next; }
+/^# of duplicate test names/ { if (insummary == 1) dupcnt += \$6; next; }
/^$/ { if (insummary == 1)
{ insummary = 0; curvar = "" }
next
@@ -431,6 +434,8 @@ END {
if (untstcnt != 0) printf ("# of untested testcases\t\t%d\n", untstcnt)
if (unrescnt != 0) printf ("# of unresolved testcases\t%d\n", unrescnt)
if (unsupcnt != 0) printf ("# of unsupported tests\t\t%d\n", unsupcnt)
+ if (pathcnt != 0) printf ("# of paths in test names\t%d\n", pathcnt)
+ if (dupcnt != 0) printf ("# of duplicate test names\t%d\n", dupcnt)
}
EOF
@@ -452,6 +457,7 @@ cat << EOF > $TOTAL_AWK
BEGIN {
tool="$TOOL"
passcnt=0; failcnt=0; untstcnt=0; xpasscnt=0; xfailcnt=0; kfailcnt=0; unsupcnt=0; unrescnt=0; dgerrorcnt=0
+ pathcnt=0; dupcnt=0
}
/^# of DejaGnu errors/ { dgerrorcnt += \$5 }
/^# of expected passes/ { passcnt += \$5 }
@@ -463,6 +469,8 @@ BEGIN {
/^# of untested testcases/ { untstcnt += \$5 }
/^# of unresolved testcases/ { unrescnt += \$5 }
/^# of unsupported tests/ { unsupcnt += \$5 }
+/^# of paths in test names/ { pathcnt += \$7 }
+/^# of duplicate test names/ { dupcnt += \$6 }
END {
printf ("\n\t\t=== %s Summary ===\n\n", tool)
if (dgerrorcnt != 0) printf ("# of DejaGnu errors\t\t%d\n", dgerrorcnt)
@@ -475,6 +483,8 @@ END {
if (untstcnt != 0) printf ("# of untested testcases\t\t%d\n", untstcnt)
if (unrescnt != 0) printf ("# of unresolved testcases\t%d\n", unrescnt)
if (unsupcnt != 0) printf ("# of unsupported tests\t\t%d\n", unsupcnt)
+ if (pathcnt != 0) printf ("# of paths in test names\t%d\n", pathcnt)
+ if (dupcnt != 0) printf ("# of duplicate test names\t%d\n", dupcnt)
}
EOF
--
2.25.3
next prev parent reply other threads:[~2020-04-30 11:20 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-23 17:53 [PATCH 0/4] Automatic detection of test name problems Andrew Burgess
2020-04-23 17:53 ` [PATCH 1/4] gdb/testsuite: Remove build paths from test names Andrew Burgess
2020-04-24 14:00 ` Simon Marchi
2020-04-23 17:53 ` [PATCH 2/4] gdb/testsuite: Detect and warn if paths are used in " Andrew Burgess
2020-04-23 20:26 ` Keith Seitz
2020-04-27 15:58 ` Andrew Burgess
2020-04-27 16:42 ` Keith Seitz
2020-04-27 19:06 ` Andrew Burgess
2020-04-23 17:53 ` [PATCH 3/4] gdb/testsuite: Detect and warn about duplicate " Andrew Burgess
2020-04-23 20:28 ` Keith Seitz
2020-04-23 17:53 ` [PATCH 4/4] contrib: Handle GDB specific test result types Andrew Burgess
2020-04-23 20:25 ` [PATCH 0/4] Automatic detection of test name problems Keith Seitz
2020-04-27 22:01 ` [PATCHv2 0/3] " Andrew Burgess
2020-04-27 22:01 ` [PATCHv2 1/3] gdb/testsuite: Detect and warn if paths are used in test names Andrew Burgess
2020-04-27 22:01 ` [PATCHv2 2/3] gdb/testsuite: Detect and warn about duplicate " Andrew Burgess
2020-04-27 22:01 ` [PATCHv2 3/3] contrib: Handle GDB specific test result types Andrew Burgess
2020-04-28 19:08 ` [PATCHv2 0/3] Automatic detection of test name problems Keith Seitz
2020-04-29 9:02 ` Andrew Burgess
2020-04-29 15:04 ` Simon Marchi
2020-04-29 15:38 ` Andrew Burgess
2020-04-29 16:03 ` Keith Seitz
2020-04-29 18:22 ` Simon Marchi
2020-04-30 11:20 ` [PATCHv3 " Andrew Burgess
2020-04-30 11:20 ` [PATCHv3 1/3] gdb/testsuite: Detect and warn if paths are used in test names Andrew Burgess
2020-04-30 11:20 ` [PATCHv3 2/3] gdb/testsuite: Detect and warn about duplicate " Andrew Burgess
2020-07-31 21:34 ` Simon Marchi
2020-08-03 10:02 ` Andrew Burgess
2020-08-03 12:18 ` Simon Marchi
2020-04-30 11:20 ` Andrew Burgess [this message]
2020-04-30 18:01 ` [PATCHv3 0/3] Automatic detection of test name problems Tom Tromey
2020-05-11 21:30 ` Andrew Burgess
2020-05-12 16:48 ` Andrew Burgess
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e3ae04c67f73ec87f8ff4ef665d7c777fa8e7082.1588245462.git.andrew.burgess@embecosm.com \
--to=andrew.burgess@embecosm.com \
--cc=gdb-patches@sourceware.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox