* [rfa/testsuite/mi] Recognize a few incorrect outputs
@ 2001-09-28 12:15 Daniel Jacobowitz
[not found] ` <3BB4DEBB.1020906@cygnus.com>
0 siblings, 1 reply; 6+ messages in thread
From: Daniel Jacobowitz @ 2001-09-28 12:15 UTC (permalink / raw)
To: gdb-patches
The last two tests which timed out instead of failing on my system were
mi-var-display.exp and mi0-var-display.exp. The regular expressions were
overly precise; the test would pass if the compiler's debug info marked the
first line of incr_a as being the assignment, but failed (and timed out) if
the first line was considered the opening brace. I've made the latter case
a FAIL for now, since it means we stop before we can validly get a
backtrace. After the attached patch, GCC 3.0 passes the test and GCC 2.95.3
fails it without a timeout (both i386-linux).
For the curious, before this and the two other patches I've submitted in the
past 24 hours, a testsuite run took 40:42.40. Now it takes 5:46.87. I like
this much better. It could be cut down somewhat more, I expect:
91.81s user 22.74s system 33% cpu
but removing sleeps is somewhat risky and we have no good way for sub-second
sleeps; and they're probably inappropriate on remote targets anyway.
<gripe>
A lot of GDB tests seem to be written with only pass and timeout
alternatives, or only with overly-specialized fails. The hypocrite-alert
readers of this message will note that I'm guilty of the same thing; this is
Just Enough to make them catch a few errors I could think of, not enough to
recognize completely wrong output. Someday, someone more motivated than I
should clean this up.
</gripe>
Is this OK to commit?
--
Daniel Jacobowitz Carnegie Mellon University
MontaVista Software Debian GNU/Linux Developer
2001-09-28 Daniel Jacobowitz <drow@mvista.com>
* gdb.mi/mi-var-display.exp (continue to incr_a): Recognize
some incorrect output instead of timing out.
* gdb.mi/mi-var-display.exp (continue to incr_a): Likewise.
Index: gdb.mi/mi-var-display.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi-var-display.exp,v
retrieving revision 1.5
diff -u -r1.5 mi-var-display.exp
--- mi-var-display.exp 2001/08/19 01:23:43 1.5
+++ mi-var-display.exp 2001/09/28 19:03:41
@@ -591,6 +591,10 @@
-re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\\\[\{name=\"a\",value=\"2\.*\"\}\\\],file=\".*var-cmd.c\",line=\"85\"\}\r\n$mi_gdb_prompt$" {
pass "continue to incr_a"
}
+ -re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\\\[\{name=\"a\",value=\"\.*\"\}\\\],file=\".*var-cmd.c\",line=\"8\[345\]\"\}\r\n$mi_gdb_prompt$" {
+ # Compiler debug info is wrong...
+ fail "continue to incr_a"
+ }
timeout {
fail "continue to incr_a (timeout)"
}
Index: gdb.mi/mi0-var-display.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi0-var-display.exp,v
retrieving revision 1.4
diff -u -r1.4 mi0-var-display.exp
--- mi0-var-display.exp 2001/08/19 01:23:43 1.4
+++ mi0-var-display.exp 2001/09/28 19:03:42
@@ -591,6 +591,10 @@
-re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\{\{name=\"a\",value=\"2\.*\"\}\},file=\".*var-cmd.c\",line=\"85\"\}\r\n$mi_gdb_prompt$" {
pass "continue to incr_a"
}
+ -re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\{\{name=\"a\",value=\".*\"\}\},file=\".*var-cmd.c\",line=\"8\[345\]\"\}\r\n$mi_gdb_prompt$" {
+ # Compiler debug info is wrong...
+ fail "continue to incr_a"
+ }
timeout {
fail "continue to incr_a (timeout)"
}
^ permalink raw reply [flat|nested] 6+ messages in thread[parent not found: <3BB4DEBB.1020906@cygnus.com>]
* Re: [rfa/testsuite/mi] Recognize a few incorrect outputs [not found] ` <3BB4DEBB.1020906@cygnus.com> @ 2001-09-28 13:38 ` Daniel Jacobowitz 2001-09-28 15:54 ` Daniel Jacobowitz 1 sibling, 0 replies; 6+ messages in thread From: Daniel Jacobowitz @ 2001-09-28 13:38 UTC (permalink / raw) To: Andrew Cagney; +Cc: fnasser, gdb-patches On Fri, Sep 28, 2001 at 04:34:03PM -0400, Andrew Cagney wrote: > > <gripe> > > A lot of GDB tests seem to be written with only pass and timeout > > alternatives, or only with overly-specialized fails. The hypocrite-alert > > readers of this message will note that I'm guilty of the same thing; this is > > Just Enough to make them catch a few errors I could think of, not enough to > > recognize completely wrong output. Someday, someone more motivated than I > > should clean this up. > > </gripe> > > For the MI, this is a pretty obvious fix. I've been doing the same > thing my self (when I noticed it). One suggestion, can you make that > fail expression less strict so that it picks. Something like: > > <correct-output> (gdb) <more-output> (gdb) > pass > > .* (gdb) .* (gdb) > fail > > timeout > fail > > alternatively (hmm, better?), keep the expression as you have it but add > a comment in paren vis: > > fail "continue to incr_a (compiler bug info is wrong)" Can we do that? I was under the impression that "(timeout)" was special. If we can, I'll add both cases - for the wrong debug info and for some unknown failing output. I'm tempted to mark such tests XFAIL if we can obviously detect that the compiler is at fault; admittedly, the test isn't perfect, but it's still better than the current state of affairs... -- Daniel Jacobowitz Carnegie Mellon University MontaVista Software Debian GNU/Linux Developer ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [rfa/testsuite/mi] Recognize a few incorrect outputs [not found] ` <3BB4DEBB.1020906@cygnus.com> 2001-09-28 13:38 ` Daniel Jacobowitz @ 2001-09-28 15:54 ` Daniel Jacobowitz 2001-09-28 16:29 ` Andrew Cagney 2001-10-01 7:28 ` Fernando Nasser 1 sibling, 2 replies; 6+ messages in thread From: Daniel Jacobowitz @ 2001-09-28 15:54 UTC (permalink / raw) To: Andrew Cagney; +Cc: fnasser, gdb-patches On Fri, Sep 28, 2001 at 04:34:03PM -0400, Andrew Cagney wrote: > > <gripe> > > A lot of GDB tests seem to be written with only pass and timeout > > alternatives, or only with overly-specialized fails. The hypocrite-alert > > readers of this message will note that I'm guilty of the same thing; this is > > Just Enough to make them catch a few errors I could think of, not enough to > > recognize completely wrong output. Someday, someone more motivated than I > > should clean this up. > > </gripe> > > For the MI, this is a pretty obvious fix. I've been doing the same > thing my self (when I noticed it). One suggestion, can you make that > fail expression less strict so that it picks. Something like: > > <correct-output> (gdb) <more-output> (gdb) > pass > > .* (gdb) .* (gdb) > fail > > timeout > fail > > alternatively (hmm, better?), keep the expression as you have it but add > a comment in paren vis: > > fail "continue to incr_a (compiler bug info is wrong)" Is this about what you had in mind? -- Daniel Jacobowitz Carnegie Mellon University MontaVista Software Debian GNU/Linux Developer 2001-09-28 Daniel Jacobowitz <drow@mvista.com> * gdb.mi/mi-var-display.exp (continue to incr_a): Recognize some incorrect output instead of timing out. * gdb.mi/mi-var-display.exp (continue to incr_a): Likewise. Index: gdb.mi/mi-var-display.exp =================================================================== RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi-var-display.exp,v retrieving revision 1.5 diff -u -r1.5 mi-var-display.exp --- mi-var-display.exp 2001/08/19 01:23:43 1.5 +++ mi-var-display.exp 2001/09/28 22:52:37 @@ -591,6 +591,12 @@ -re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\\\[\{name=\"a\",value=\"2\.*\"\}\\\],file=\".*var-cmd.c\",line=\"85\"\}\r\n$mi_gdb_prompt$" { pass "continue to incr_a" } + -re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\\\[\{name=\"a\",value=\"\.*\"\}\\\],file=\".*var-cmd.c\",line=\"8\[345\]\"\}\r\n$mi_gdb_prompt$" { + fail "continue to incr_a (compiler debug info incorrect)" + } + -re "\\^running\r\n${mi_gdb_prompt}.*\r\n$mi_gdb_prompt$" { + fail "continue to incr_a (unknown output)" + } timeout { fail "continue to incr_a (timeout)" } Index: gdb.mi/mi0-var-display.exp =================================================================== RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi0-var-display.exp,v retrieving revision 1.4 diff -u -r1.4 mi0-var-display.exp --- mi0-var-display.exp 2001/08/19 01:23:43 1.4 +++ mi0-var-display.exp 2001/09/28 22:52:37 @@ -591,6 +591,12 @@ -re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\{\{name=\"a\",value=\"2\.*\"\}\},file=\".*var-cmd.c\",line=\"85\"\}\r\n$mi_gdb_prompt$" { pass "continue to incr_a" } + -re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\{\{name=\"a\",value=\".*\"\}\},file=\".*var-cmd.c\",line=\"8\[345\]\"\}\r\n$mi_gdb_prompt$" { + fail "continue to incr_a (compiler debug info incorrect)" + } + -re "\\^running\r\n${mi_gdb_prompt}.*\r\n$mi_gdb_prompt$" { + fail "continue to incr_a (unknown output)" + } timeout { fail "continue to incr_a (timeout)" } ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [rfa/testsuite/mi] Recognize a few incorrect outputs 2001-09-28 15:54 ` Daniel Jacobowitz @ 2001-09-28 16:29 ` Andrew Cagney 2001-10-01 7:28 ` Fernando Nasser 1 sibling, 0 replies; 6+ messages in thread From: Andrew Cagney @ 2001-09-28 16:29 UTC (permalink / raw) To: Daniel Jacobowitz; +Cc: fnasser, gdb-patches > diff -u -r1.5 mi-var-display.exp > --- mi-var-display.exp 2001/08/19 01:23:43 1.5 > +++ mi-var-display.exp 2001/09/28 22:52:37 > @@ -591,6 +591,12 @@ > -re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\\\[\{name=\"a\",value=\"2\.*\"\}\\\],file=\".*var-cmd.c\",line=\"85\"\}\r\n$mi_gdb_prompt$" { > pass "continue to incr_a" > } > + -re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\\\[\{name=\"a\",value=\"\.*\"\}\\\],file=\".*var-cmd.c\",line=\"8\[345\]\"\}\r\n$mi_gdb_prompt$" { > + fail "continue to incr_a (compiler debug info incorrect)" > + } > + -re "\\^running\r\n${mi_gdb_prompt}.*\r\n$mi_gdb_prompt$" { > + fail "continue to incr_a (unknown output)" > + } > timeout { > fail "continue to incr_a (timeout)" > } > Yes. Andrew ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [rfa/testsuite/mi] Recognize a few incorrect outputs 2001-09-28 15:54 ` Daniel Jacobowitz 2001-09-28 16:29 ` Andrew Cagney @ 2001-10-01 7:28 ` Fernando Nasser 2001-10-01 13:30 ` Daniel Jacobowitz 1 sibling, 1 reply; 6+ messages in thread From: Fernando Nasser @ 2001-10-01 7:28 UTC (permalink / raw) To: Daniel Jacobowitz; +Cc: Andrew Cagney, fnasser, gdb-patches The patch is fine. Just to answer your previous question regarding Andrew's suggestion: The text that identify the tests must be the same on both fail and pass. However, a <reason> for the fail is allowed between parenthesis at the end of ther string. The test comparison scripts are supposed to ignore that part for test identification purposes. The most used example of the aboce is the string "(timeout)". Thanks for the patch. And thanks Andrew for the help with this one. Regards, Fernando Daniel Jacobowitz wrote: > > On Fri, Sep 28, 2001 at 04:34:03PM -0400, Andrew Cagney wrote: > > > <gripe> > > > A lot of GDB tests seem to be written with only pass and timeout > > > alternatives, or only with overly-specialized fails. The hypocrite-alert > > > readers of this message will note that I'm guilty of the same thing; this is > > > Just Enough to make them catch a few errors I could think of, not enough to > > > recognize completely wrong output. Someday, someone more motivated than I > > > should clean this up. > > > </gripe> > > > > For the MI, this is a pretty obvious fix. I've been doing the same > > thing my self (when I noticed it). One suggestion, can you make that > > fail expression less strict so that it picks. Something like: > > > > <correct-output> (gdb) <more-output> (gdb) > > pass > > > > .* (gdb) .* (gdb) > > fail > > > > timeout > > fail > > > > alternatively (hmm, better?), keep the expression as you have it but add > > a comment in paren vis: > > > > fail "continue to incr_a (compiler bug info is wrong)" > > Is this about what you had in mind? > > -- > Daniel Jacobowitz Carnegie Mellon University > MontaVista Software Debian GNU/Linux Developer > > 2001-09-28 Daniel Jacobowitz <drow@mvista.com> > > * gdb.mi/mi-var-display.exp (continue to incr_a): Recognize > some incorrect output instead of timing out. > * gdb.mi/mi-var-display.exp (continue to incr_a): Likewise. > > Index: gdb.mi/mi-var-display.exp > =================================================================== > RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi-var-display.exp,v > retrieving revision 1.5 > diff -u -r1.5 mi-var-display.exp > --- mi-var-display.exp 2001/08/19 01:23:43 1.5 > +++ mi-var-display.exp 2001/09/28 22:52:37 > @@ -591,6 +591,12 @@ > -re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\\\[\{name=\"a\",value=\"2\.*\"\}\\\],file=\".*var-cmd.c\",line=\"85\"\}\r\n$mi_gdb_prompt$" { > pass "continue to incr_a" > } > + -re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\\\[\{name=\"a\",value=\"\.*\"\}\\\],file=\".*var-cmd.c\",line=\"8\[345\]\"\}\r\n$mi_gdb_prompt$" { > + fail "continue to incr_a (compiler debug info incorrect)" > + } > + -re "\\^running\r\n${mi_gdb_prompt}.*\r\n$mi_gdb_prompt$" { > + fail "continue to incr_a (unknown output)" > + } > timeout { > fail "continue to incr_a (timeout)" > } > Index: gdb.mi/mi0-var-display.exp > =================================================================== > RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi0-var-display.exp,v > retrieving revision 1.4 > diff -u -r1.4 mi0-var-display.exp > --- mi0-var-display.exp 2001/08/19 01:23:43 1.4 > +++ mi0-var-display.exp 2001/09/28 22:52:37 > @@ -591,6 +591,12 @@ > -re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\{\{name=\"a\",value=\"2\.*\"\}\},file=\".*var-cmd.c\",line=\"85\"\}\r\n$mi_gdb_prompt$" { > pass "continue to incr_a" > } > + -re "\\^running\r\n${mi_gdb_prompt}\\*stopped,reason=\"breakpoint-hit\",bkptno=\"3\",thread-id=\"\[01\]\",frame=\{addr=\"$hex\",func=\"incr_a\",args=\{\{name=\"a\",value=\".*\"\}\},file=\".*var-cmd.c\",line=\"8\[345\]\"\}\r\n$mi_gdb_prompt$" { > + fail "continue to incr_a (compiler debug info incorrect)" > + } > + -re "\\^running\r\n${mi_gdb_prompt}.*\r\n$mi_gdb_prompt$" { > + fail "continue to incr_a (unknown output)" > + } > timeout { > fail "continue to incr_a (timeout)" > } -- Fernando Nasser Red Hat Canada Ltd. E-Mail: fnasser@redhat.com 2323 Yonge Street, Suite #300 Toronto, Ontario M4P 2C9 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [rfa/testsuite/mi] Recognize a few incorrect outputs 2001-10-01 7:28 ` Fernando Nasser @ 2001-10-01 13:30 ` Daniel Jacobowitz 0 siblings, 0 replies; 6+ messages in thread From: Daniel Jacobowitz @ 2001-10-01 13:30 UTC (permalink / raw) To: Fernando Nasser; +Cc: gdb-patches On Mon, Oct 01, 2001 at 10:23:05AM -0400, Fernando Nasser wrote: > The patch is fine. OK, committed. > Just to answer your previous question regarding Andrew's suggestion: > > The text that identify the tests must be the same on both fail and pass. > However, a <reason> for the fail is allowed between parenthesis at the > end of ther string. The test comparison scripts are supposed to ignore > that part for test identification purposes. > > The most used example of the aboce is the string "(timeout)". Thanks. That makes a lot more sense to me. -- Daniel Jacobowitz Carnegie Mellon University MontaVista Software Debian GNU/Linux Developer ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2001-10-01 13:30 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2001-09-28 12:15 [rfa/testsuite/mi] Recognize a few incorrect outputs Daniel Jacobowitz
[not found] ` <3BB4DEBB.1020906@cygnus.com>
2001-09-28 13:38 ` Daniel Jacobowitz
2001-09-28 15:54 ` Daniel Jacobowitz
2001-09-28 16:29 ` Andrew Cagney
2001-10-01 7:28 ` Fernando Nasser
2001-10-01 13:30 ` Daniel Jacobowitz
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox