[rfa/testsuite] make annota1 regexps more generous

[rfa/testsuite] make annota1 regexps more generous

[David Carlton]

I noticed today that annota1.exp seems to be generating some spurious
FAILs on my machine, namely

FAIL: gdb.base/annota1.exp: breakpoint info
FAIL: gdb.base/annota1.exp: backtrace from shlibrary
FAIL: gdb.base/annota1.exp: send SIGUSR1
FAIL: gdb.base/annota1.exp: break at 28

In all cases, there's a regexp that looks for
${srcdir}/${subdir}/${srcfile}, but the ${srcdir}/ component is
missing.  It doesn't seem to me that that should cause a fail; here's
a patch that makes the ${srcdir}/ component optional.

Because of Kevin's recent patch to the breakpoint info failure, it
makes the most sense to me to make ${subdir}/ optional as well, given
that he has an instance where, on one (but not all?) of those tests,
both ${srcdir} and ${subdir} are missing.  So that's what I've done.
(The patch looks messy, but that's just because the regexps in
question are so big: all I'm doing is adding a few parentheses and
question marks.)

Of course, it's possible that this really is a regression and that I'm
not correctly understanding what those tests are looking for.  I'm
using GCC 3.1 on Red Hat 7.3, for what that's worth.

Just out of curiosity, how many unexpected failures should I be
getting?  I'm usually getting 100 or so, which seems like an
unfortunately large number to me: either GDB has lots of regressions,
or the testsuite is misdiagnosing passes as failures, or there are
lots of FAILs that should be changed to XFAIL.  I'm hoping that many
of them are misdiagnoses; maybe I'll spend some time looking at that
when I'm not teaching and when I'm sick of symbol tables.

David Carlton
carlton@math.stanford.edu

2002-09-30  David Carlton  <carlton@math.stanford.edu>

	* gdb.base/annota1.exp: Enclose all uses of ${srcdir}/ and
	${subdir}/ in regular expressions in ()?.

Index: annota1.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.base/annota1.exp,v
retrieving revision 1.11
diff -u -p -r1.11 annota1.exp
--- annota1.exp	28 Sep 2002 01:12:04 -0000	1.11
+++ annota1.exp	30 Sep 2002 22:08:37 -0000
@@ -115,7 +115,7 @@ gdb_expect {
 #
 send_gdb "info break\n" 
 gdb_expect {
- -re "\r\n\032\032post-prompt\r\n\r\n\032\032breakpoints-headers\r\n\r\n\032\032field 0\r\nNum \r\n\032\032field 1\r\nType           \r\n\032\032field 2\r\nDisp \r\n\032\032field 3\r\nEnb \r\n\032\032field 4\r\nAddress    +\r\n\032\032field 5\r\nWhat\r\n\r\n\032\032breakpoints-table\r\n\r\n\032\032record\r\n\r\n\032\032field 0\r\n1   \r\n\032\032field 1\r\nbreakpoint     \r\n\032\032field 2\r\nkeep \r\n\032\032field 3\r\ny   \r\n\032\032field 4\r\n$hex +\r\n\032\032field 5\r\nin main at (${srcdir}/${subdir}/)?${srcfile}:$main_line\r\n\r\n\032\032breakpoints-table-end\r\n$gdb_prompt$" \
+ -re "\r\n\032\032post-prompt\r\n\r\n\032\032breakpoints-headers\r\n\r\n\032\032field 0\r\nNum \r\n\032\032field 1\r\nType           \r\n\032\032field 2\r\nDisp \r\n\032\032field 3\r\nEnb \r\n\032\032field 4\r\nAddress    +\r\n\032\032field 5\r\nWhat\r\n\r\n\032\032breakpoints-table\r\n\r\n\032\032record\r\n\r\n\032\032field 0\r\n1   \r\n\032\032field 1\r\nbreakpoint     \r\n\032\032field 2\r\nkeep \r\n\032\032field 3\r\ny   \r\n\032\032field 4\r\n$hex +\r\n\032\032field 5\r\nin main at (${srcdir}/)?(${subdir}/)?${srcfile}:$main_line\r\n\r\n\032\032breakpoints-table-end\r\n$gdb_prompt$" \
 	{pass "breakpoint info"}
    -re ".*$gdb_prompt$" { fail "breakpoint info" }
    timeout { fail "breakpoint info (timeout)" }
@@ -247,7 +247,7 @@ gdb_expect {
 #
 send_gdb "backtrace\n"
 gdb_expect {
-  -re "\r\n\032\032post-prompt\r\n\r\n\032\032frame-begin 0 $hex\r\n.0  \r\n(\032\032frame-address\r\n$hex\r\n\032\032frame-address-end\r\n in \r\n)*\032\032frame-function-name\r\nprintf\r\n\032\032frame-args\r\n \\(.*frame-end\r\n\r\n\032\032frame-begin 1 $hex\r\n.1  \r\n\032\032frame-address\r\n$hex\r\n\032\032frame-address-end\r\n in \r\n\032\032frame-function-name\r\nmain\r\n\032\032frame-args\r\n \\(\\)\r\n\032\032frame-source-begin\r\n at \r\n\032\032frame-source-file\r\n${srcdir}/${subdir}/${srcfile}\r\n\032\032frame-source-file-end\r\n:\r\n\032\032frame-source-line\r\n.*\r\n\032\032frame-source-end\r\n\r\n\r\n\032\032frame-end\r\n(\r\n\032\032frame-begin .*\r\n\r\n\032\032frame-end\r\n)*$gdb_prompt$" \
+  -re "\r\n\032\032post-prompt\r\n\r\n\032\032frame-begin 0 $hex\r\n.0  \r\n(\032\032frame-address\r\n$hex\r\n\032\032frame-address-end\r\n in \r\n)*\032\032frame-function-name\r\nprintf\r\n\032\032frame-args\r\n \\(.*frame-end\r\n\r\n\032\032frame-begin 1 $hex\r\n.1  \r\n\032\032frame-address\r\n$hex\r\n\032\032frame-address-end\r\n in \r\n\032\032frame-function-name\r\nmain\r\n\032\032frame-args\r\n \\(\\)\r\n\032\032frame-source-begin\r\n at \r\n\032\032frame-source-file\r\n(${srcdir}/)?(${subdir}/)?${srcfile}\r\n\032\032frame-source-file-end\r\n:\r\n\032\032frame-source-line\r\n.*\r\n\032\032frame-source-end\r\n\r\n\r\n\032\032frame-end\r\n(\r\n\032\032frame-begin .*\r\n\r\n\032\032frame-end\r\n)*$gdb_prompt$" \
 	  { pass "backtrace from shlibrary" }
   -re ".*$gdb_prompt$"     { fail "backtrace from shlibrary" }
   timeout	            { fail "backtrace from shlibrary (timeout)" }
@@ -263,7 +263,7 @@ gdb_expect {
 #
 send_gdb "signal SIGUSR1\n"
 gdb_expect {
-  -re "\r\n\032\032post-prompt\r\nContinuing with signal SIGUSR1.\r\n\r\n\032\032starting\r\n\r\n\032\032frames-invalid\r\n\r\n\032\032breakpoint 2\r\n\r\nBreakpoint 2, \r\n\032\032frame-begin 0 $hex\r\n\r\n\032\032frame-function-name\r\nhandle_USR1\r\n\032\032frame-args\r\n \\(\r\n\032\032arg-begin\r\nsig\r\n\032\032arg-name-end\r\n=\r\n\032\032arg-value -\r\n$decimal\r\n\032\032arg-end\r\n\\)\r\n\032\032frame-source-begin\r\n at \r\n\032\032frame-source-file\r\n${srcdir}/${subdir}/${srcfile}\r\n\032\032frame-source-file-end\r\n:\r\n\032\032frame-source-line\r\n.*\r\n\032\032frame-source-end\r\n\r\n\r\n\032\032source.*annota1.c:.*:.*:beg:$hex\r\n\r\n\032\032frame-end\r\n\r\n\032\032stopped\r\n$gdb_prompt$" \
+  -re "\r\n\032\032post-prompt\r\nContinuing with signal SIGUSR1.\r\n\r\n\032\032starting\r\n\r\n\032\032frames-invalid\r\n\r\n\032\032breakpoint 2\r\n\r\nBreakpoint 2, \r\n\032\032frame-begin 0 $hex\r\n\r\n\032\032frame-function-name\r\nhandle_USR1\r\n\032\032frame-args\r\n \\(\r\n\032\032arg-begin\r\nsig\r\n\032\032arg-name-end\r\n=\r\n\032\032arg-value -\r\n$decimal\r\n\032\032arg-end\r\n\\)\r\n\032\032frame-source-begin\r\n at \r\n\032\032frame-source-file\r\n(${srcdir}/)?(${subdir}/)?${srcfile}\r\n\032\032frame-source-file-end\r\n:\r\n\032\032frame-source-line\r\n.*\r\n\032\032frame-source-end\r\n\r\n\r\n\032\032source.*annota1.c:.*:.*:beg:$hex\r\n\r\n\032\032frame-end\r\n\r\n\032\032stopped\r\n$gdb_prompt$" \
 	{ pass "send SIGUSR1" }
   -re ".*$gdb_prompt$"     { fail "send SIGUSR1" }
   timeout	            { fail "send SIGUSR1 (timeout)" }
@@ -322,7 +322,7 @@ gdb_expect {
 #
 send_gdb "break main\n"
 gdb_expect {
-     -re "post-prompt.*\032\032breakpoints-invalid.*Breakpoint 4 at $hex: file ${srcdir}/${subdir}/${srcfile}, line $main_line.*$gdb_prompt$" \
+     -re "post-prompt.*\032\032breakpoints-invalid.*Breakpoint 4 at $hex: file (${srcdir}/)?(${subdir}/)?${srcfile}, line $main_line.*$gdb_prompt$" \
 	     { pass "break at 28" }
      -re ".*$gdb_prompt$"  { fail "break at 28" }
      timeout { fail "break at 28 (timeout)" }

Re: [rfa/testsuite] make annota1 regexps more generous

[Daniel Jacobowitz]

On Mon, Sep 30, 2002 at 03:24:59PM -0700, David Carlton wrote:
> I noticed today that annota1.exp seems to be generating some spurious
> FAILs on my machine, namely
> 
> FAIL: gdb.base/annota1.exp: breakpoint info
> FAIL: gdb.base/annota1.exp: backtrace from shlibrary
> FAIL: gdb.base/annota1.exp: send SIGUSR1
> FAIL: gdb.base/annota1.exp: break at 28
> 
> In all cases, there's a regexp that looks for
> ${srcdir}/${subdir}/${srcfile}, but the ${srcdir}/ component is
> missing.  It doesn't seem to me that that should cause a fail; here's
> a patch that makes the ${srcdir}/ component optional.
> 
> Because of Kevin's recent patch to the breakpoint info failure, it
> makes the most sense to me to make ${subdir}/ optional as well, given
> that he has an instance where, on one (but not all?) of those tests,
> both ${srcdir} and ${subdir} are missing.  So that's what I've done.
> (The patch looks messy, but that's just because the regexps in
> question are so big: all I'm doing is adding a few parentheses and
> question marks.)
> 
> Of course, it's possible that this really is a regression and that I'm
> not correctly understanding what those tests are looking for.  I'm
> using GCC 3.1 on Red Hat 7.3, for what that's worth.
> 
> Just out of curiosity, how many unexpected failures should I be
> getting?  I'm usually getting 100 or so, which seems like an
> unfortunately large number to me: either GDB has lots of regressions,
> or the testsuite is misdiagnosing passes as failures, or there are
> lots of FAILs that should be changed to XFAIL.  I'm hoping that many
> of them are misdiagnoses; maybe I'll spend some time looking at that
> when I'm not teaching and when I'm sick of symbol tables.

Depends on your compiler.  I had it down to a dozen or so MI failures
on x86 and a couple of XPASS's, using GCC 2.95.3.  3.2 is higher
because I need to finish a lot of C++ work... then there are some
random failures (a la pthreads; unpredictable).  And new regressions of
course.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: [rfa/testsuite] make annota1 regexps more generous

[Fernando Nasser]

(adding Elena)

It all depends on what emacs needs.  If emacs expect the full path then 
we have a regression.  If emacs is happy with ${srcdir}/${subdir} or 
both being omitted then we fix the test patterns.

Does someone know the answer?

Regards,
Fernando

Daniel Jacobowitz wrote:
> On Mon, Sep 30, 2002 at 03:24:59PM -0700, David Carlton wrote:
> 
>>I noticed today that annota1.exp seems to be generating some spurious
>>FAILs on my machine, namely
>>
>>FAIL: gdb.base/annota1.exp: breakpoint info
>>FAIL: gdb.base/annota1.exp: backtrace from shlibrary
>>FAIL: gdb.base/annota1.exp: send SIGUSR1
>>FAIL: gdb.base/annota1.exp: break at 28
>>
>>In all cases, there's a regexp that looks for
>>${srcdir}/${subdir}/${srcfile}, but the ${srcdir}/ component is
>>missing.  It doesn't seem to me that that should cause a fail; here's
>>a patch that makes the ${srcdir}/ component optional.
>>
>>Because of Kevin's recent patch to the breakpoint info failure, it
>>makes the most sense to me to make ${subdir}/ optional as well, given
>>that he has an instance where, on one (but not all?) of those tests,
>>both ${srcdir} and ${subdir} are missing.  So that's what I've done.
>>(The patch looks messy, but that's just because the regexps in
>>question are so big: all I'm doing is adding a few parentheses and
>>question marks.)
>>
>>Of course, it's possible that this really is a regression and that I'm
>>not correctly understanding what those tests are looking for.  I'm
>>using GCC 3.1 on Red Hat 7.3, for what that's worth.
>>
>>Just out of curiosity, how many unexpected failures should I be
>>getting?  I'm usually getting 100 or so, which seems like an
>>unfortunately large number to me: either GDB has lots of regressions,
>>or the testsuite is misdiagnosing passes as failures, or there are
>>lots of FAILs that should be changed to XFAIL.  I'm hoping that many
>>of them are misdiagnoses; maybe I'll spend some time looking at that
>>when I'm not teaching and when I'm sick of symbol tables.
> 
> 
> Depends on your compiler.  I had it down to a dozen or so MI failures
> on x86 and a couple of XPASS's, using GCC 2.95.3.  3.2 is higher
> because I need to finish a lot of C++ work... then there are some
> random failures (a la pthreads; unpredictable).  And new regressions of
> course.
> 


-- 
Fernando Nasser
Red Hat Canada Ltd.                     E-Mail:  fnasser@redhat.com
2323 Yonge Street, Suite #300
Toronto, Ontario   M4P 2C9

Re: [rfa/testsuite] make annota1 regexps more generous

[Eli Zaretskii]

> Date: Tue, 03 Dec 2002 10:56:37 -0500
> From: Fernando Nasser <fnasser@redhat.com>
> 
> It all depends on what emacs needs.  If emacs expect the full path then 
> we have a regression.  If emacs is happy with ${srcdir}/${subdir} or 
> both being omitted then we fix the test patterns.
> 
> Does someone know the answer?

You could try asking on emacs-devel@gnu.org, if no one else here
knows.

Re: [rfa/testsuite] make annota1 regexps more generous

[Michael Elizabeth Chastain]

David Carlton wrote:
> Just out of curiosity, how many unexpected failures should I be
> getting?  I'm usually getting 100 or so, which seems like an
> unfortunately large number to me: either GDB has lots of regressions,
> or the testsuite is misdiagnosing passes as failures, or there are
> lots of FAILs that should be changed to XFAIL.  I'm hoping that many
> of them are misdiagnoses; maybe I'll spend some time looking at that
> when I'm not teaching and when I'm sick of symbol tables.

Short answer: 50 to 100 at this point in history.

Long answer: it depends a lot on the compiler and the debug format.
See http://www.shout.net/~mec/sunday/2002-11-25/index.html and
then look at the "Counts" table for the counts that I am getting.

Michael C