Re: [patch/rfc] Remove all setup_xfail's from testsuite/gdb.mi/

Andrew Cagney
2003-01-15T15:55:20Z

[-- Attachment #1: Type: text/plain, Size: 215 bytes --]

Fernando,

What is the resolution of this thread?
http://sources.redhat.com/ml/gdb-patches/2002-10/msg00508.html
(you'll probably want to read all of it :-)
(I'm now going back and emptying my mail box :-()

Andrew

[-- Attachment #2: mailbox-message://ac131313@movemail/fsf/gdb/patches#24629765 --]
[-- Type: message/rfc822, Size: 18092 bytes --]

[-- Attachment #2.1.1: Type: text/plain, Size: 1108 bytes --]

Hello,

GDB's testsuite is known to be full of xfails that are really kfails or
testsuite bugs. Rather than try to audit each xfail in turn, the
proposal as been to rip out all the xfails (creating a clean slate) and
start marking up the tests from scratch - two steps forward but first
one step back.

I figure I might as well try to get the ball rolling on this and find
out just how much real resistance there is going to be to a change like
this. To that end, this removes all xfail's from the gdb.mi testsuite.
Similar tests, for the other directories, would follow.

On a GNU/Linux i386 system (Red Hat 7.2 I think) I see:

Before:

=== gdb Summary ===

# of expected passes 8298
# of unexpected failures 60
# of unexpected successes 10
# of expected failures 172
# of unsupported tests 2

After:

=== gdb Summary ===

# of expected passes 8298
# of unexpected failures 88
# of unexpected successes 10
# of expected failures 144
# of unsupported tests 2

Comments?

Andrew

[-- Attachment #2.1.2: diffs --]
[-- Type: text/plain, Size: 13873 bytes --]

2002-10-24 Andrew Cagney <cagney@redhat.com>

* mi-var-block.exp, mi1-var-block.exp: Remove all uses of
setup_xfail and clear_xfail.
* mi-var-cmd.exp, mi1-var-cmd.exp: Ditto.
* mi1-var-display.exp, mi-var-display.exp: Ditto.
* mi-var-child.exp, mi1-var-child.exp: Ditto.
* mi-break.exp, mi1-break.exp: Remove all uses of setup_xfail.
Update copyright.

Index: gdb.mi/mi-break.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi-break.exp,v
retrieving revision 1.5
diff -u -r1.5 mi-break.exp
--- gdb.mi/mi-break.exp 27 Jun 2001 17:27:07 -0000 1.5
+++ gdb.mi/mi-break.exp 24 Oct 2002 18:07:01 -0000
@@ -1,4 +1,4 @@
-# Copyright 1999 Free Software Foundation, Inc.
+# Copyright 1999, 2002 Free Software Foundation, Inc.

# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -101,27 +101,22 @@
# -break-insert -r .*llee
# -break-list

- setup_xfail "*-*-*"
mi_gdb_test "122-break-insert -r main" \
"122\\^done,bkpt=\{number=\"5\",addr=\"$hex\",file=\".*basics.c\",line=\"32\"\}" \
"break-insert -r operation"

- setup_xfail "*-*-*"
mi_gdb_test "133-break-insert -r callee2" \
"133\\^done,bkpt=\{number=\"6\",addr=\"$hex\",file=\".*basics.c\",line=\"22\"\}" \
"insert breakpoint with regexp callee2"

- setup_xfail "*-*-*"
mi_gdb_test "144-break-insert -r callee" \
"144\\^done,bkpt=\{number=\"7\",addr=\"$hex\",file=\".*basics.c\",line=\"27\"\},bkpt=\{number=\"8\",addr=\"$hex\",file=\".*basics.c\",line=\"22\"\},bkpt=\{number=\"9\",addr=\"$hex\",file=\".*basics.c\",line=\"17\"\},bkpt=\{number=\"10\",addr=\"$hex\",file=\".*basics.c\",line=\"8\"\}" \
"insert breakpoint with regexp callee"

- setup_xfail "*-*-*"
mi_gdb_test "155-break-insert -r \.\*llee" \
"155\\^done,bkpt=\{number=\"11\",addr=\"$hex\",file=\".*basics.c\",line=\"27\"\},bkpt=\{number=\"12\",addr=\"$hex\",file=\".*basics.c\",line=\"22\"\},bkpt=\{number=\"13\",addr=\"$hex\",file=\".*basics.c\",line=\"17\"\},bkpt=\{number=\"14\",addr=\"$hex\",file=\".*basics.c\",line=\"8\"\}" \
"insert breakpoint with regexp .*llee"

- setup_xfail "*-*-*"
mi_gdb_test "166-break-list" \
"1\\\^done,BreakpointTable=\{nr_rows=\".\",nr_cols=\".\",hdr=\\\[\{width=\".*\",alignment=\".*\",col_name=\"number\",colhdr=\"Num\"\}.*colhdr=\"Type\".*colhdr=\"Disp\".*colhdr=\"Enb\".*colhdr=\"Address\".*colhdr=\"What\".*\\\],body=\\\[bkpt=\{number=\"5\",type=\"breakpoint\",disp=\"keep\",enabled=\"y\",addr=\"$hex\",func=\"main\",file=\".*basics.c\",line=\"32\",times=\"0\"\},.*\}\\\]\}" \
"list of breakpoints"
Index: gdb.mi/mi-var-block.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi-var-block.exp,v
retrieving revision 1.8
diff -u -r1.8 mi-var-block.exp
--- gdb.mi/mi-var-block.exp 3 Oct 2002 20:03:55 -0000 1.8
+++ gdb.mi/mi-var-block.exp 24 Oct 2002 18:07:01 -0000
@@ -142,17 +142,13 @@
# Test: c_variable-3.7
# Desc: check that outer foo in scope and inner foo out of scope
# Note: also a known gdb problem
-setup_xfail *-*-*
mi_gdb_test "-var-update inner_foo" \
"\\^done,changelist=\{FIXME\}" \
"update inner_foo: should be out of scope: KNOWN PROBLEM"
-clear_xfail *-*-*

-setup_xfail *-*-*
mi_gdb_test "-var-evaluate-expression inner_foo" \
"\\^done,value=\{FIXME\}" \
"evaluate inner_foo: should be out of scope: KNOWN PROBLEM"
-clear_xfail *-*-*

mi_gdb_test "-var-update foo" \
"\\^done,changelist=\\\[\\\]" \
@@ -173,11 +169,9 @@

# Test: c_variable-3.8
# Desc: check that foo2 out of scope (known gdb problem)
-setup_xfail *-*-*
mi_gdb_test "-var-update foo2" \
"\\^done,changelist=\{FIXME\}" \
"update foo2: should be out of scope: KNOWN PROBLEM"
-clear_xfail *-*-*

# step to "cb = 21;"
send_gdb "-exec-step\n"
@@ -191,16 +185,12 @@

# Test: c_variable-3.9
# Desc: check that only cb is in scope (known gdb problem)
-setup_xfail *-*-*
mi_gdb_test "-var-update foo2" \
"\\^done,changelist=\\\[FIXME\\\]" \
"update foo2 should be out of scope: KNOWN PROBLEM"
-clear_xfail *-*-*
-setup_xfail *-*-*
mi_gdb_test "-var-update foo" \
"\\^done,changelist=\{FIXME\}" \
"update foo should be out of scope: KNOWN PROBLEM"
-clear_xfail *-*-*
mi_gdb_test "-var-update cb" \
"\\^done,changelist=\\\[\\\]" \
"update cb"
Index: gdb.mi/mi-var-child.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi-var-child.exp,v
retrieving revision 1.12
diff -u -r1.12 mi-var-child.exp
--- gdb.mi/mi-var-child.exp 3 Oct 2002 20:03:55 -0000 1.12
+++ gdb.mi/mi-var-child.exp 24 Oct 2002 18:07:01 -0000
@@ -1176,11 +1176,9 @@
# Test: c_variable-5.51
# Desc: check that psnp->next->long_ptr (and [1].long_ptr) changed
# Why does this have a FIXME?
-setup_xfail *-*-*
mi_gdb_test "-var-update *" \
"FIXME\\^done,changelist=\\\[\{name=\"psnp->ptrs.0.next.long_ptr\",in_scope=\"true\",type_changed=\"false\"\}\\\]" \
"update all vars psnp->next->long_ptr (and 1.long_ptr) changed"
-clear_xfail *-*-*

# This command produces this error message:
# &"warning: varobj_list: assertion failed - mycount <> 0\n"
Index: gdb.mi/mi-var-cmd.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi-var-cmd.exp,v
retrieving revision 1.14
diff -u -r1.14 mi-var-cmd.exp
--- gdb.mi/mi-var-cmd.exp 23 Oct 2002 23:57:28 -0000 1.14
+++ gdb.mi/mi-var-cmd.exp 24 Oct 2002 18:07:02 -0000
@@ -429,11 +429,9 @@
# special case that, since it is not what a human expects to
# see.

-setup_xfail *-*-*
mi_gdb_test "-var-update *" \
"\\^done,changelist=\{FIXME: WHAT IS CORRECT HERE\}" \
"update all vars: changed FIXME"
-clear_xfail *-*-*

mi_step_to "subroutine1" "\{name=\"i\",value=\".*\"\},\{name=\"l\",value=\".*\"\}" \
"var-cmd.c" "148" "step at subroutine1 (2)"
Index: gdb.mi/mi-var-display.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi-var-display.exp,v
retrieving revision 1.8
diff -u -r1.8 mi-var-display.exp
--- gdb.mi/mi-var-display.exp 17 Sep 2002 20:30:24 -0000 1.8
+++ gdb.mi/mi-var-display.exp 24 Oct 2002 18:07:02 -0000
@@ -491,13 +491,11 @@
"\\^done,name=\"e\",numchild=\"0\",type=\"enum foo\"" \
"create local variable e"

-setup_xfail "*-*-*"
# Test: c_variable-7.51
# Desc: value of e
mi_gdb_test "-var-evaluate-expression e" \
"\\^done,value=\"FIXME\"" \
"eval variable e"
-clear_xfail "*-*-*"

# Test: c_variable-7.52
# Desc: type of e
@@ -529,13 +527,11 @@
"\\^done,name=\"anone\",numchild=\"0\",type=\"enum \{\\.\\.\\.\}\"" \
"create local variable anone"

-setup_xfail "*-*-*"
# Test: c_variable-7.61
# Desc: value of anone
mi_gdb_test "-var-evaluate-expression anone" \
"\\^done,value=\"A\"" \
"eval variable anone"
-clear_xfail "*-*-*"

# Test: c_variable-7.70
Index: gdb.mi/mi1-break.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi1-break.exp,v
retrieving revision 1.2
diff -u -r1.2 mi1-break.exp
--- gdb.mi/mi1-break.exp 1 Oct 2002 16:01:37 -0000 1.2
+++ gdb.mi/mi1-break.exp 24 Oct 2002 18:07:02 -0000
@@ -1,4 +1,4 @@
-# Copyright 1999 Free Software Foundation, Inc.
+# Copyright 1999, 2002 Free Software Foundation, Inc.

# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -101,27 +101,22 @@
# -break-insert -r .*llee
# -break-list

- setup_xfail "*-*-*"
mi_gdb_test "122-break-insert -r main" \
"122\\^done,bkpt=\{number=\"5\",addr=\"$hex\",file=\".*basics.c\",line=\"32\"\}" \
"break-insert -r operation"

- setup_xfail "*-*-*"
mi_gdb_test "133-break-insert -r callee2" \
"133\\^done,bkpt=\{number=\"6\",addr=\"$hex\",file=\".*basics.c\",line=\"22\"\}" \
"insert breakpoint with regexp callee2"

- setup_xfail "*-*-*"
mi_gdb_test "144-break-insert -r callee" \
"144\\^done,bkpt=\{number=\"7\",addr=\"$hex\",file=\".*basics.c\",line=\"27\"\},bkpt=\{number=\"8\",addr=\"$hex\",file=\".*basics.c\",line=\"22\"\},bkpt=\{number=\"9\",addr=\"$hex\",file=\".*basics.c\",line=\"17\"\},bkpt=\{number=\"10\",addr=\"$hex\",file=\".*basics.c\",line=\"8\"\}" \
"insert breakpoint with regexp callee"

- setup_xfail "*-*-*"
mi_gdb_test "155-break-insert -r \.\*llee" \
"155\\^done,bkpt=\{number=\"11\",addr=\"$hex\",file=\".*basics.c\",line=\"27\"\},bkpt=\{number=\"12\",addr=\"$hex\",file=\".*basics.c\",line=\"22\"\},bkpt=\{number=\"13\",addr=\"$hex\",file=\".*basics.c\",line=\"17\"\},bkpt=\{number=\"14\",addr=\"$hex\",file=\".*basics.c\",line=\"8\"\}" \
"insert breakpoint with regexp .*llee"

- setup_xfail "*-*-*"
mi_gdb_test "166-break-list" \
"1\\\^done,BreakpointTable=\{nr_rows=\".\",nr_cols=\".\",hdr=\\\[\{width=\".*\",alignment=\".*\",col_name=\"number\",colhdr=\"Num\"\}.*colhdr=\"Type\".*colhdr=\"Disp\".*colhdr=\"Enb\".*colhdr=\"Address\".*colhdr=\"What\".*\\\],body=\\\[bkpt=\{number=\"5\",type=\"breakpoint\",disp=\"keep\",enabled=\"y\",addr=\"$hex\",func=\"main\",file=\".*basics.c\",line=\"32\",times=\"0\"\},.*\}\\\]\}" \
"list of breakpoints"
Index: gdb.mi/mi1-var-block.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi1-var-block.exp,v
retrieving revision 1.2
diff -u -r1.2 mi1-var-block.exp
--- gdb.mi/mi1-var-block.exp 1 Oct 2002 16:01:37 -0000 1.2
+++ gdb.mi/mi1-var-block.exp 24 Oct 2002 18:07:02 -0000
@@ -142,17 +142,13 @@
# Test: c_variable-3.7
# Desc: check that outer foo in scope and inner foo out of scope
# Note: also a known gdb problem
-setup_xfail *-*-*
mi_gdb_test "-var-update inner_foo" \
"\\^done,changelist=\{FIXME\}" \
"update inner_foo: should be out of scope: KNOWN PROBLEM"
-clear_xfail *-*-*

-setup_xfail *-*-*
mi_gdb_test "-var-evaluate-expression inner_foo" \
"\\^done,value=\{FIXME\}" \
"evaluate inner_foo: should be out of scope: KNOWN PROBLEM"
-clear_xfail *-*-*

mi_gdb_test "-var-update foo" \
"\\^done,changelist=\{\}" \
@@ -173,11 +169,9 @@

# Test: c_variable-3.8
# Desc: check that foo2 out of scope (known gdb problem)
-setup_xfail *-*-*
mi_gdb_test "-var-update foo2" \
"\\^done,changelist=\{FIXME\}" \
"update foo2: should be out of scope: KNOWN PROBLEM"
-clear_xfail *-*-*

# step to "cb = 21;"
send_gdb "-exec-step\n"
@@ -191,16 +185,12 @@

# Test: c_variable-3.9
# Desc: check that only cb is in scope (known gdb problem)
-setup_xfail *-*-*
mi_gdb_test "-var-update foo2" \
"\\^done,changelist=\{FIXME\}" \
"update foo2 should be out of scope: KNOWN PROBLEM"
-clear_xfail *-*-*
-setup_xfail *-*-*
mi_gdb_test "-var-update foo" \
"\\^done,changelist=\{FIXME\}" \
"update foo should be out of scope: KNOWN PROBLEM"
-clear_xfail *-*-*
mi_gdb_test "-var-update cb" \
"\\^done,changelist=\{\}" \
"update cb"
Index: gdb.mi/mi1-var-child.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi1-var-child.exp,v
retrieving revision 1.2
diff -u -r1.2 mi1-var-child.exp
--- gdb.mi/mi1-var-child.exp 1 Oct 2002 16:01:37 -0000 1.2
+++ gdb.mi/mi1-var-child.exp 24 Oct 2002 18:07:03 -0000
@@ -1176,11 +1176,9 @@
# Test: c_variable-5.51
# Desc: check that psnp->next->long_ptr (and [1].long_ptr) changed
# Why does this have a FIXME?
-setup_xfail *-*-*
mi_gdb_test "-var-update *" \
"FIXME\\^done,changelist=\{name=\"psnp->ptrs.0.next.long_ptr\",in_scope=\"true\",type_changed=\"false\"\}" \
"update all vars psnp->next->long_ptr (and 1.long_ptr) changed"
-clear_xfail *-*-*

# This command produces this error message:
# &"warning: varobj_list: assertion failed - mycount <> 0\n"
Index: gdb.mi/mi1-var-cmd.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi1-var-cmd.exp,v
retrieving revision 1.3
diff -u -r1.3 mi1-var-cmd.exp
--- gdb.mi/mi1-var-cmd.exp 23 Oct 2002 23:57:28 -0000 1.3
+++ gdb.mi/mi1-var-cmd.exp 24 Oct 2002 18:07:03 -0000
@@ -429,11 +429,9 @@
# special case that, since it is not what a human expects to
# see.

-setup_xfail *-*-*
mi_gdb_test "-var-update *" \
"\\^done,changelist=\{FIXME: WHAT IS CORRECT HERE\}" \
"update all vars: changed FIXME"
-clear_xfail *-*-*

mi_step_to "subroutine1" "\{name=\"i\",value=\".*\"\},\{name=\"l\",value=\".*\"\}" \
"var-cmd.c" "148" "step at subroutine1 (2)"
Index: gdb.mi/mi1-var-display.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/gdb.mi/mi1-var-display.exp,v
retrieving revision 1.2
diff -u -r1.2 mi1-var-display.exp
--- gdb.mi/mi1-var-display.exp 1 Oct 2002 16:01:37 -0000 1.2
+++ gdb.mi/mi1-var-display.exp 24 Oct 2002 18:07:03 -0000
@@ -491,13 +491,11 @@
"\\^done,name=\"e\",numchild=\"0\",type=\"enum foo\"" \
"create local variable e"

-setup_xfail "*-*-*"
# Test: c_variable-7.51
# Desc: value of e
mi_gdb_test "-var-evaluate-expression e" \
"\\^done,value=\"FIXME\"" \
"eval variable e"
-clear_xfail "*-*-*"

# Test: c_variable-7.52
# Desc: type of e
@@ -529,13 +527,11 @@
"\\^done,name=\"anone\",numchild=\"0\",type=\"enum \{\\.\\.\\.\}\"" \
"create local variable anone"

-setup_xfail "*-*-*"
# Test: c_variable-7.61
# Desc: value of anone
mi_gdb_test "-var-evaluate-expression anone" \
"\\^done,value=\"A\"" \
"eval variable anone"
-clear_xfail "*-*-*"

# Test: c_variable-7.70