[rfa/testsuite] One pass/fail per expect list

[rfa/testsuite] One pass/fail per expect list

[Andrew Cagney]

[-- Attachment #1: Type: text/plain, Size: 202 bytes --]

Hello,

The attached tweaks gdb_expect_list{} so that it only prints one pass / 
fail / ... message for the testcase.

What are peoples thoughts on this change and if positive, is the patch ok?

Andrew

[-- Attachment #2: diffs --]
[-- Type: text/plain, Size: 1860 bytes --]

2002-06-14  Andrew Cagney  <ac131313@redhat.com>

	* lib/gdb.exp (gdb_expect_list): Only print one pass, fail, or
	supress message.

Index: lib/gdb.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/lib/gdb.exp,v
retrieving revision 1.20
diff -u -r1.20 gdb.exp
--- lib/gdb.exp	11 Jun 2002 20:37:05 -0000	1.20
+++ lib/gdb.exp	14 Jun 2002 19:21:30 -0000
@@ -1265,6 +1265,7 @@
     set ok 1
     if { $suppress_flag } {
 	set ok 0
+	unresolved "${test}"
     }
     while { ${index} < [llength ${list}] } {
 	set pattern [lindex ${list} ${index}]
@@ -1273,41 +1274,42 @@
 	    if { ${ok} } {
 		gdb_expect {
 		    -re "${pattern}${sentinel}" {
-			pass "${test}, pattern ${index} + sentinel"
+			# pass "${test}, pattern ${index} + sentinel"
 		    }
 		    -re "${sentinel}" {
-			fail "${test}, pattern ${index} + sentinel"
+			fail "${test} (pattern ${index} + sentinel)"
 			set ok 0
 		    }
 		    timeout {
-			fail "${test}, pattern ${index} + sentinel (timeout)"
+			fail "${test} (pattern ${index} + sentinel) (timeout)"
 			set ok 0
 		    }
 		}
 	    } else {
-		unresolved "${test}, pattern ${index} + sentinel"
+		# unresolved "${test}, pattern ${index} + sentinel"
 	    }
 	} else {
 	    if { ${ok} } {
 		gdb_expect {
 		    -re "${pattern}" {
-			pass "${test}, pattern ${index}"
+			# pass "${test}, pattern ${index}"
 		    }
 		    -re "${sentinel}" {
-			fail "${test}, pattern ${index}"
+			fail "${test} (pattern ${index})"
 			set ok 0
 		    }
 		    timeout {
-			fail "${test}, pattern ${index} (timeout)"
+			fail "${test} (pattern ${index}) (timeout)"
 			set ok 0
 		    }
 		}
 	    } else {
-		unresolved "${test}, pattern ${index}"
+		# unresolved "${test}, pattern ${index}"
 	    }
 	}
     }
     if { ${ok} } {
+	pass "${test}"
 	return 0
     } else {
 	return 1

Re: [rfa/testsuite] One pass/fail per expect list

[Fernando Nasser]

Andrew Cagney wrote:
> 
> Hello,
> 
> The attached tweaks gdb_expect_list{} so that it only prints one pass /
> fail / ... message for the testcase.
> 
> What are peoples thoughts on this change and if positive, is the patch ok?
> 

That is great!  I like it.  It does not inflate the number of failures
and makes it easier for the error log interpreters and databases as they
won't see repeated tests.

Thank you!


-- 
Fernando Nasser
Red Hat - Toronto                       E-Mail:  fnasser@redhat.com
2323 Yonge Street, Suite #300
Toronto, Ontario   M4P 2C9

Re: [rfa/testsuite] One pass/fail per expect list

[Andrew Cagney]

> Andrew Cagney wrote:
> 
>> 
>> Hello,
>> 
>> The attached tweaks gdb_expect_list{} so that it only prints one pass /
>> fail / ... message for the testcase.
>> 
>> What are peoples thoughts on this change and if positive, is the patch ok?
>> 
> 
> 
> That is great!  I like it.  It does not inflate the number of failures
> and makes it easier for the error log interpreters and databases as they
> won't see repeated tests.
> 
> Thank you!

It's in.  The number of pass/fails becomes a little bit more stable (and 
yes the motivation came from trying to compare test results :-).

Andrew

Re: [rfa/testsuite] One pass/fail per expect list

[Kevin Buettner]

On Jun 14,  3:46pm, Andrew Cagney wrote:

> The attached tweaks gdb_expect_list{} so that it only prints one pass / 
> fail / ... message for the testcase.
> 
> What are peoples thoughts on this change and if positive, is the patch ok?

Now that it's in and I've had a chance to use it, the only part that
I don't like is:

> -		unresolved "${test}, pattern ${index} + sentinel"
> +		# unresolved "${test}, pattern ${index} + sentinel"

I found it useful to see the (potentially long list of) UNRESOLVED
messages after the FAIL so that I knew how many other parts of the
test hadn't been checked.

Kevin

Re: [rfa/testsuite] One pass/fail per expect list

[Andrew Cagney]

> On Jun 14,  3:46pm, Andrew Cagney wrote:
> 
> 
>> The attached tweaks gdb_expect_list{} so that it only prints one pass / 
>> fail / ... message for the testcase.
>> 
>> What are peoples thoughts on this change and if positive, is the patch ok?
> 
> 
> Now that it's in and I've had a chance to use it, the only part that
> I don't like is:
> 
> 
>> -		unresolved "${test}, pattern ${index} + sentinel"
>> +		# unresolved "${test}, pattern ${index} + sentinel"
> 
> 
> I found it useful to see the (potentially long list of) UNRESOLVED
> messages after the FAIL so that I knew how many other parts of the
> test hadn't been checked.


What about including the total number of patterns in the test result 
message vis:
	.... (pattern N of NN)

Andrew

Re: [rfa/testsuite] One pass/fail per expect list

[Kevin Buettner]

On Aug 5, 11:56pm, Andrew Cagney wrote:

> >> The attached tweaks gdb_expect_list{} so that it only prints one pass / 
> >> fail / ... message for the testcase.
> >> 
> >> What are peoples thoughts on this change and if positive, is the patch ok?
> > 
> > Now that it's in and I've had a chance to use it, the only part that
> > I don't like is:
> > 
> > 
> >> -		unresolved "${test}, pattern ${index} + sentinel"
> >> +		# unresolved "${test}, pattern ${index} + sentinel"
> > 
> > 
> > I found it useful to see the (potentially long list of) UNRESOLVED
> > messages after the FAIL so that I knew how many other parts of the
> > test hadn't been checked.
> 
> 
> What about including the total number of patterns in the test result 
> message vis:
> 	.... (pattern N of NN)

In the FAIL message, right?  If so, I think that'd be okay.

Kevin

Re: [rfa/testsuite] One pass/fail per expect list

[Andrew Cagney]

> On Aug 5, 11:56pm, Andrew Cagney wrote:
> 
> 
>> >> The attached tweaks gdb_expect_list{} so that it only prints one pass / 
>> >> fail / ... message for the testcase.
>> >> 
>> >> What are peoples thoughts on this change and if positive, is the patch ok?
> 
>> > 
>> > Now that it's in and I've had a chance to use it, the only part that
>> > I don't like is:
>> > 
>> > 
> 
>> >> -		unresolved "${test}, pattern ${index} + sentinel"
>> >> +		# unresolved "${test}, pattern ${index} + sentinel"
> 
>> > 
>> > 
>> > I found it useful to see the (potentially long list of) UNRESOLVED
>> > messages after the FAIL so that I knew how many other parts of the
>> > test hadn't been checked.
> 
>> 
>> 
>> What about including the total number of patterns in the test result 
>> message vis:
>> 	.... (pattern N of NN)
> 
> 
> In the FAIL message, right?  If so, I think that'd be okay.

Yes.

Andrew