[RFC] testsuite/gdb.base/arithmet.exp: re-write

[RFC] testsuite/gdb.base/arithmet.exp: re-write

[Michael Elizabeth Chastain]

Here is a rewrite of gdb.base/arithmet.exp.

I have two reasons for rewriting this file.  First, I changed all the
send_gdb/gdb_expect constructions to single-line gdb_test calls.  This
reduces the file from 375 lines to 117 lines and makes it more readable.

Second, I made all the test names unique, mostly by removing extra
variable assignments.  This is my real motivation.

Testing: I tested this on native Red Hat Linux 7 and native Solaris 2.8.

Because this is a rewrite, I'm asking for comments first, not for approval
to commit.  Do you think that this is a worthwhile endeavor?  If it
is, do you have any comments on the new file?  If it is not worthwhile,
do you have alternate suggestions for how to fix the duplicate test names?

Also, the new file is easier to read than the differential patch,
so I'm simply attaching the new file.

Michael

===

2001-06-10  Michael Chastain  <chastain@redhat.com>

	* gdb.base/arithmet.exp: Use gdb_test instead of send_gdb/gdb_expect.
	Make all test names unique.

===

# Copyright 1998, 1999, 2001 Free Software Foundation, Inc.

# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
# 
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.  

# Please email any bugs, comments, and/or additions to this file to:
# bug-gdb@prep.ai.mit.edu

# This file was written by Elena Zannoni (ezannoni@cygnus.com)
# Rewritten to use gdb_test by Michael Chastain (chastain@redhat.com)

# This file is part of the gdb testsuite
#
# tests for correctness of arithmetic operators, associativity and precedence
# with integer type variables
# 

if $tracelevel then {
	strace $tracelevel
	}

#
# test running programs
#
set prms_id 0
set bug_id 0

set testfile "int-type"
set srcfile ${testfile}.c
set binfile ${objdir}/${subdir}/${testfile}

if  { [gdb_compile "${srcdir}/${subdir}/${srcfile}" "${binfile}" executable {debug additional_flags=-w}] != "" } {
     gdb_suppress_entire_file "Testcase compile failed, so all tests in this file will automatically fail."
    }

gdb_exit
gdb_start
gdb_reinitialize_dir $srcdir/$subdir
gdb_load ${binfile}


#
# set it up at a breakpoint so we can play with the variable values
#

if ![runto_main] then {
    perror "couldn't run to breakpoint"
    continue
}

#
# test expressions with "int" types
#

gdb_test "set variable x=14" ""
gdb_test "set variable y=2"  ""
gdb_test "set variable z=2"  ""
gdb_test "set variable w=3"  ""

gdb_test "print x" "14"
gdb_test "print y"  "2"
gdb_test "print z"  "2"
gdb_test "print w"  "3"
 
gdb_test "print x+y" "16"
gdb_test "print x-y" "12"
gdb_test "print x*y" "28"
gdb_test "print x/y"  "7"
gdb_test "print x%y"  "0"

#  x  y  z  w
# 14  2  2  3

# Test associativity of +, -, *, % ,/

gdb_test "print x+y+z" "18"
gdb_test "print x-y-z" "10"
gdb_test "print x*y*z" "56"
gdb_test "print x/y/z"  "3"
gdb_test "print x%y%z"  "0"

# test precedence rules on pairs of arithmetic operators

gdb_test "set variable x=10" ""
gdb_test "set variable y=4"  ""

#  x  y  z  w
# 10  4  2  3

gdb_test "print x+y-z" "12"
gdb_test "print x+y*z" "18"
gdb_test "print x+y%w" "11"
gdb_test "print x+y/w" "11"
gdb_test "print x-y*z"  "2"
gdb_test "print x-y%z" "10"
gdb_test "print x-y/z"  "8"
gdb_test "print x*y/z" "20"
gdb_test "print x*y%w"  "1"
gdb_test "print x/y%w"  "2"

# test use of parentheses to enforce different order of evaluation

gdb_test "print x-(y+w)"  "3"
gdb_test "print x/(y*w)"  "0"
gdb_test "print x-(y/w)"  "9"
gdb_test "print (x+y)*w" "42"

Re: [RFC] testsuite/gdb.base/arithmet.exp: re-write

[Fernando Nasser]

Michael Elizabeth Chastain wrote:
> 
> Because this is a rewrite, I'm asking for comments first, not for approval
> to commit.  Do you think that this is a worthwhile endeavor?  If it
> is, do you have any comments on the new file?  If it is not worthwhile,
> do you have alternate suggestions for how to fix the duplicate test names?
> 

Absolutely!  It is worthwhile. And it is very nice of you of taking the
time to do this.

When tests use the gdb_test they re-use all the knowledge that we put in
that function code (like the possible outcomes and so one).  And the
unique names help people that use scripts to process the tests.

-- 
Fernando Nasser
Red Hat Canada Ltd.                     E-Mail:  fnasser@redhat.com
2323 Yonge Street, Suite #300
Toronto, Ontario   M4P 2C9

Re: [RFC] testsuite/gdb.base/arithmet.exp: re-write

[Jim Blandy]

Michael Elizabeth Chastain <chastain@cygnus.com> writes:
> Because this is a rewrite, I'm asking for comments first, not for approval
> to commit.  Do you think that this is a worthwhile endeavor?  If it
> is, do you have any comments on the new file?  If it is not worthwhile,
> do you have alternate suggestions for how to fix the duplicate test
> names?

The readability improvement sounds worthwhile to me.

Can you characterize the differences (if any) between the operations
GDB executes in the old and proposed new versions?

Re: [RFC] testsuite/gdb.base/arithmet.exp: re-write

[Michael Elizabeth Chastain]

Michael Snyder writes:
> gdb_test "print x-y-z" "10" "test x-y-z == 10"
> gdb_test "print x-y-z" "9"  "test x-y-z == 9"

The third parameter works well for this case.

Here is an awkward case:

  gdb_test "set variable z=2" ""
  ...
  gdb_test "set variable z=3" ""
  ...
  gdb_test "set variable z=2" ""
  ...
  gdb_test "set variable z=3" ""
  ...

I chose to make an operational change by re-organizing the assignments:

  gdb_test "set variable z=2" ""
  gdb_test "set variable w=3" ""
  ...
  # no more assignments
  # use either "z" or "w" as appropriate

Alternatively, I could do some 3rd-argument clutter:

  gdb_test "set variable z=2" "" ".1. set variable z=2"
  ...
  gdb_test "set variable z=3" "" ".2. set variable z=3"
  ...
  gdb_test "set variable z=2" "" ".3. set variable z=2"
  ...
  gdb_test "set variable z=3" "" ".4. set variable z=3"
  ...

There's really no general clean way to do this.  And there's no tool
in common use that tells people "whoops, you just wrote a duplicate test".

Michael

Re: [RFC] testsuite/gdb.base/arithmet.exp: re-write

[Jim Blandy]

That sounds totally fine to me --- thanks for explaining it.

I'm not a maintainer of the test suite, but I would be happy to see
this change go in.

Michael Elizabeth Chastain <chastain@cygnus.com> writes:

> 
> Jim Blandy writes:
> 
> > Can you characterize the differences (if any) between the operations
> > GDB executes in the old and proposed new versions?
> 
> arithmet.exp operates on a program with four variables, "x" "y" "z" and "w".
> It assigns values to those variables and then does a lot of simple
> add-subtract-multiply-divide operations to check operator precedence
> and associativity.
> 
> The tests themselves are already unique (the same expression is never
> used twice) so my new version executes the same tests as the old version.
> 
> The assignment statements are different.  The old version has a lot of
> alternation between "set variable z=2", "set variable z=3", and
> "set variable z=2" again.  My new version has two groups of
> "set variable" statements and that's it.
> 
> I believe that the point of arithmet.exp is to test the expressions,
> not the assignments.  If desired, I could do the work in several phases,
> where the first phase is totally operationally compatible (and still
> has duplicate test names), so that people can be comfortable with a
> massive changeover to "gdb_test" and then operational changes can be
> reviewed separately (and much more effectively).
> 
> Also, I'm not totally happy with the way that I achieve uniqueness.
> I achieve it quietly by knowing that this kind of code is bad:
> 
>   gdb_test "set variable z=2" ""
>   gdb_test "print x-y-z" "10"
>   gdb_test "set variable z=3" ""
>   gdb_test "print x-y-z" "9"	# oops, duplicate test
> 
> So the test script gets influenced by this external constraint that
> doesn't actually get enforced when people make changes.  We live with
> it I guess.
> 
> Michael
> 

Re: [RFC] testsuite/gdb.base/arithmet.exp: re-write

[Michael Elizabeth Chastain]

Jim Blandy writes:

> Can you characterize the differences (if any) between the operations
> GDB executes in the old and proposed new versions?

arithmet.exp operates on a program with four variables, "x" "y" "z" and "w".
It assigns values to those variables and then does a lot of simple
add-subtract-multiply-divide operations to check operator precedence
and associativity.

The tests themselves are already unique (the same expression is never
used twice) so my new version executes the same tests as the old version.

The assignment statements are different.  The old version has a lot of
alternation between "set variable z=2", "set variable z=3", and
"set variable z=2" again.  My new version has two groups of
"set variable" statements and that's it.

I believe that the point of arithmet.exp is to test the expressions,
not the assignments.  If desired, I could do the work in several phases,
where the first phase is totally operationally compatible (and still
has duplicate test names), so that people can be comfortable with a
massive changeover to "gdb_test" and then operational changes can be
reviewed separately (and much more effectively).

Also, I'm not totally happy with the way that I achieve uniqueness.
I achieve it quietly by knowing that this kind of code is bad:

  gdb_test "set variable z=2" ""
  gdb_test "print x-y-z" "10"
  gdb_test "set variable z=3" ""
  gdb_test "print x-y-z" "9"	# oops, duplicate test

So the test script gets influenced by this external constraint that
doesn't actually get enforced when people make changes.  We live with
it I guess.

Michael