Mirror of the gdb-patches mailing list
 help / color / mirror / Atom feed
* Re: [testsuite] add gdb.cp/gdb1355.exp
@ 2003-09-20 22:03 Michael Elizabeth Chastain
  0 siblings, 0 replies; 19+ messages in thread
From: Michael Elizabeth Chastain @ 2003-09-20 22:03 UTC (permalink / raw)
  To: ac131313; +Cc: carlton, gdb-patches

mec> Neither.  Just agreeing with what I'm assuming is David's assertion that 
mec> KFAIL and XFAIL both lead to people sweeping problems under the carpet.

Okay, I see.

I'm gonna go with FAIL because David likes it better and I'm
pretty neutral.

ac> Judgment call.  If its possible for gdb to identify the corrupt input 
ac> then it needs to alert the user to the problem, and failing to do this 
ac> is a bug.

I agree, but I don't have the resources to pursue this line of
inquiry.

Michael C


^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: [testsuite] add gdb.cp/gdb1355.exp
@ 2003-09-20 21:59 Michael Elizabeth Chastain
  0 siblings, 0 replies; 19+ messages in thread
From: Michael Elizabeth Chastain @ 2003-09-20 21:59 UTC (permalink / raw)
  To: ac131313; +Cc: carlton, gdb-patches

mec> Also I don't want to pay much attention to this until gdb 6.0
mec> is out the door.

ac> We'll all agree to that one :-)

Okay, let's leave it at that for a while.

Michael C


^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: [testsuite] add gdb.cp/gdb1355.exp
@ 2003-09-18 21:27 Michael Elizabeth Chastain
  2003-09-20 21:46 ` Andrew Cagney
  0 siblings, 1 reply; 19+ messages in thread
From: Michael Elizabeth Chastain @ 2003-09-18 21:27 UTC (permalink / raw)
  To: ac131313, carlton; +Cc: gdb-patches

ac> I also intend again proposing that existing XFAILS get yanked.
dc> That sounds okay to me.

Mmmm, I would like to dig down a level, rather than just blow
away setup_xfail calls.

First I would like to decide which hp configurations and which
*compilers* we support, do some test runs on them, and weed out stuff
that we don't need.  That would help out a lot.  Similarly for Sun.

Also I don't want to pay much attention to this until gdb 6.0
is out the door.

dc> There are a few XFAILS which have a bug number associated with them
dc> that's around 2400 or so: these are especially urgent, because it won't
dc> be all _that_ long before the bug number might start looking like a GDB
dc> bug number.

So gdb has a PR-2k problem?  :)

Michael C


^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: [testsuite] add gdb.cp/gdb1355.exp
@ 2003-09-18 21:22 Michael Elizabeth Chastain
  0 siblings, 0 replies; 19+ messages in thread
From: Michael Elizabeth Chastain @ 2003-09-18 21:22 UTC (permalink / raw)
  To: carlton; +Cc: gdb-patches

David C writes:

  1) The bug is completely undiagnosed.
  2) The bug is diagnosed but known not to be fixed.
  3) The bug was diagnosed, thought to be fixed, but not actually fixed.

  We have three states here, and only two bug labels.
  
I agree with this state list.  The problem is that the difference
between the states exists only in our minds.

  (gdb) print 2+2
  5

dc> That's the static way of looking at the situation; another is a
dc> dynamic way, namely how will the output of the tests change between
dc> runs.  In both of our proposals, if a regression pops up, we'll see a
dc> state transition: either PASS=>FAIL or PASS=>KFAIL.

I agree.  To me, that's the important part.  People should be looking
for transitions (comparing two gdb.sum's) rather than looking at
levels (looking at a single gdb.sum ... and then they ignore the
ERROR, WARNING, KFAIL, XFAIL, UNRESOLVED, UNTESTED, UNSUPPORTED
results too!)

dc> But my proposal also guarantees that there will always be a state
dc> transition after a bug is (thought to be) fixed: either KFAIL=>PASS or
dc> KFAIL=>FAIL.  In your proposal, the latter scenario won't lead to a
dc> state transition: it will remain KFAIL.

Ah, I'm really starting to see where you're coming from, with the
idea that someone should change the test script after someone
fixes the bug.

I guess it's not so bad.  I'm starting to come around.

Michael C


^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: [testsuite] add gdb.cp/gdb1355.exp
@ 2003-09-18 15:33 Michael Elizabeth Chastain
  2003-09-20 21:40 ` Andrew Cagney
  0 siblings, 1 reply; 19+ messages in thread
From: Michael Elizabeth Chastain @ 2003-09-18 15:33 UTC (permalink / raw)
  To: ac131313; +Cc: carlton, gdb-patches

ac> David is correct.  Look at the number of additions to gdb.asm since it 
ac> was changed to FAIL instead of ERROR.
ac> 
ac> However, we've adopted KFAIL XFAIL.

Ummm, I am dense here ... are you favoring FAIL or XFAIL
for this test result?

(The situation is that gcc produces bad debug info which causes
 gdb to give bad results.  The gcc bug has been fixed but we have
 a gdb test for the bug in case it comes back).

Michael C


^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: [testsuite] add gdb.cp/gdb1355.exp
@ 2003-09-18  2:01 Michael Elizabeth Chastain
  2003-09-18 14:27 ` Andrew Cagney
  2003-09-18 15:38 ` David Carlton
  0 siblings, 2 replies; 19+ messages in thread
From: Michael Elizabeth Chastain @ 2003-09-18  2:01 UTC (permalink / raw)
  To: carlton, gdb-patches

Okay, I'm willing to make it an XFAIL rather than a KFAIL.
I remember those threads.  The XFAIL feels odd but it does stand
for "external" and this bug is indeed a textbook external bug.

As far as making it a FAIL goes, I think David's argument comes down to:
people pay attention to FAIL, but not to XFAIL or KFAIL.  I really don't
like that situation.  I see the question as: whether to fight the
situation and use more accurate XFAIL/KFAIL rather than generic FAIL, or
to acquiesce to situation and give people what they expect to see.
I favor the former.

> Scenario 1: The bug is fixed.  A year later, the bug in question
> regresses.  But, if you just run 'make check', this isn't so obvious:
> no message gets printed in the output, and while the numbers at the
> end of the output will change, those numbers fluctuate for lots of
> reasons.  So it will only be noticed by somebody who actually looks at
> gdb.sum (or does the moral equivalent); ideally that will happen, but
> I'd rather not count on it (especially if the bug is on an obscure
> platform).

I'm willing to count on it.  Because if they just grep for '^FAIL'
then they will miss any tests that barf out completely with an
ERROR at the beginning and abort.  I have little sympathy for
people who analyze gdb.sum that way.

dc> Scenario 2: The fix works on platform A, but not on platform B.  And
dc> nobody using platform B has been paying close enough attention to the
dc> discussion of the bug to realize that the test is supposed to start
dc> passing.  Then, nobody might ever realize that something's wrong:
dc> platform A users think that everything is fine (which is the case for
dc> them), and platform B have no indication from the testsuite that
dc> something is wrong and that the patch didn't work as indicated.

Ah, this is the old scenario "change the KFAIL/XFAIL to a FAIL"
makes people on platform B aware that something has regressed
(I always investigate transitions from KFAIL -> FAIL).
I have some sympathy for this.

dc> Which is why, in the situation at question, I like it that you have the
dc> explicit failure branches, containing an informative comment as to when
dc> the bug occurred - I just want the letter 'k' removed in a few places.
dc> :-)

So we have two proposals here: I'd like to make this an XFAIL,
and David would like to make it a FAIL.

Daniel, or anyone else, do you have a preference?

Michael C


^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: [testsuite] add gdb.cp/gdb1355.exp
@ 2003-09-18  0:54 Michael Elizabeth Chastain
  2003-09-18  0:56 ` Daniel Jacobowitz
  0 siblings, 1 reply; 19+ messages in thread
From: Michael Elizabeth Chastain @ 2003-09-18  0:54 UTC (permalink / raw)
  To: carlton; +Cc: gdb-patches

dc> For one thing, it would be an XFAIL, because it's a GCC bug,
dc> not a GDB bug.

The test script, gdb.cp/gdb1355.exp, refers to PR gdb/1355.
gdb/1355 is an external PR and it refers to PR gcc/12066.
So there is a gdb PR in there.

dc> For another thing, though, the bug in question has been fixed,
dc> so we don't expect it to fail: if it does, it should show up as a FAIL.

This has been a controversy in the past, too.

My view is that "KFAIL" means "Known FAIL", which basically means
there is a PR for it (the PR is the locus of knowledge).

dc> I would leave in the new test, with branches and comments as is,
dc> but I would change all the occurrences of kfail to fail.

I prefer gdb1355.exp the way it is but I would be okay with that change
if other people want it that way.

Michael C


^ permalink raw reply	[flat|nested] 19+ messages in thread
* [testsuite] add gdb.cp/gdb1355.exp
@ 2003-09-18  0:00 Michael Elizabeth Chastain
  2003-09-18  0:48 ` David Carlton
  0 siblings, 1 reply; 19+ messages in thread
From: Michael Elizabeth Chastain @ 2003-09-18  0:00 UTC (permalink / raw)
  To: gdb-patches

Here are some tests for PR gdb/1355, which is a reference to
PR gcc/12066.  This was a bug in gcc which busted the basic
stabs types with C++ (C and Java worked fine).

If you use a version of gcc with this bug, plenty of other
things break, so this test does not add any coverage.
It does add an explicit KFAIL that points to the specific PR.

Testing: I tested this with gcc 2.95.3 and gcc 3.3.1 and one
of the broken versions of gcc HEAD, with both -dwarf-2 and stabs+.

I am committing this now.

Michael C

=== gdb1355.exp

# Copyright 2003 Free Software Foundation, Inc.

# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
# 
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.  

# Tests for PR gdb/1355, which is a reference to PR gcc/12066.
# 2003-08-26  Michael Chastain <mec@shout.net>

# This file is part of the gdb testsuite.

set ws "\[\r\n\t \]*"
set nl "\[\r\n\]+"

if $tracelevel then {
    strace $tracelevel
    }

if { [skip_cplus_tests] } { continue }

#
# test running programs
#
set prms_id 0
set bug_id 0

set testfile "gdb1355"
set srcfile ${testfile}.cc
set binfile ${objdir}/${subdir}/${testfile}

if  { [gdb_compile "${srcdir}/${subdir}/${srcfile}" "${binfile}" executable {debug c++}] != "" } {
     gdb_suppress_entire_file "Testcase compile failed, so all tests in this file will automatically fail."
}

if [get_compiler_info ${binfile} "c++"] {
    return -1
}

gdb_exit
gdb_start
gdb_reinitialize_dir $srcdir/$subdir
gdb_load ${binfile}

if ![runto_main] then {
    perror "couldn't run to main"
    continue
} 

# See http://sources.redhat.com/gdb/bugs/1355
# See http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12066
#
# g++ -gstabs+ does not emit stabs for fundamental types.
# They get emitted later inside other types, so they have no names
# and gdb cannot handle them.

set s_head  "${ws}(struct|class) mystruct \{(${ws}public:|)"
set s_tail  ".*"

set f_i     "${ws}int m_int;"
set f_c     "${ws}char m_char;"
set f_li    "${ws}long int m_long_int;"
set f_ui    "${ws}unsigned int m_unsigned_int;"
set f_lui   "${ws}long unsigned int m_long_unsigned_int;"
set f_si    "${ws}short int m_short_int;"
set f_sui   "${ws}short unsigned int m_short_unsigned_int;"
set f_uc    "${ws}unsigned char m_unsigned_char;"
set f_f     "${ws}float m_float;"
set f_d     "${ws}double m_double;"
set f_ld    "${ws}long double m_long_double;"
set f_b     "${ws}bool m_bool;"

set itc     "<invalid type code ${decimal}>"
set bad_i   "${ws}(${itc}|int) m_int;";
set bad_c   "${ws}(${itc}|char) m_char;"
set bad_li  "${ws}(${itc}|long int) m_long_int;"
set bad_ui  "${ws}(${itc}|unsigned int) m_unsigned_int;"
set bad_lui "${ws}(${itc}|long unsigned int) m_long_unsigned_int;"
set bad_si  "${ws}(${itc}|short int) m_short_int;"
set bad_sui "${ws}(${itc}|short unsigned int) m_short_unsigned_int;"
set bad_uc  "${ws}(${itc}|unsigned char) m_unsigned_char;"
set bad_f   "${ws}(${itc}|float) m_float;"
set bad_d   "${ws}(${itc}|double) m_double;"
set bad_ld  "${ws}(${itc}|long double) m_long_double;"
set bad_b   "${ws}(${itc}|bool) m_bool;"

gdb_test_multiple "ptype s1" "ptype s1" {
    -re "type = ${s_head}${f_i}${f_c}${f_li}${f_ui}${f_lui}${f_si}${f_sui}${f_uc}${f_f}${f_d}${f_ld}${f_b}${s_tail}\}$nl$gdb_prompt $" {
	pass "ptype s1"
    }
    -re "type = ${s_head}${bad_i}${bad_c}${bad_li}${bad_ui}${bad_lui}${bad_si}${bad_sui}${bad_uc}${bad_f}${bad_d}${bad_ld}${bad_b}${s_tail}\}$nl$gdb_prompt $" {
	# This happened with gcc HEAD 2003-08-20 08:00:00 UTC, -gstabs+.
	kfail "gdb/1355" "ptype s1"
    }
}

gdb_test_multiple "print s1" "print s1" {
    -re "$decimal = \{m_int = 117, m_char = 97 'a', m_long_int = 118, m_unsigned_int = 119, m_long_unsigned_int = 120, m_short_int = 123, m_short_unsigned_int = 124, m_unsigned_char = 98 'b', m_float = 125, m_double = 126, m_long_double = 127, m_bool = true\}$nl$gdb_prompt $" {
	pass "print s1"
    }
    -re "$decimal = \{m_int = 117, m_char = 97 'a', m_long_int = 118, m_unsigned_int = 119, m_long_unsigned_int = 120, m_short_int = 123, m_short_unsigned_int = 124, m_unsigned_char = 98 'b', m_float = 125, m_double = 126, m_long_double = 127, m_bool = 117\}$nl$gdb_prompt $" {
	# This pattern is very picky, but if more different output
	# shows up, I can just add more arms.  -- chastain 2003-08-26
	#
	# This happened with gcc HEAD 2003-08-20 08:00:00 UTC, -gstabs+.
	# Look at the value of m_bool.  It looks like gdb latched onto
	# random int type and then used the data at structure offset 0.
	kfail "gdb/1355" "print s1"
    }
}

=== gdb1355.c

struct mystruct
{
  int m_int;
  char m_char;
  long int m_long_int;
  unsigned int m_unsigned_int;
  long unsigned int m_long_unsigned_int;
  // long long int m_long_long_int;
  // long long unsigned int m_long_long_unsigned_int;
  short int m_short_int;
  short unsigned int m_short_unsigned_int;
  unsigned char m_unsigned_char;
  float m_float;
  double m_double;
  long double m_long_double;
  // complex int m_complex_int;
  // complex float m_complex_float;
  // complex long double m_complex_long_double;
  // wchar_t m_wchar_t;
  bool m_bool;
};

struct mystruct s1 =
{
    117, 'a', 118, 119, 120,
    // 121, 122,
    123, 124, 'b', 125.0, 126.0, 127.0,
    // complex int, complex float, complex long double, wchar_t,
    true
};

int main ()
{
  return 0;
}


^ permalink raw reply	[flat|nested] 19+ messages in thread
end of thread, other threads:[~2003-09-21  0:51 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2003-09-20 22:03 [testsuite] add gdb.cp/gdb1355.exp Michael Elizabeth Chastain
  -- strict thread matches above, loose matches on Subject: below --
2003-09-20 21:59 Michael Elizabeth Chastain
2003-09-18 21:27 Michael Elizabeth Chastain
2003-09-20 21:46 ` Andrew Cagney
2003-09-20 22:31   ` Daniel Jacobowitz
2003-09-21  0:51     ` Andrew Cagney
2003-09-18 21:22 Michael Elizabeth Chastain
2003-09-18 15:33 Michael Elizabeth Chastain
2003-09-20 21:40 ` Andrew Cagney
2003-09-18  2:01 Michael Elizabeth Chastain
2003-09-18 14:27 ` Andrew Cagney
2003-09-18 16:05   ` David Carlton
2003-09-18 15:38 ` David Carlton
2003-09-18  0:54 Michael Elizabeth Chastain
2003-09-18  0:56 ` Daniel Jacobowitz
2003-09-18  1:20   ` David Carlton
2003-09-18  1:27     ` David Carlton
2003-09-18  0:00 Michael Elizabeth Chastain
2003-09-18  0:48 ` David Carlton

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox