Re: Test suite docs

[Daniel Jacobowitz <drow@false.org>]

On Sat, Jan 13, 2007 at 05:07:42PM +0200, Eli Zaretskii wrote:
> > >   . Where do I find the canonical results for my platform?
> > 
> > In theory one should not see any FAILS, and one should work on
> > eliminating any KFAILS.

I agree with Mark and I have been working extensively over the last
month just to eliminate FAILs on one single platform - we have not
cared for the testsuite well, and it's a fantastically hard thing
to do because of all the variables.

In practice, I always do exactly what Joel described.  I keep the
last gdb.sum around and diff them.  I just use diff.  At work we
have some rather more sophisticated scripts to compare .sum files
but I'm not familiar with them.

> FWIW, I present below the failures and unexpected successes on the
> system where I ran the test suite (an x86_64 Ubuntu box).  With the
> exception of Ada/gnatmake related failures, they all seem to be real
> problems.  In particular, maint.exp, corefile.exp, sigaltstack.exp,
> sigbpt.exp, sigstep.exp, and staticthreads/watchthreads look
> disturbing.  Does anyone else get them on a GNU/Linux system?

I would not be disturbed by any of them.

>     FAIL: gdb.base/corefile.exp: accessing mmapped data in core file (mapping address not found in core file)

A change in behavior of the Linux kernel.  David Miller and I talked
about it again a week or two ago.  I know how to make the test pass
again but I'm waiting to see what we do to the kernel behavior first.

>     FAIL: gdb.base/maint.exp: maint info sections .text

No idea.

>     FAIL: gdb.base/sigaltstack.exp: finish from catch LEAF (the program exited)

The frame_unwind_address_in_block work I was discussing with Mark is
for this test and the related signal tests...

>     FAIL: gdb.base/sigbpt.exp: stepi; stepi out of handler

...except that some of them are caused by a bug in glibc, which was
just recently fixed; your system's probably has the bug.  It's not
a serious bug in practice, for anything except the gdb testsuite.

>     FAIL: gdb.threads/staticthreads.exp: running to main in runto
>     FAIL: gdb.threads/staticthreads.exp: Continue to main's call of sem_post
>     FAIL: gdb.threads/staticthreads.exp: handle SIG32 helps

I have submitted potential patches for this bug to the glibc
maintainers periodically for over a year.  I've about given up.

>     FAIL: gdb.threads/watchthreads.exp: threaded watch loop
>     FAIL: gdb.threads/watchthreads.exp: first watchpoint on args[1] hit
>     FAIL: gdb.threads/watchthreads.exp: watchpoint on args[0] hit in thread
>     FAIL: gdb.threads/watchthreads.exp: watchpoint on args[1] hit in thread
>     FAIL: gdb.threads/watchthreads.exp: combination of threaded watchpoints = 30

GDB doesn't support this feature.  We added the testcase but not the
rest of the patch.  As I reduce the list of failures on my first target
platform to almost none this is coming back to the top of my list.
I asked someone from Red Hat to submit an updated version but
got no response; I'll poke them again later.

-- 
Daniel Jacobowitz
CodeSourcery