Re: Testsuite status/proper configure?

Re: Testsuite status/proper configure?

[Michael Elizabeth Chastain]

Your test process looks okay to me.

These are similar to my test results.  I run the test suite each weekend
week and post the results to:

  http://sources.redhat.com/ml/gdb-testers

Nitpick #1: dejagnu does not report the count of ERRORs and WARNINGs.
I have to count these by hand:

  grep -c ERROR gdb.sum
  grep -c WARNING gdb.sum

The triple "# ERRORs, # WARNINGs, # FAILs" is more useful than "# FAILs",
because if something goes wrong, the results can have a lot of ERRORs
and WARNINGs with fewer FAILs.

Nitpick #2: it helps to know which compiler you ran the test suite with.
gdb should behave the same no matter what compiler that it's built
with.  But the debug info in the programs-under-test depends a lot on
what compiler it's built with and the version of that compiler.  In
particular, there's a big difference between v2 gcc and v3 gcc.
(I use a v3 gcc when I test).

Basically your test process is good and those numbers are real.

MichaelC

Re: Testsuite status/proper configure?

[Andreas Jaeger]

Michael Elizabeth Chastain <chastain@cygnus.com> writes:

> Your test process looks okay to me.
> 
> These are similar to my test results.  I run the test suite each weekend
> week and post the results to:
> 
>   http://sources.redhat.com/ml/gdb-testers

Thanks.  Now I have confidence to my results and will post them in the
proper way (with all needed info) next time.  The number of FAILS
suprised me.

> Nitpick #1: dejagnu does not report the count of ERRORs and WARNINGs.
> I have to count these by hand:
> 
>   grep -c ERROR gdb.sum
>   grep -c WARNING gdb.sum
> The triple "# ERRORs, # WARNINGs, # FAILs" is more useful than "# FAILs",
> because if something goes wrong, the results can have a lot of ERRORs
> and WARNINGs with fewer FAILs.

I'll add these next time.

> Nitpick #2: it helps to know which compiler you ran the test suite with.
> gdb should behave the same no matter what compiler that it's built
> with.  But the debug info in the programs-under-test depends a lot on
> what compiler it's built with and the version of that compiler.  In
> particular, there's a big difference between v2 gcc and v3 gcc.
> (I use a v3 gcc when I test).

I've used gcc 2.95.3.

> Basically your test process is good and those numbers are real.

Thanks for the confirmation,
Andreas
-- 
 Andreas Jaeger
  SuSE Labs aj@suse.de
   private aj@arthur.inka.de
    http://www.suse.de/~aj

Re: Testsuite status/proper configure?

[Mark Kettenis]

Andreas Jaeger <aj@suse.de> writes:

> I've used gcc 2.95.3.

Just FYI, I usually had just under 30 unexpected fails on
i586-pc-linux-gnu.  It has just gone up to about 40 due to failures in
the newly added mi testsuite.

That's with GCC 2.95.2.

Mark

Testsuite status/proper configure?

[Andreas Jaeger]

I'm a little bit puzzled.   I checked out the full sourceware source
tree, run 
/cvs/sourceware-src/configure --prefix=/opt/gdb --disable-shared --enable-threads \
         --with-gnu-as --with-gnu-ld \
         --enable-build-warnings --disable-nls --disable-tui --disable-gdbtk

and then make all-gdb to build gdb on i686-linux-gnu (with glibc 2.2.2).

Running the testsuite with make check-gdb gave me lots of failures:
                === gdb Summary ===

# of expected passes            7515
# of unexpected failures        145
# of unexpected successes       32
# of expected failures          192
# of unresolved testcases       120
/builds/sourceware-src/build-gdb/gdb/testsuite/../../gdb/gdb version  2001-07-08-cvs -nx

Have I done something wrong?  Is there a better method for building
only gdb from the complete sourceware tree?

Andreas
-- 
 Andreas Jaeger
  SuSE Labs aj@suse.de
   private aj@arthur.inka.de
    http://www.suse.de/~aj