From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 5045 invoked by alias); 6 Apr 2009 20:22:48 -0000 Received: (qmail 5037 invoked by uid 22791); 6 Apr 2009 20:22:47 -0000 X-SWARE-Spam-Status: No, hits=-2.5 required=5.0 tests=AWL,BAYES_00 X-Spam-Check-By: sourceware.org Received: from sibelius.xs4all.nl (HELO sibelius.xs4all.nl) (82.92.89.47) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Mon, 06 Apr 2009 20:22:40 +0000 Received: from brahms.sibelius.xs4all.nl (kettenis@localhost.sibelius.xs4all.nl [127.0.0.1]) by brahms.sibelius.xs4all.nl (8.14.3/8.14.3) with ESMTP id n36KMEpC005873; Mon, 6 Apr 2009 22:22:15 +0200 (CEST) Received: (from kettenis@localhost) by brahms.sibelius.xs4all.nl (8.14.3/8.14.3/Submit) id n36KMD5u016711; Mon, 6 Apr 2009 22:22:13 +0200 (CEST) Date: Mon, 06 Apr 2009 20:22:00 -0000 Message-Id: <200904062022.n36KMD5u016711@brahms.sibelius.xs4all.nl> From: Mark Kettenis To: tromey@redhat.com CC: bauerman@br.ibm.com, ppluzhnikov@google.com, gdb-patches@sourceware.org In-reply-to: (message from Tom Tromey on Mon, 06 Apr 2009 13:50:48 -0600) Subject: Re: relying on testsuite results References: <200901121846.51709.pedro@codesourcery.com> <200902062335.17737.pedro@codesourcery.com> <8ac60eac0902082223q2192830cu8b75f6424fca6c68@mail.gmail.com> <200902092216.54762.pedro@codesourcery.com> <8ac60eac0904061159v4deb9d48n1cf791463e587e54@mail.gmail.com> <1239045480.8871.11.camel@localhost.localdomain> Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org X-SW-Source: 2009-04/txt/msg00112.txt.bz2 > From: Tom Tromey > Reply-To: tromey@redhat.com > > I generally do full regression tests for each patch. ("Generally" > because I occasionally forget, or something goes wrong and I don't > notice.) This is nice, although slow, and unfortunately also open to > some kinds of major failure (e.g., a change of compiler negatively > affecting baseline results). And there you touch upon an important reason why it is so hard to make all tests PASS. In many cases our testsuite isn't just testing GDB in isolation, but also the compiler, kernel or threads library. Unfortunately the developers of those components don't run the GDB testsuite as part of their regression testing. So things are sometimes broken and take a while to get fixed again. That said, on my preferred platform (OpenBSD) the testsuite produces fairly consistent results. We use a stable toolchain (GCC 3.3.5, binutils 2.15) which helps. And a fair number of the threads-related tests FAIL because of some Linuxisms that crept, which means they often dom't even get to the racy bits. But it is also because in the past I spent time on actually fixing broken tests.