From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 30811 invoked by alias); 27 Aug 2012 15:57:34 -0000 Received: (qmail 30801 invoked by uid 22791); 27 Aug 2012 15:57:33 -0000 X-SWARE-Spam-Status: No, hits=-6.4 required=5.0 tests=AWL,BAYES_00,KHOP_RCVD_UNTRUST,RCVD_IN_DNSWL_HI,RCVD_IN_HOSTKARMA_W,RP_MATCHES_RCVD,SPF_HELO_PASS X-Spam-Check-By: sourceware.org Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Mon, 27 Aug 2012 15:57:16 +0000 Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q7RFvFc5028805 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Mon, 27 Aug 2012 11:57:15 -0400 Received: from host2.jankratochvil.net (ovpn-116-33.ams2.redhat.com [10.36.116.33]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id q7RFvBcO005563 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 27 Aug 2012 11:57:13 -0400 Date: Mon, 27 Aug 2012 15:57:00 -0000 From: Jan Kratochvil To: Yao Qi Cc: Pedro Alves , gdb-patches@sourceware.org Subject: Re: [PATCH 2/2] Remove pass in skip_unwinder_tests Message-ID: <20120827155710.GA11061@host2.jankratochvil.net> References: <877gt1zbr5.fsf@fleche.redhat.com> <1345715389-20955-1-git-send-email-yao@codesourcery.com> <1345715389-20955-2-git-send-email-yao@codesourcery.com> <20120824133738.GB5219@host2.jankratochvil.net> <5037A087.1090703@redhat.com> <20120824161854.GA10953@host2.jankratochvil.net> <503B4CF7.5090509@codesourcery.com> <20120827130641.GA5592@host2.jankratochvil.net> <503B8EE3.6040906@codesourcery.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <503B8EE3.6040906@codesourcery.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-IsSubscribed: yes Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org X-SW-Source: 2012-08/txt/msg00802.txt.bz2 On Mon, 27 Aug 2012 17:14:43 +0200, Yao Qi wrote: > XFAIL can be used for "unsupported system > feature" in gdb test, but XFAIL (nor PASS) can not be used in > checking the feature of system, because checking itself is not a > test. I'd like to call them (proc skip_XXX and proc support_XXX in > lib/gdb.exp) 'test configuration checking' which determines the set > of tests to run according to the configuration of GDB and other > factors. They should not contribute PASS/XFAIL/FAIL/... to test > result, because they are not tests. OK, this is a different opinion on what is a test and what is not a test. If the gdb.sum should contain only the output of the goals of the testfile then there should be only IIUC for example for gdb.trace/trace-mt.exp: PASS: gdb.trace/trace-mt.exp: trace on: tfind frame 0 PASS: gdb.trace/trace-mt.exp: trace on: tfind frame 1 PASS: gdb.trace/trace-mt.exp: trace on: tfind PASS: gdb.trace/trace-mt.exp: trace off: tfind frame 0 PASS: gdb.trace/trace-mt.exp: trace off: tfind frame 1 PASS: gdb.trace/trace-mt.exp: trace off: tfind PASS: gdb.trace/trace-mt.exp: step over trace: tstop PASS: gdb.trace/trace-mt.exp: ftrace on: tfind frame 0 PASS: gdb.trace/trace-mt.exp: ftrace on: tfind frame 1 PASS: gdb.trace/trace-mt.exp: ftrace on: tfind PASS: gdb.trace/trace-mt.exp: ftrace off: tfind frame 0 PASS: gdb.trace/trace-mt.exp: ftrace off: tfind frame 1 PASS: gdb.trace/trace-mt.exp: ftrace off: tfind PASS: gdb.trace/trace-mt.exp: step over ftrace: tstop as all the other testcases (43-14==29; see below the full testfile run) are only supporting there the goals above and they are no real tests on their own. Like the testing of system features are only supporting the goal testcases of the testfile. I see the testcases as a separation of the unstructured GDB output. As the matching is done only by regexes the responses are commonly out-of-sync leading to various racy bugs (like PR testsuite/12649). Some common markers (testcases) make the matching easier separating out racy results from real regressions. Dropping results of all the supportive testcases I find both a lot of work on the testsuite, IMO it is not the style of the current testsuite, and separating out the daily racy results out of it will be even more difficult than it is. TBH I do not mind at all, any decision will be always wrong as the primary problem is that the testsuite does not use structured GDB output with common parser of the structured output (with current GDB it means GDB does not use MI for very every command and there is no MI protocol parser in the testsuite). Thanks, Jan PASS: gdb.trace/trace-mt.exp: successfully compiled posix threads test case PASS: gdb.trace/trace-mt.exp: trace on: set breakpoint always-inserted on PASS: gdb.trace/trace-mt.exp: trace on: break end PASS: gdb.trace/trace-mt.exp: trace on: break set_point1 PASS: gdb.trace/trace-mt.exp: trace on: trace set_point1 PASS: gdb.trace/trace-mt.exp: trace on: tstart PASS: gdb.trace/trace-mt.exp: trace on: continue to set_point1 1 PASS: gdb.trace/trace-mt.exp: trace on: continue to set_point1 2 PASS: gdb.trace/trace-mt.exp: trace on: continue to end PASS: gdb.trace/trace-mt.exp: trace on: tstop PASS: gdb.trace/trace-mt.exp: trace on: tfind frame 0 PASS: gdb.trace/trace-mt.exp: trace on: tfind frame 1 PASS: gdb.trace/trace-mt.exp: trace on: tfind PASS: gdb.trace/trace-mt.exp: trace off: set breakpoint always-inserted off PASS: gdb.trace/trace-mt.exp: trace off: break end PASS: gdb.trace/trace-mt.exp: trace off: break set_point1 PASS: gdb.trace/trace-mt.exp: trace off: trace set_point1 PASS: gdb.trace/trace-mt.exp: trace off: tstart PASS: gdb.trace/trace-mt.exp: trace off: continue to set_point1 1 PASS: gdb.trace/trace-mt.exp: trace off: continue to set_point1 2 PASS: gdb.trace/trace-mt.exp: trace off: continue to end PASS: gdb.trace/trace-mt.exp: trace off: tstop PASS: gdb.trace/trace-mt.exp: trace off: tfind frame 0 PASS: gdb.trace/trace-mt.exp: trace off: tfind frame 1 PASS: gdb.trace/trace-mt.exp: trace off: tfind PASS: gdb.trace/trace-mt.exp: step over trace: set non-stop 0 PASS: gdb.trace/trace-mt.exp: step over trace: break set_point1 PASS: gdb.trace/trace-mt.exp: step over trace: continue to set_point1 PASS: gdb.trace/trace-mt.exp: step over trace: trace *$pc PASS: gdb.trace/trace-mt.exp: step over trace: tstart PASS: gdb.trace/trace-mt.exp: step over trace: stepi PASS: gdb.trace/trace-mt.exp: step over trace: tstop PASS: gdb.trace/trace-mt.exp: successfully compiled posix threads test case PASS: gdb.trace/trace-mt.exp: IPA loaded PASS: gdb.trace/trace-mt.exp: ftrace on: set breakpoint always-inserted on PASS: gdb.trace/trace-mt.exp: ftrace on: break end PASS: gdb.trace/trace-mt.exp: ftrace on: break set_point1 PASS: gdb.trace/trace-mt.exp: ftrace on: ftrace set_point1 PASS: gdb.trace/trace-mt.exp: ftrace on: tstart PASS: gdb.trace/trace-mt.exp: ftrace on: continue to set_point1 1 PASS: gdb.trace/trace-mt.exp: ftrace on: continue to set_point1 2 PASS: gdb.trace/trace-mt.exp: ftrace on: continue to end PASS: gdb.trace/trace-mt.exp: ftrace on: tstop