From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id 3oDyF8Bgo2LXLAEAWB0awg (envelope-from ) for ; Fri, 10 Jun 2022 11:18:24 -0400 Received: by simark.ca (Postfix, from userid 112) id 4F92D1E223; Fri, 10 Jun 2022 11:18:24 -0400 (EDT) Authentication-Results: simark.ca; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=bEoMmggN; dkim-atps=neutral X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-3.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id B5FBC1E220 for ; Fri, 10 Jun 2022 11:18:23 -0400 (EDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C488D392AC39 for ; Fri, 10 Jun 2022 15:18:20 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C488D392AC39 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1654874300; bh=hftTxvATRlO3MzSEQFi4so8TjUF4vi/3yr0lw80oEUQ=; h=Subject:To:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=bEoMmggN08gNiJfSKcMmuDFoYEnHdx0cDfXdbUXu8fdG7SWzl45Cs27JoJLUDNnci 1/kTDBVGW+SaYK4+8VfZp4Jze6HU7uGWba1qkrr33EMtzFoor/XmMWthFJgQIA2GDB MSBg+1YDC20naf6J2HN27kS//MHFuT78vzqWaqoE= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by sourceware.org (Postfix) with ESMTPS id A63923851C24; Fri, 10 Jun 2022 15:17:54 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org A63923851C24 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 25ADtRXr000520; Fri, 10 Jun 2022 15:17:42 GMT Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3gm7931vam-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 10 Jun 2022 15:17:41 +0000 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 25AEpUM6007718; Fri, 10 Jun 2022 15:17:41 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma04dal.us.ibm.com with ESMTP id 3gfy1auubr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 10 Jun 2022 15:17:41 +0000 Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 25AFHd2o28639496 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 Jun 2022 15:17:40 GMT Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E4161C605A; Fri, 10 Jun 2022 15:17:39 +0000 (GMT) Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 33180C605D; Fri, 10 Jun 2022 15:17:39 +0000 (GMT) Received: from lexx (unknown [9.160.81.62]) by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP; Fri, 10 Jun 2022 15:17:39 +0000 (GMT) Message-ID: <259023ae926b29de7ef58de777ec1f43ee72e620.camel@vnet.ibm.com> Subject: Re: [builder] gdb_check_step: remove gdb.gdb/selftest.exp To: Luis Machado , Mark Wielaard , Carl Love Date: Fri, 10 Jun 2022 10:17:38 -0500 In-Reply-To: <85fa6e2d-caf5-8afe-a7ec-40cc62ff347a@arm.com> References: <20220608185138.675964-1-mark@klomp.org> <2281be8e-8920-7ba5-9e96-ffd713462b9c@arm.com> <8bfa7ec63758afc45bde1f10b0e4ab91e21e9d06.camel@klomp.org> <85fa6e2d-caf5-8afe-a7ec-40cc62ff347a@arm.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 (3.28.5-18.el8) X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: j999-EmqkzA9KCDCYrICxYQwx17Gly0u X-Proofpoint-GUID: j999-EmqkzA9KCDCYrICxYQwx17Gly0u Content-Transfer-Encoding: 7bit X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.517,FMLib:17.11.64.514 definitions=2022-06-10_06,2022-06-09_02,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 malwarescore=0 suspectscore=0 clxscore=1011 phishscore=0 adultscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501 spamscore=0 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2204290000 definitions=main-2206100060 X-BeenThere: gdb@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: will schmidt via Gdb Reply-To: will schmidt Cc: gdb@sourceware.org, buildbot@sourceware.org Errors-To: gdb-bounces+public-inbox=simark.ca@sourceware.org Sender: "Gdb" On Fri, 2022-06-10 at 11:58 +0100, Luis Machado wrote: > On 6/10/22 11:50, Mark Wielaard wrote: > > Hi, > > > > On Fri, 2022-06-10 at 01:21 +0200, Mark Wielaard wrote: > > > On Fri, Jun 10, 2022 at 01:09:19AM +0200, Mark Wielaard wrote: > > > > On Thu, Jun 09, 2022 at 10:37:58AM +0100, Luis Machado wrote: > > > > > I always use gdb.base/break.exp as a good smoke test. If that > > > > > one > > > > > fails, then things > > > > > are really broken. > > > > > > > > > > I think gdb.base/break*.exp should make a good smoke test > > > > > list. > > > > > We just need to exclude > > > > > gdb.base/break-interp.exp, which is problematic on some > > > > > targets. > > > > > > > > It never is just easy is it? :) You are right, I saw break- > > > > interp.exp > > > > fail... I tried to come up with a regexp but gave up given > > > > that it > > > > has to go throug python first and then we don't know whether > > > > the > > > > worker uses bash as /bin/sh so I just added them all (exclusing > > > > break-interp.exp) as a list. > > > > > > Sigh, sorry, looks like gdb.base/break-unload-file.exp also > > > sometimes > > > fails. > > > I have removed from the list. Hopefully the remaining list does > > > actually pass. > > > > And it didn't :{ > > > > Yeah. As expected, the GDB testsuite is a bit delicate when you start > dealing with > multiple architectures and modes. But I think this is good progress > already. > > > Both debian-ppc64 and fedora-ppc64le failed (UNRESOLVED) > > gdb.base/break-idempotent.exp under both native-gdbserver and > > native- > > extended-gdbserver > > https://builder.sourceware.org/buildbot/#builders/76/builds/446 > > https://builder.sourceware.org/buildbot/#builders/85/builds/294 > > Those might be genuine issues. I'm cc-ing Carl and Will so they can > chime in. RROR: no fileid for debian-ppc64-builder UNRESOLVED: gdb.base/break-idempotent.exp: pie=nopie: always_inserted=off: watch: gdb_breakpoint: set breakpoint at main (timeout) I've looked briefly at the logs. I recall one of those tests having a FAIL with one of the test permutations, but not an UNRESOLVED. We'll need to poke further. has_hw_wp_support: Default, hardware watchpoint not deteced I see a typo in there somewhere... :-) > > > So I have removed that one too from the list. > > The CI test list now looks like: > > > > # Only a small subset of tests that are fast and known to PASS. > > gdb_test_exp = ("TESTS= " > > "gdb.base/break-always.exp " > > "gdb.base/break-caller-line.exp " > > "gdb.base/break-entry.exp " > > "gdb.base/break.exp " > > "gdb.base/break-fun-addr.exp " > > "gdb.base/break-include.exp " > > "gdb.base/break-inline.exp " > > "gdb.base/break-main-file-remove-fail.exp " > > "gdb.base/break-on-linker-gcd-function.exp " > > "gdb.base/breakpoint-in-ro-region.exp " > > "gdb.base/breakpoint-shadow.exp " > > "gdb.base/break-probes.exp " > > "gdb.gdb/unittest.exp " > > "gdb.server/unittest.exp ") > > > > Which will be run three times with make gdb-check, once without a > > target_board, once with native-gdbserver and once with native- > > extended- > > gdbserver on centos-x86_64, fedora-x86_64, debian-armhf, debian- > > arm64, > > fedora-s390x, debian-ppc64, fedora-ppc64le, opensusetw-x86_64, > > opensuseleap-x86_64 (debian-armhf only does a build, no make gdb- > > check > > because of https://sourceware.org/bugzilla/show_bug.cgi?id=28561) > > I'll add a debian-i386 builder so there is more 32bit coverage. > > > > All are green now (with the latest change to remove break- > > idempotent) > > https://builder.sourceware.org/buildbot/#/builders?tags=gdb > > Looks nice! :-) > > > Question is if this is a good list, does it need more tests? And > > should > > it maybe be maintained in the binutils-gdb repo instead of in the > > builder repo? > > > > For example we could have a make check-gdb-ci target which does > > what > > the buildbot would do (and then the buildbot could just call that). > > Having a new check-gdb target that only does minimal smoke tests > should > be easy to do. Once we determine a subset of critical tests, we can > put something > together if folks think it is a good idea. I like it, as it make it > easier to deal with > stability issues of GDB's testsuite across different targets. > > > Cheers, > > > > Mark