From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp.polymtl.ca (smtp.polymtl.ca [132.207.4.11]) by sourceware.org (Postfix) with ESMTPS id F23CB383E80D for ; Tue, 26 May 2020 20:38:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org F23CB383E80D Received: from simark.ca (simark.ca [158.69.221.121]) (authenticated bits=0) by smtp.polymtl.ca (8.14.7/8.14.7) with ESMTP id 04QKcpb8001856 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2020 16:38:56 -0400 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp.polymtl.ca 04QKcpb8001856 Received: from [172.16.0.95] (192-222-181-218.qc.cable.ebox.net [192.222.181.218]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPSA id 1670D1E792; Tue, 26 May 2020 16:38:50 -0400 (EDT) Subject: Re: How to run multiple target boards To: Tom de Vries , Pedro Alves , "gdb-patches@sourceware.org" References: <20200512130002.GA3290@delia> <73a5e27b-a709-8861-1f88-a07a0a2a71f9@polymtl.ca> <256a0db7-98c2-7769-7991-981f59106ee4@suse.de> <82395a9a-1595-e16c-8977-5f929f9041cc@polymtl.ca> <85005cf2-783d-7048-1cfa-8b648854c981@suse.de> <3319bd1f-a39f-1bf0-f291-e166edb2ab97@redhat.com> From: Simon Marchi Message-ID: Date: Tue, 26 May 2020 16:38:50 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: tl Content-Transfer-Encoding: 7bit X-Poly-FromMTA: (simark.ca [158.69.221.121]) at Tue, 26 May 2020 20:38:51 +0000 X-Spam-Status: No, score=-9.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 May 2020 20:39:00 -0000 On 2020-05-26 6:56 a.m., Tom de Vries wrote: > On 26-05-2020 12:39, Pedro Alves wrote: >> On 5/26/20 9:49 AM, Tom de Vries wrote: >>> [ was: Re: [committed][gdb/testsuite] Fix incorrect string concat in >>> jit-elf.exp ] >>> >>> On 13-05-2020 15:45, Simon Marchi wrote: >>>> FWIW, I just learned it's possible to pass a list of target boards to --target_board: >>>> >>>> --target_board='unix native-gdbserver native-extended-gdbserver' >>>> >>>> And it gives a combined result at the end. So I'll make myself an alias to test on all >>>> the commonly used boards, probably: >>>> >>>> - cc-with-debug-names.exp >>>> - cc-with-dwz.exp >>>> - cc-with-dwz-m.exp >>>> - cc-with-gdb-index.exp >>>> - debug-types.exp >>>> - dwarf4-gdb-index.exp >>>> - fission-dwp.exp >>>> - fission.exp >>>> - native-extended-gdbserver.exp >>>> - native-gdbserver.exp >>>> - native-stdio-gdbserver.exp >>>> - readnow.exp >> >> What are the advantages of doing this compared to a script that runs the >> testsuite for each of the boards, separately? >> > > I'd say: > - the target boards not overwriting the .log and .sum files of other > target boards > - the joint summary. > > Thanks, > - Tom > >>>> >>>> When working on a single test, it's usually not to long to run them all. Of course, it's >>>> not really practical to run the complete testsuite twice (before and after) for each board >>>> for every change we do... >>> >>> Hi Simon, >>> >>> I just tried this out. >>> >>> My test scripts use make check, but I didn't manage to make that work >>> yet, so I tried out using runtest directly (and I may be missing >>> something obvious, given that I haven't used this before): >>> ... >>> $ cd build/gdb/testsuite >>> $ runtest gdb.base/gold-gdb-index.exp --target_board='cc-with-gdb-index >>> unix' >>> ... >> >> One issue this with that I've run into in the past, is with global >> variables leaking from one board to the other. >> >> And I've just tried a quick test, and lucky me I seem to have run into >> something like that immediately: >> >> $ make check RUNTESTFLAGS="--target_board='native-gdbserver unix' break.exp" >> ... >> Schedule of variations: >> native-gdbserver >> unix >> ... >> FAIL: gdb.base/break.exp: run until function breakpoint >> FAIL: gdb.base/break.exp: run until breakpoint set at a line number (the program is no longer running) >> FAIL: gdb.base/break.exp: run until file:function(6) breakpoint (the program is no longer running) >> FAIL: gdb.base/break.exp: run until file:function(5) breakpoint (the program is no longer running) >> (snip a bunch more) >> >> (Running break.exp for each board in isolation passes cleanly, of course.) >> >> Another thing is that the test messages in gdb.sum don't indicate which >> board issued the PASS/FAIL (on each result), making it more difficult >> to analyze. Ok, so in retrospect, not such a good idea :(. Simon