From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 30178 invoked by alias); 8 Aug 2018 15:24:49 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Received: (qmail 30077 invoked by uid 89); 8 Aug 2018 15:24:45 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-2.8 required=5.0 tests=AWL,BAYES_00,SPF_HELO_PASS autolearn=ham version=3.3.2 spammy= X-HELO: mx1.redhat.com Received: from mx3-rdu2.redhat.com (HELO mx1.redhat.com) (66.187.233.73) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 08 Aug 2018 15:24:41 +0000 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4FA517A7E7; Wed, 8 Aug 2018 15:24:40 +0000 (UTC) Received: from [127.0.0.1] (ovpn04.gateway.prod.ext.ams2.redhat.com [10.39.146.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3FE972166BA0; Wed, 8 Aug 2018 15:24:39 +0000 (UTC) Subject: Re: Update dg-extract-results.* from gcc To: Rainer Orth References: <515bfbff-e2f7-3574-2726-ee1592786891@redhat.com> Cc: gdb-patches@sourceware.org From: Pedro Alves Message-ID: <400a6540-c7fb-8de8-d5dc-3f07e2d632ca@redhat.com> Date: Wed, 08 Aug 2018 15:24:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-SW-Source: 2018-08/txt/msg00161.txt.bz2 On 08/08/2018 03:36 PM, Rainer Orth wrote: >> On 07/20/2018 12:02 PM, Rainer Orth wrote: >>> When looking at the gdb.sum file produced by dg-extract-results.sh on >>> Solaris 11/x86, I noticed some wrong sorting, like this: >>> >>> PASS: gdb.ada/addr_arith.exp: print something'address + 0 >>> PASS: gdb.ada/addr_arith.exp: print 0 + something'address >>> PASS: gdb.ada/addr_arith.exp: print something'address - 0 >>> PASS: gdb.ada/addr_arith.exp: print 0 - something'address >>> >>> Looking closer, I noticed that while dg-extract-results.sh had been >>> copied over from contrib in the gcc repo, the corresponding >>> dg-extract-results.py file had not. The latter not only fixes the >>> sorting problem I'd observed, but is also way faster than the shell >>> version (like a factor of 50 faster). >> >> We used to have the dg-extract-results.py file, but we deleted it >> because it caused (funnily enough, sorting) problems. See: >> >> https://sourceware.org/ml/gdb-patches/2015-02/msg00333.html >> >> Has that sorting stability issue been meanwhile fixed upstream? > > not that I can see: between the version of dg-extract-results.py removed > in early 2015 and the one in current gcc trunk, there's only added > handling for DejaGnu ERRORs and another minor change to do with > summaries that doesn't seem to change anything wrt. sorting on first > blush. OK. > Howver, I've just run make -j16 check three times in a row on > amd64-pc-solaris2.11, followed by make -j48 check, and the only > differences were to to 200+ racy tests, the vast majority of them in > gdb.threads. Thanks for testing. > Maybe the prior problems have been due to bugs in older > versions of python? Might be. IIRC, the sorting that would change would be the order that the different individual gdb.sum results would be merged (the per-gdb.sum order was stable). So comparing two runs, you'd get something like, in one run this: gdb.base/foo.exp:PASS: test1 gdb.base/foo.exp:PASS: test2 gdb.base/foo.exp:PASS: test3 gdb.base/bar.exp:PASS: testA gdb.base/bar.exp:PASS: testB gdb.base/bar.exp:PASS: testC and another run this: gdb.base/bar.exp:PASS: testA gdb.base/bar.exp:PASS: testB gdb.base/bar.exp:PASS: testC gdb.base/foo.exp:PASS: test1 gdb.base/foo.exp:PASS: test2 gdb.base/foo.exp:PASS: test3 which would result in all those tests spuriously showing up in a gdb.sum old/new diff. I'm not sure whether we were seeing that if you compared runs of the same tree multiple times. It could be that it only happened when comparing the results of different trees, which contained a slightly different set of tests and testcases, like for example comparing testresults of a patched master against the testresults of master from a month or week ago, which is something I frequently do, for example. *time passes* Wait wait wait, can you clarify what you meant by wrong sorting in: PASS: gdb.ada/addr_arith.exp: print something'address + 0 PASS: gdb.ada/addr_arith.exp: print 0 + something'address PASS: gdb.ada/addr_arith.exp: print something'address - 0 PASS: gdb.ada/addr_arith.exp: print 0 - something'address ? Why do you think those results _should_ be sorted? And in what order? Typically, the order/sequence in which the tests of a given exp file is executed is important. The order in the gdb.sum file must be the order in which the fail/pass calls are written/issued in the .exp file. It'd be absolutely incorrect to alphabetically sort the gdb.sum output. Is that what the .py version does? That's not what I recall, though. I guess I may be confused. Thanks, Pedro Alves