From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 14260 invoked by alias); 19 Sep 2013 00:36:02 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Received: (qmail 14246 invoked by uid 89); 19 Sep 2013 00:36:02 -0000 Received: from mail-ve0-f202.google.com (HELO mail-ve0-f202.google.com) (209.85.128.202) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-SHA encrypted) ESMTPS; Thu, 19 Sep 2013 00:36:02 +0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=2.7 required=5.0 tests=AWL,BAYES_20,KHOP_THREADED,RP_MATCHES_RCVD,SPAM_URI,SPF_SOFTFAIL autolearn=no version=3.3.2 X-HELO: mail-ve0-f202.google.com Received: by mail-ve0-f202.google.com with SMTP id jy13so941962veb.3 for ; Wed, 18 Sep 2013 17:35:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:mime-version:content-type :content-transfer-encoding:message-id:date:to:cc:subject:in-reply-to :references; bh=oSrcWjXtuCUp2tngThB1hRuXneQKSXSWHs5ZETXCcOo=; b=VIT6APEmfifHA6Grsfc1SqVnbKKqT2etagrJYxJUN244jaIZmxXX6+KrqOzekQWVN8 WiFhjCb3hpAb296Oqz56gBV/BpTDUJd3VjP0qKOnDBTer/fTE6qicu5GTcBGj8joOrev Qjn3PQS6D8prw6xBkQMeSU7V8KWmRKXEcSv+0x3ij3EPwdOAyoPgLKjjG6kMwp61TPO7 f8LGVOKW4wG7kfdBtp7tFFaVaFW7VkGdTY2S1muUucKUyrPzS+AU2XQpXoMrOxl3g31L 25z/LIKBRKxC6/OKOb9yUex13V/ftBSI0cCxE4Kx8jVhaOvXOydUtW+T9q3Uv2fb7kYK 57NA== X-Gm-Message-State: ALoCoQkxoqrpy2tdHLhqTZMLl+eC6CZ4dTpkPxGjjtlm+34qXaMBJe2Pqg6vNjeqzKIO39NepKje4xH6/EAL5BlmsFPd6XI8GUvwK3B/CGoXOHk5t5jE+Y6Pi/2EAZAoBlads6O8Cq5GFk82PrmlJ8TugsxGRXXWzZpkue4/tZ0t4RNYdnVrtGK0fWDbkivklAFvPEq/JZJO4FFgp52h+00Dveta8tJgug== X-Received: by 10.236.163.67 with SMTP id z43mr15448608yhk.27.1379550958613; Wed, 18 Sep 2013 17:35:58 -0700 (PDT) Received: from corp2gmr1-1.hot.corp.google.com (corp2gmr1-1.hot.corp.google.com [172.24.189.92]) by gmr-mx.google.com with ESMTPS id s21si744437yhc.1.1969.12.31.16.00.00 (version=TLSv1.1 cipher=AES128-SHA bits=128/128); Wed, 18 Sep 2013 17:35:58 -0700 (PDT) Received: from ruffy.mtv.corp.google.com (ruffy.mtv.corp.google.com [172.17.128.44]) by corp2gmr1-1.hot.corp.google.com (Postfix) with ESMTP id E5A5A31C078; Wed, 18 Sep 2013 17:35:57 -0700 (PDT) From: Doug Evans MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <21050.18157.372594.328077@ruffy.mtv.corp.google.com> Date: Thu, 19 Sep 2013 00:36:00 -0000 To: Yao Qi Cc: "Agovic, Sanimir" , "gdb-patches@sourceware.org" Subject: Re: [RFC] GDB performance testing infrastructure In-Reply-To: <521D6862.6050100@codesourcery.com> References: <520B7F70.6070207@codesourcery.com> <0377C58828D86C4588AEEC42FC3B85A71764E92B@IRSMSX105.ger.corp.intel.com> <521D6862.6050100@codesourcery.com> X-IsSubscribed: yes X-SW-Source: 2013-09/txt/msg00638.txt.bz2 Yao Qi writes: > On 08/27/2013 09:49 PM, Agovic, Sanimir wrote: > >> * Remote debugging. It is slower to read from the remote target, and > >> > worse, GDB reads the same memory regions in multiple times, or reads > >> > the consecutive memory by multiple packets. > >> > > > Once gdb and gdbserver share most of the target code, the overhead will be > > caused by the serial protocol roundtrips. But this will take a while... > > Sanimir, thanks for your comments! > > One of the motivations of the performance testing is to measure the > overhead of RSP in some scenarios, and look for the opportunities to > improve it, or add a completely new protocol, which is an extreme case. For reference sake, a big part of the "reading same memory region multiple times" and "consecutive memory by multiple packets" is gdb's inability to use its dcache (apropos dcache) for text segments. Blech. > Once the infrastructure is ready, we can write some tests to see how > efficient or inefficient RSP is. "set debug remote 1" and you're there. 1/2 :-) "But seriously ..." Latency can be a huge problem with any remote protocol. Running gdb+gdbserver on the same machine can hide issues without tracking, e.g., packet counts in addition to cpu/wall time. [*both* cpu and wall time are useful] I hope the test harness will incorporate this. > >> > * Tracepoint. Tracepoint is designed to be efficient on collecting > >> > data in the inferior, so we need performance tests to guarantee that > >> > tracepoint is still efficient enough. Note that we a test > >> > `gdb.trace/tspeed.exp', but there are still some rooms to improve. > >> > > > Afaik the tracepoint functionality is quite separated from gdb may be tested > > in isolation. Having a generic benchmark framework covering the most parts of > > gdb is probably_the_ way to go but I see some room for specialized benchmarks > > e.g. for tracepoints with a custom driver. But my knowledge is too vague on > > the topic. > > > > Well, it is sort of design trade-off. We need a framework generic > enough to handle most of the testing requirements for different GDB > modules, (such as solib, symbols, backtrace, disassemble, etc), on the > other hand, we want each test is specialized for the corresponding GDB > module, so that we can find more details. > > I am inclined to handle testing to _all_ modules under this generic > framework. Agreed. > >> > 2. Detect performance regressions. We collected the performance data > >> > of each micro-benchmark, and we need to detect or identify the > >> > performance regression by comparing with the previous run. It is > >> > more powerful to associate it with continuous testing. > >> > > > Something really simple, so simple that one could run it silently with every > > make invokation. For a newcomer, it took me some time to get used to make > > check e.g. setup, run, and interpret the tests with various settings. Something > > simpler would help to run it more often. > > > > Yes, I agree, everything should be simple. I assume that people > running performance testing should be familiar with GDB regular > regression test, like 'make check'. We'll provide 'make check-perf' to > run performance testing ,and it doesn't add extra difficulties on top of > 'make check', from user's point of view, IMO. > > > I like to add the Machine Interface (MI) to the list, but it is quite rudimentary: > > > > $ gdb -interpreter mi -q debugee > > [...] > > -enable-timings > > ^done > > (gdb) > > -break-insert -f main > > ^done,bkpt={...},time={wallclock="0.00656",user="0.00000",system="0.00000"} > > [...] > > (gdb) > > -exec-step > > ^running > > *running,thread-id="1" > > (gdb) > > *stopped,[...],time={wallclock="0.19425",user="0.09700",system="0.04200"} > > (gdb) > > > > With -enable-timings[1] enabled, every result record has a time triple > > appended, even for async[2] ones. If we come up with a full mi parser > > one could run tests w/o timings. A mi result is quite json-ish. > > Thanks for the input. > > > > > (To be honest I do not know how timings are composed of =D) > > > > In addition there are some tools for plotting benchmark results[3]. > > > > [1]http://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI-Miscellaneous-Commands.html > > [2]https://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI-Async-Records.html > > [3]http://speed.pypy.org/ > > I am using speed to track and show the performance data I got from the > GDB performance tests. It is able to associate the performance data to > the commit, so easy to find which commit causes regressions. However, > my impression is that speed or its dependent packages are not > well-maintained nowadays. > > After some search online, I like the chromium performance test and its > plot, personally. It is integrated with buildbot (a customized version). > > http://build.chromium.org/f/chromium/perf/dashboard/overview.html > > However, as I said in this proposal, let us focus on goal #1 first, get > the framework ready and collect performance data. Agreed. Let's get a good framework in place.