From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 7002 invoked by alias); 25 May 2010 22:03:51 -0000 Received: (qmail 6941 invoked by uid 22791); 25 May 2010 22:03:50 -0000 X-SWARE-Spam-Status: No, hits=-5.8 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_HI,SPF_HELO_PASS,T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Tue, 25 May 2010 22:03:45 +0000 Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o4PM3iUd019130 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Tue, 25 May 2010 18:03:44 -0400 Received: from ns3.rdu.redhat.com (ns3.rdu.redhat.com [10.11.255.199]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o4PM3hKF029320; Tue, 25 May 2010 18:03:44 -0400 Received: from opsy.redhat.com (ovpn01.gateway.prod.ext.phx2.redhat.com [10.5.9.1]) by ns3.rdu.redhat.com (8.13.8/8.13.8) with ESMTP id o4PM3g2l012543; Tue, 25 May 2010 18:03:43 -0400 Received: by opsy.redhat.com (Postfix, from userid 500) id 7C1073782C5; Tue, 25 May 2010 16:03:42 -0600 (MDT) From: Tom Tromey To: Jan Kratochvil Cc: gdb-patches@sourceware.org Subject: Re: performance talk [Re: RFC: implement DW_OP_bit_piece] References: <20100520210135.GA31238@host0.dyn.jankratochvil.net> <20100525174253.GA12968@host0.dyn.jankratochvil.net> Reply-To: Tom Tromey Date: Tue, 25 May 2010 22:23:00 -0000 In-Reply-To: <20100525174253.GA12968@host0.dyn.jankratochvil.net> (Jan Kratochvil's message of "Tue, 25 May 2010 19:42:53 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org X-SW-Source: 2010-05/txt/msg00584.txt.bz2 >>>>> "Jan" == Jan Kratochvil writes: Tom> If you know of specific bad cases, I'm very interested in that. Tom> I think we understand the symbol reading problem pretty well now, but Tom> other stuff could at least use bug reports. Jan> Checked now we were talking about `attach' being fast enough (~2s Jan> on OOo.org) but I was testing it with Jan> `archer-tromey-delayed-symfile2'. Then `bt full' is still slow - Jan> tens of secs. Yeah, this is known. See below. At this point I'm much more interested in performance problems elsewhere in gdb. This particular one is, while not "solved", at least very well explored. A solution is on the horizon... Here's a dump of the status: This particular branch uses the aranges section to lazily read DWARF. This makes "attach" very fast, because we don't hit the disk very much. However, "bt" and "bt full" are still quite slow. If you want the bt output for OO.o you can actually see it pause for a noticeable amount of time at one particular point. What is happening here is that at this point we need some particular piece of debuginfo (I think a type, but I am not sure I looked into the details). Since we don't have any sort of by-name index, we have to read psymtabs and then search. The fix for this is to add a by-name map. You might think we could use debug_pubnames and debug_pubtypes, but those are broken beyond repair for various reasons: historical gcc bugs make them unreliable, they don't mention everything gdb needs, they aren't canonicalized, and what's worse, it turns out that actually reading them in is too slow. This is where the mmap()able debug index comes in. This approach yields good attach times *and* good backtrace times, at the cost of having to build a separate index per objfile. If you want to try this, it is all on archer-tromey-optional-psymtab. I can tell you how to make the index files if you do this. What makes this fast is that the by-name index means we can just read the CUs we care about. Reading any particular CU is fast enough that we don't need anything smarter. I am planning to propose this as an F14 feature: make index files for all the debuginfo, and at the same time drop .debug_pub* (to save some space). I figured I would delay submission to FSF gdb until we get this underway; my view is that this is more of a "distro" feature (creating the index files is just as slow as simply starting gdb) and there is no point in pushing it if the distro isn't interested. Another idea I implemented is to read psymtabs in the background. This works pretty well: gdb starts some worker threads and reads in psymtabs starting with the largest one (the smaller the debuginfo, the less likely you are to notice the pause). I haven't submitted this for two reasons. First, it is not quite complete. It has a questionable BFD hack, and I haven't yet made the complaint() system threadsafe. Second, I assume it will be contentious and/or too ugly. It uses __thread in various spots and it changes cp-name-parser.y to depend on bison (by making it a pure parser), not to mention that threads are widely hated. I'd be happy to revisit this if people are interested. Note that it is not as good as the index approach. These two approaches are actually complementary. We could put them both in. Finally, one or both of these branches has a little optimization so that if you "gdb -readnow", we don't create psymtabs at all. This could go in now; I just haven't bothered since I think -readnow is only ever actually used to locate psymtab expansion bugs. Tom