From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id YBKIGZHQyWAEBgAAWB0awg (envelope-from ) for ; Wed, 16 Jun 2021 06:21:05 -0400 Received: by simark.ca (Postfix, from userid 112) id 64A611F163; Wed, 16 Jun 2021 06:21:05 -0400 (EDT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED, MAILING_LIST_MULTI,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id 0F5701E54D for ; Wed, 16 Jun 2021 06:21:05 -0400 (EDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id CA7CE39960D7 for ; Wed, 16 Jun 2021 10:21:04 +0000 (GMT) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by sourceware.org (Postfix) with ESMTPS id 0FF313982C1B for ; Wed, 16 Jun 2021 10:20:28 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 0FF313982C1B Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0C7711FD47; Wed, 16 Jun 2021 10:20:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1623838827; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iKkcqZOFmNXqpLyiGAgjx8gDJUXiTE+Do1TF+fXFJIU=; b=D/tsgMZuMCJpXP97SPgRXD7TXbdJmUeoPNA80ispBKwK9xamb+EfQDBzVN2wreXJgU0VUk gwA35NOivKgHrqpsI/Rv++n7GxpApd1uHFdTVEVe6yOdvxJ6L/97JVE9msIVjDrXGhRpz6 khnE/I9k5vy16gr1EgbH2sVyXCGlASk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1623838827; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iKkcqZOFmNXqpLyiGAgjx8gDJUXiTE+Do1TF+fXFJIU=; b=HIRhPzuT8AC1MZ29ow/U8HvGiHkYT2mZzo+IPAHJCcZ013d98ECEwWbvDf+PLnbE0Gos+t ACkIzsAbvtdX8UBQ== Received: from imap3-int (imap-alt.suse-dmz.suse.de [192.168.254.47]) by imap.suse.de (Postfix) with ESMTP id DA5CE118DD; Wed, 16 Jun 2021 10:20:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1623838827; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iKkcqZOFmNXqpLyiGAgjx8gDJUXiTE+Do1TF+fXFJIU=; b=D/tsgMZuMCJpXP97SPgRXD7TXbdJmUeoPNA80ispBKwK9xamb+EfQDBzVN2wreXJgU0VUk gwA35NOivKgHrqpsI/Rv++n7GxpApd1uHFdTVEVe6yOdvxJ6L/97JVE9msIVjDrXGhRpz6 khnE/I9k5vy16gr1EgbH2sVyXCGlASk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1623838827; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iKkcqZOFmNXqpLyiGAgjx8gDJUXiTE+Do1TF+fXFJIU=; b=HIRhPzuT8AC1MZ29ow/U8HvGiHkYT2mZzo+IPAHJCcZ013d98ECEwWbvDf+PLnbE0Gos+t ACkIzsAbvtdX8UBQ== Received: from director2.suse.de ([192.168.254.72]) by imap3-int with ESMTPSA id fYabM2rQyWBWSQAALh3uQQ (envelope-from ); Wed, 16 Jun 2021 10:20:26 +0000 Subject: Re: [RFC][gdb/symtab] Lazy expansion of full symbol table To: Tom Tromey References: <20210614093908.GA22709@delia> <87pmwoxj3j.fsf@tromey.com> <533bf7e4-d96c-a6b7-8c37-a4141ebdc761@suse.de> <87im2fxnr7.fsf@tromey.com> From: Tom de Vries Message-ID: Date: Wed, 16 Jun 2021 12:20:26 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: <87im2fxnr7.fsf@tromey.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gdb-patches@sourceware.org Errors-To: gdb-patches-bounces+public-inbox=simark.ca@sourceware.org Sender: "Gdb-patches" On 6/15/21 3:26 PM, Tom Tromey wrote: >>> I don't have this executable > > Tom> Uploaded to https://ftp.suse.com/pub/people/tdevries/gdb/pr23710/cc1 > > Thanks, I grabbed it. > >>> but FWIW my scanner rewrite is ~10x faster >>> than the current psymtab reader. > > Tom> Interesting, that's > Tom> https://github.com/tromey/gdb/commits/submit/no-more-psym ? > > Tom> I've tried that branch with the cc1 example, and ran into: > Tom> ... > Tom> DW_FORM_strp pointing outside of .debug_str section [in module cc1] > > Thank you for trying it. This was a bug in the CU importing code. I've > fixed it and will push momentarily -- but be warned, I rebase this > branch a lot... > Sure, understood. > On my machine, with "maint time 1", /bin/gdb reports: > > (gdb) file ./cc1 > 2021-06-15 07:18:29.554 - command started > Reading symbols from ./cc1... > 2021-06-15 07:18:35.118 - command finished > Command execution time: 5.553344 (cpu), 5.563964 (wall) > > The branch says: > > (gdb) file ./cc1 > 2021-06-15 07:18:50.688 - command started > Reading symbols from ./cc1... > 2021-06-15 07:18:51.074 - command finished > Command execution time: 1.260050 (cpu), 0.386460 (wall) > > So a bit more than 10x in this case. Wow, nice :) FWIW, my measurement is: 1. branch base 739025e89c4 ... $ gdb -q -ex "set trace-commands on" -ex "maint time 1" -ex "file lto/cc1" +maint time 1 +file lto/cc1 Reading symbols from lto/cc1... 2021-06-16 11:51:54.007 - command finished Startup time: 2.949213 (cpu), 2.887778 (wall) (gdb) ... 2. branch 5bc56d745fd ... $ gdb -q -ex "set trace-commands on" -ex "maint time 1" -ex "file lto/cc1" +maint time 1 +file lto/cc1 Reading symbols from lto/cc1... 2021-06-16 12:00:35.060 - command finished Startup time: 0.947957 (cpu), 0.408178 (wall) (gdb) ... Which is a factor 7 speedup. > I wish it were a tiny bit faster, > so that it wasn't perceptible, but meh, DWARF wasn't designed to be fast > to read. > > Currently I think the scanner is complete, I just need to make some > changes to the lookup code, and maybe also how the new index entries are > stored, so that completion is more efficient. Of course there may still > be bugs, but at the moment I don't anticipate any reader slowdowns. I did an overnight build and test with the updated branch (5bc56d745fd) and ran into some trouble. The first internal-error I investigated happens when parsing the libstdc++ .debug package (so, it was not specific to the test-case). It seems the branch has some trouble with the dwz layout where an abbrev entry is shared between different CUs: ... src/gdb/dwarf2/abbrev-cache.c:59: internal-error: void abbrev_cache::add(abbrev_table_up): Assertion `*slot == nullptr' failed. ... I also could reproduce this problem with "maint set worker-threads 1". I set a breakpoint on abbrev_cache::add and abbrev_cache::find, both limited to section offset 0. I see then a lot of finds, followed by two adds (and the second add triggers the assert). [ If you want to reproduce, download f.i. https://download.opensuse.org/update/leap/15.2/oss_debug/x86_64/libstdc++6-debuginfo-10.2.1+git583-lp152.4.1.x86_64.rpm . ] Thanks, - Tom