From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id qJgQIh/ox2CqVAAAWB0awg (envelope-from ) for ; Mon, 14 Jun 2021 19:37:03 -0400 Received: by simark.ca (Postfix, from userid 112) id 899AC1F163; Mon, 14 Jun 2021 19:37:03 -0400 (EDT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED, MAILING_LIST_MULTI,T_DKIM_INVALID,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id 76B451E54D for ; Mon, 14 Jun 2021 19:37:02 -0400 (EDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 35B123982403 for ; Mon, 14 Jun 2021 23:37:02 +0000 (GMT) Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by sourceware.org (Postfix) with ESMTPS id E24BE386FC22 for ; Mon, 14 Jun 2021 23:36:50 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org E24BE386FC22 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de Received: from imap.suse.de (imap-alt.suse-dmz.suse.de [192.168.254.47]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D5192219A1; Mon, 14 Jun 2021 23:36:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1623713809; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KAeZ/oaYmA+j8yYP5I57OL7bWkkgv1Fs4IwLKcLwFWI=; b=INufwAjaBvqIWELYWmHRw3j2iyiQSVbBaAmLM5rBkRJR+fqC6Pl0bNPmoaYUicI0OYWsqI Ua9BPrpuysP8IkX7Hnq58ngPrFxCjQWu6CwqEmGSFcRzt3p/h83R6/5NkiMsNz0hXAw8SJ iNKVAXwlzYmlo/ex5qmV+MReFmI+pNE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1623713809; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KAeZ/oaYmA+j8yYP5I57OL7bWkkgv1Fs4IwLKcLwFWI=; b=2V/mWreyvhhK1aHl2LUtN2148CSNx3WpHRYPZBkk1QiPD96Yw3fHvJfKg+3CGzz9eqp+Ov Fb0WAkmzMuA78fDg== Received: from imap3-int (imap-alt.suse-dmz.suse.de [192.168.254.47]) by imap.suse.de (Postfix) with ESMTP id AEA79118DD; Mon, 14 Jun 2021 23:36:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1623713809; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KAeZ/oaYmA+j8yYP5I57OL7bWkkgv1Fs4IwLKcLwFWI=; b=INufwAjaBvqIWELYWmHRw3j2iyiQSVbBaAmLM5rBkRJR+fqC6Pl0bNPmoaYUicI0OYWsqI Ua9BPrpuysP8IkX7Hnq58ngPrFxCjQWu6CwqEmGSFcRzt3p/h83R6/5NkiMsNz0hXAw8SJ iNKVAXwlzYmlo/ex5qmV+MReFmI+pNE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1623713809; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KAeZ/oaYmA+j8yYP5I57OL7bWkkgv1Fs4IwLKcLwFWI=; b=2V/mWreyvhhK1aHl2LUtN2148CSNx3WpHRYPZBkk1QiPD96Yw3fHvJfKg+3CGzz9eqp+Ov Fb0WAkmzMuA78fDg== Received: from director2.suse.de ([192.168.254.72]) by imap3-int with ESMTPSA id fK6VKRHox2CAfwAALh3uQQ (envelope-from ); Mon, 14 Jun 2021 23:36:49 +0000 Subject: Re: [RFC][gdb/symtab] Lazy expansion of full symbol table To: Tom Tromey References: <20210614093908.GA22709@delia> <87pmwoxj3j.fsf@tromey.com> From: Tom de Vries Message-ID: <533bf7e4-d96c-a6b7-8c37-a4141ebdc761@suse.de> Date: Tue, 15 Jun 2021 01:36:49 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: <87pmwoxj3j.fsf@tromey.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gdb-patches@sourceware.org Errors-To: gdb-patches-bounces+public-inbox=simark.ca@sourceware.org Sender: "Gdb-patches" On 6/14/21 10:54 PM, Tom Tromey wrote: >>>>>> "Tom" == Tom de Vries writes: > > Tom> [ I'm not posting the experimental patch series as such for now. Available > Tom> here ( https://github.com/vries/gdb/commits/lazy-full-symtab ). ] > > Tom> In PR23710, the stated problem is that gdb is slow and memory hungry when > Tom> consuming debug information generated by GCC with LTO. > > Tom> | Minimal symbols | 0.18 | > Tom> | Partial symbols | 2.34 | > Tom> | Full symbols | 3.34 | > > I don't have this executable Uploaded to https://ftp.suse.com/pub/people/tdevries/gdb/pr23710/cc1 > but FWIW my scanner rewrite is ~10x faster > than the current psymtab reader. > Interesting, that's https://github.com/tromey/gdb/commits/submit/no-more-psym ? I've tried that branch with the cc1 example, and ran into: ... DW_FORM_strp pointing outside of .debug_str section [in module cc1] ... > Tom> A way to fix this is to do processing of the full symbols in a lazy fashion. > > Tom> This patch series implements a first attempt at this, for now intended not to > Tom> be functionally correct, but to assess the kind of performance improvement we > Tom> get from this. > > Tom> With current trunk (commit 987610f2d68), we get 3.44, instead of the 6.44 > Tom> without this patch series. > > That's about in line with the preliminary results I saw as well. > You can see that branch at tromey/lazily-read-function-bodies, but it's > probably unusable since I last rebased it in 2017. > Ack, thanks for the pointer. > Tom> The problem is that we need a way to do this gradually instead: > Tom> - expand a few symbols > Tom> - get the correspoding symbol table > Tom> - expand a few more symbols > Tom> - get the updated symbol table contain all expanded symbols > > Tom> I'm not sure what is the smartest way to do that. > > Tom> My current idea is to try to keep the builder around rather than destroy it, > Tom> and have it generate an updated symbol table each time. > > Tom> Is this a good idea? > > Tom> Any other comments? > > I think it's a good idea to do something. Historically I didn't look > into speeding up CU expansion on the theory that it didn't really matter > -- most CUs are small -- but LTO upends that. > > I never finished the earlier patch that I referenced, but the idea there > was to skip function bodies when reading a CU. Then, the functions > could be fully read when needed. I got this to work for by-name > lookups, but I never finished the part where a by-address lookup would > resize the blockvector, etc. > > This approach could be extended to also not read type bodies until > needed. This could work by inflating the type during check_typedef. > This can be a little dangerous because we do miss these calls from time > to time; but maybe nearly as good would be to only lazily read > "complicated" types, and use the accessor methods on those types to > ensure they are inflated properly. > > > Another deeper approach would be to abstract away symtabs (and I guess > the blockvector) the way that we did for psymtabs: hide the details > behind the "quick" methods and let each debug reader decide for itself > how to store and look up symbols. This is probably more work, but would > allow even more laziness. For example with the new indexer the symbols > could simply hang off the entries in the index. > OK, I'll try to take a look at the branch and digest your comments. Thanks, - Tom