Re: [RFC][gdb/symtab] Lazy expansion of full symbol table

[Tom Tromey <tom@tromey.com>]

>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:

Tom> [ I'm not posting the experimental patch series as such for now.  Available
Tom> here ( https://github.com/vries/gdb/commits/lazy-full-symtab ). ]

Tom> In PR23710, the stated problem is that gdb is slow and memory hungry when
Tom> consuming debug information generated by GCC with LTO.

Tom> | Minimal symbols | 0.18           |
Tom> | Partial symbols | 2.34           |
Tom> | Full symbols    | 3.34           |

I don't have this executable but FWIW my scanner rewrite is ~10x faster
than the current psymtab reader.

Tom> A way to fix this is to do processing of the full symbols in a lazy fashion.

Tom> This patch series implements a first attempt at this, for now intended not to
Tom> be functionally correct, but to assess the kind of performance improvement we
Tom> get from this.

Tom> With current trunk (commit 987610f2d68), we get 3.44, instead of the 6.44
Tom> without this patch series.

That's about in line with the preliminary results I saw as well.
You can see that branch at tromey/lazily-read-function-bodies, but it's
probably unusable since I last rebased it in 2017.

Tom> The problem is that we need a way to do this gradually instead:
Tom> - expand a few symbols
Tom> - get the correspoding symbol table
Tom> - expand a few more symbols
Tom> - get the updated symbol table contain all expanded symbols

Tom> I'm not sure what is the smartest way to do that.

Tom> My current idea is to try to keep the builder around rather than destroy it,
Tom> and have it generate an updated symbol table each time.

Tom> Is this a good idea?

Tom> Any other comments?

I think it's a good idea to do something.  Historically I didn't look
into speeding up CU expansion on the theory that it didn't really matter
-- most CUs are small -- but LTO upends that.

I never finished the earlier patch that I referenced, but the idea there
was to skip function bodies when reading a CU.  Then, the functions
could be fully read when needed.  I got this to work for by-name
lookups, but I never finished the part where a by-address lookup would
resize the blockvector, etc.

This approach could be extended to also not read type bodies until
needed.  This could work by inflating the type during check_typedef.
This can be a little dangerous because we do miss these calls from time
to time; but maybe nearly as good would be to only lazily read
"complicated" types, and use the accessor methods on those types to
ensure they are inflated properly.


Another deeper approach would be to abstract away symtabs (and I guess
the blockvector) the way that we did for psymtabs: hide the details
behind the "quick" methods and let each debug reader decide for itself
how to store and look up symbols.  This is probably more work, but would
allow even more laziness.  For example with the new indexer the symbols
could simply hang off the entries in the index.

Tom