Re: [RFC PATCH 7/8] gdb/ctf: don't use psymtabs, create symtabs directly

Re: [RFC PATCH 7/8] gdb/ctf: don't use psymtabs, create symtabs directly

[Tom Tromey]

>>>>> "Jan" == Jan Vrany <jan@vrany.io> writes:

>> In order to access the symtabs, elfctf_build_symtabs installs the
>> expanded_symbols_functions quick_symbol_functions implementation, which
>> essentially searches in the existing symtabs.  I am pretty sure this is
>> not 100% correct, because this would search unrelated symtabs, if for
>> instance the CTF debug info co-existed with DWARF info.  But it's good
>> enough for a prototype.

Jan> True, but does that matter? Should that matter?

If it's possible then it could matter.

Consider if you have both a CTF and a DWARF "readnow" implementation
attached to an objfile.  Now all searches are done twice -- probably
won't be incorrect but it will be slower.

In some earlier thread I proposed fixing this by adding a marker to the
compunit_symtab to record where it came from.  Then the "expanded
symbols for JIT" expanded-symbols instance could limit its search.

However:

Jan> Also it seems to me that this version does not allow for both CTF and
Jan> DWARF in single objfile: 

... do we have any spot any more that records symbols from multiple
readers?  Since another option would be to get rid of that ability;
unless we think it'll be used again.

Tom

Re: [RFC PATCH 7/8] gdb/ctf: don't use psymtabs, create symtabs directly

[Jan Vrany via Gdb]

On Tue, 2026-02-03 at 07:35 -0700, Tom Tromey wrote:
> > > > > > "Jan" == Jan Vrany <jan@vrany.io> writes:
> 
> > > In order to access the symtabs, elfctf_build_symtabs installs the
> > > expanded_symbols_functions quick_symbol_functions implementation, which
> > > essentially searches in the existing symtabs.  I am pretty sure this is
> > > not 100% correct, because this would search unrelated symtabs, if for
> > > instance the CTF debug info co-existed with DWARF info.  But it's good
> > > enough for a prototype.
> 
> Jan> True, but does that matter? Should that matter?
> 
> If it's possible then it could matter.
> 
> Consider if you have both a CTF and a DWARF "readnow" implementation
> attached to an objfile.  Now all searches are done twice -- probably
> won't be incorrect but it will be slower.

In case the search fails, yes. Otherwise the first implementation that
finds something wins, the others are not tried. It looks to me this can
be easily solved by adding each implementation only once.

As a side note, when debugging quick symbols I noticed that very often
the same thing is searched for in quick succession. 

> 
> In some earlier thread I proposed fixing this by adding a marker to the
> compunit_symtab to record where it came from.  Then the "expanded
> symbols for JIT" expanded-symbols instance could limit its search.
> 
> However:
> 
> Jan> Also it seems to me that this version does not allow for both CTF and
> Jan> DWARF in single objfile:
> 
> ... do we have any spot any more that records symbols from multiple
> readers?  Since another option would be to get rid of that ability;
> unless we think it'll be used again.

In "Python JIT API" series, I allow to add symbols any objfile, even
the one with existing DWARF. To make this work, I push expanded_symbols_functions
to the end of qf list. Whether this is a good or bad idea, I do not know - the reason 
for this was to support an arguably extreme usecase where dynamic code is generated 
into statically allocated buffer or section. I'm happy to reintroduce these checks (and
therefore not supporting this usecase) if it feels "safer".

Jan

> 
> Tom

Re: [RFC PATCH 7/8] gdb/ctf: don't use psymtabs, create symtabs directly

[Simon Marchi via Gdb]

On 2/3/26 10:14 AM, Jan Vrany via Gdb wrote:
> On Tue, 2026-02-03 at 07:35 -0700, Tom Tromey wrote:
>>>>>>> "Jan" == Jan Vrany <jan@vrany.io> writes:
>>
>>>> In order to access the symtabs, elfctf_build_symtabs installs the
>>>> expanded_symbols_functions quick_symbol_functions implementation, which
>>>> essentially searches in the existing symtabs.  I am pretty sure this is
>>>> not 100% correct, because this would search unrelated symtabs, if for
>>>> instance the CTF debug info co-existed with DWARF info.  But it's good
>>>> enough for a prototype.
>>
>> Jan> True, but does that matter? Should that matter?
>>
>> If it's possible then it could matter.
>>
>> Consider if you have both a CTF and a DWARF "readnow" implementation
>> attached to an objfile.  Now all searches are done twice -- probably
>> won't be incorrect but it will be slower.
> 
> In case the search fails, yes. Otherwise the first implementation that
> finds something wins, the others are not tried. It looks to me this can
> be easily solved by adding each implementation only once.
> 
> As a side note, when debugging quick symbols I noticed that very often
> the same thing is searched for in quick succession. 

There is this "set always-read-ctf" setting, which makes it possible to
have both DWARF and CTF.

The order of "quick functions" matters, it will affect which debug info
provider will be searched first.  At least this is what this commit
message taught me:

https://gitlab.com/gnutools/binutils-gdb/-/commit/0d5adb56c85da38a0f95e872fda05cc6446010c3

I don't really like that we rely on the order in which things are read
(and therefore the order in which quick_symbol_functions objects are
pushed, I suppose) to define the priority.  It seems like it would be
clearer if all quick_symbol_functions implementations were assigned an
arbitrary "weight" to define the priority.  But that's unrelated to this
issue.

The thing is that if expanded_symbols_functions searches all symtabs, I
think it would break that priority assumption.  Right now, the order of
quick_symbol_functions instances defines the priority of search.  With
expanded_symbols_functions searching all compunit_symtabs, then the
order of compunit_symtabs may affect that priority.

To illustrate, a thought experiment: imagine you use both DWARF (using
its cooked index) and JIT (using expanded_symbols_functions), and we
define that JIT has precedence over DWARF.  The user does an action that
causes expansion of a DWARF symtab containing a symbol Foo.  That
compunit_symtab is first in the compunit_symtab list.  The JIT then
creates a compunit_symtab that contains another symbol Foo.  Because JIT
has priority, a lookup for Foo should find JIT's Foo.  However, JIT's
expanded_symbols_functions search will hit DWARF's compunit_symtab and
find DWARF's Foo symbol.

I think it would be safer for each quick_symbol_functions to search its
own stuff only.  What I'm thinking is that the
expanded_symbols_functions can hold a vector<compunit_symtab *>,
containing the compunit_symtabs created by this reader.  The search
method of expanded_symbols_functions would search those compunits.  If
you have both the JIT reader and the CTF reader active, then they would
both push there own expanded_symbols_functions instance, with their own
list of compunit_symtabs.

>> In some earlier thread I proposed fixing this by adding a marker to the
>> compunit_symtab to record where it came from.  Then the "expanded
>> symbols for JIT" expanded-symbols instance could limit its search.

This would be functionally similar to what I proposed above, I think.

Simon

Re: [RFC PATCH 7/8] gdb/ctf: don't use psymtabs, create symtabs directly

[Tom Tromey]

>>>>> "Simon" == Simon Marchi <simark@simark.ca> writes:

Simon> I think it would be safer for each quick_symbol_functions to search its
Simon> own stuff only.  What I'm thinking is that the
Simon> expanded_symbols_functions can hold a vector<compunit_symtab *>,
Simon> containing the compunit_symtabs created by this reader.  The search
Simon> method of expanded_symbols_functions would search those compunits.  If
Simon> you have both the JIT reader and the CTF reader active, then they would
Simon> both push there own expanded_symbols_functions instance, with their own
Simon> list of compunit_symtabs.

This would be totally fine and might help us get rid of the last things
that iterate over all compunits.  Aside from genuine needs -- "maint"
commands and relocation -- I tend to think this should not be done.

Tom

Re: [RFC PATCH 7/8] gdb/ctf: don't use psymtabs, create symtabs directly

[Jan Vrany via Gdb]

On Tue, 2026-02-03 at 16:05 -0700, Tom Tromey wrote:
> > > > > > "Simon" == Simon Marchi <simark@simark.ca> writes:
> 
> Simon> I think it would be safer for each quick_symbol_functions to search its
> Simon> own stuff only.  What I'm thinking is that the
> Simon> expanded_symbols_functions can hold a vector<compunit_symtab *>,
> Simon> containing the compunit_symtabs created by this reader.  The search
> Simon> method of expanded_symbols_functions would search those compunits.  If
> Simon> you have both the JIT reader and the CTF reader active, then they would
> Simon> both push there own expanded_symbols_functions instance, with their own
> Simon> list of compunit_symtabs.
> 
> This would be totally fine and might help us get rid of the last things
> that iterate over all compunits.  Aside from genuine needs -- "maint"
> commands and relocation -- I tend to think this should not be done.
> 
Okay, it seems that there's an agreement this is the way to go. 

UUIC this means that:
1) there can be multiple compunit_symtabs for the same code
   (say you have both CTF and DWARF debug info so each will have its 
   own compunit_symtab with its own, possibly different, symbols). 
2) Given a compunit_symtab, it can only contain data from one reader.
   In another words, (Python) JIT reader may not be able to add symbols
   to compunit_symtab created by DWARF reader. 

Jan

Re: [RFC PATCH 7/8] gdb/ctf: don't use psymtabs, create symtabs directly

[Tom Tromey]

>>>>> "Jan" == Jan Vrany <jan@vrany.io> writes:

Jan> In case the search fails, yes. Otherwise the first implementation that
Jan> finds something wins, the others are not tried. It looks to me this can
Jan> be easily solved by adding each implementation only once.

It depends on which method is called.
For instance linespec generally tries to collect all matches, whereas
type lookups tend to stop with the first (non-stub) type.

Jan> As a side note, when debugging quick symbols I noticed that very often
Jan> the same thing is searched for in quick succession. 

Yeah.  There's a lot of old & ugly code in there.

There's a cache in symtab.c to try to speed this up.

Jan> In "Python JIT API" series, I allow to add symbols any objfile, even
Jan> the one with existing DWARF. To make this work, I push expanded_symbols_functions
Jan> to the end of qf list. Whether this is a good or bad idea, I do not know - the reason 
Jan> for this was to support an arguably extreme usecase where dynamic code is generated 
Jan> into statically allocated buffer or section. I'm happy to reintroduce these checks (and
Jan> therefore not supporting this usecase) if it feels "safer".

I didn't look at the series yet but I think this approach will regress
some recent performance improvements.

Tom