C++ and multiple compilation units

C++ and multiple compilation units

[Jim Blandy]

Dan, a while back I think you mentioned a problem that would only
appear when debugging C++ programs using Dwarf 2 debug info that used
multiple compilation units.  I think the issue was that our current
dwarf 2 reader doesn't really handle inter-CU DIE references, and that
GCC was going to start generating them.  Our current C++ tests
probably won't catch such a problem --- almost all of the test
programs are a single CU.

Can you give more details on this?  Does GCC generate these inter-CU
references now?  For what sorts of constructs?  Do you have a test
case?

Re: C++ and multiple compilation units

[Daniel Berlin]

Jim Blandy <jimb@zwingli.cygnus.com> writes:

> Dan, a while back I think you mentioned a problem that would only
> appear when debugging C++ programs using Dwarf 2 debug info that used
> multiple compilation units.  I think the issue was that our current
> dwarf 2 reader doesn't really handle inter-CU DIE references, and that
> GCC was going to start generating them. 

Yup.
>  Our current C++ tests
> probably won't catch such a problem --- almost all of the test
> programs are a single CU.
> 
> Can you give more details on this? 
Sure.

To do elimination, you need inter-cu die references, so you can refer
to previous existing DIEs/ use linkonce sections, etc.

In fact, dwarf2.1 adds an imported_unit/partial_unit tag so that you can comdat
just about everything, and not screw up the scope (partial_unit means
ignore it when scanning toplevel cu's, we read it in when we see the
imported_unit tag))
>  Does GCC generate these inter-CU
> references now? 

No, but only because GDB can't handle them
Let me rephrase that. It *can* generate them, it just defaults to not doing so.
-feliminate-dwarf2-dups is the flag to turn it on.

>  For what sorts of constructs?
Currently it comdats symbols likely to be shared, if
-feliminate-dwarf2-dups is on. This is based on dwarf2 elimination,
i'm going to be modifying it soon to use the dwarf2.1 partial/imported
unit stuff, to do even more elimination.

In the future, it'll just do what the dwarf2.1 standard says in the
appendix (which is based on starting from what gcc did, and improving
a bit, using section groups and a new tag to specify an imported unit) 
>   Do you have a test case?
I can easily generate them, if you like. It's pretty trivial.

The inter-die cu references is what started the need the rewrite.  We assume
everywhere we are only processing the current CU, and never have to
move between them. 
So i rewrote the whole damn thing to pass around a structure that
contains the compilation unit context to do whatever in.  When we need
to switch between compilation units, we just change the context we are
passing around, reading in the abbrevs/whatever for that cu if
necessary.


--Dan

-- 
"I have the oldest typewriter in the world.  It types in pencil.
"-Steven Wright

Re: C++ and multiple compilation units

[Jim Blandy]

Daniel Berlin <dan@cgsoftware.com> writes:
> In fact, dwarf2.1 adds an imported_unit/partial_unit tag so that you
> can comdat just about everything, and not screw up the scope
> (partial_unit means ignore it when scanning toplevel cu's, we read
> it in when we see the imported_unit tag))

This is new in Draft 6, right?

> >  Does GCC generate these inter-CU
> > references now? 
> 
> No, but only because GDB can't handle them
> Let me rephrase that. It *can* generate them, it just defaults to not doing so.
> -feliminate-dwarf2-dups is the flag to turn it on.

Okay.

> >   Do you have a test case?
> I can easily generate them, if you like. It's pretty trivial.

That would be really helpful.

> The inter-die cu references is what started the need the rewrite.
> We assume everywhere we are only processing the current CU, and
> never have to move between them.

That's right.

> So i rewrote the whole damn thing to pass around a structure that
> contains the compilation unit context to do whatever in.  When we need
> to switch between compilation units, we just change the context we are
> passing around, reading in the abbrevs/whatever for that cu if
> necessary.

But once we've called read_comp_unit, we're not going any more I/O ---
we've turned the whole CU into a linked list (which should be a tree)
of `struct die_info' nodes.  So if we find a reference to another
compilation unit, it seems to me we could just call read_comp_unit
again, and extend our die list (and reference table).

We'd need a map of where each compilation unit starts and ends, so we
could tell which CU to read when we find a reference that falls
outside of our own.  But that's easily produced when we build the
partial symbols.

Suppose further that we restructured `struct die_info' to be a real
tree, not a linked list whose tree structure is only apparent from the
presence of null dies.  Each die could have a parent pointer, which
would allow us to produce scoped names.

This *sounds* a bit simpler than your changes.  But I haven't actually
done the work, so I can't be sure.

Re: C++ and multiple compilation units

[Daniel Berlin]

Jim Blandy <jimb@zwingli.cygnus.com> writes:

> Daniel Berlin <dan@cgsoftware.com> writes:
>> In fact, dwarf2.1 adds an imported_unit/partial_unit tag so that you
>> can comdat just about everything, and not screw up the scope
>> (partial_unit means ignore it when scanning toplevel cu's, we read
>> it in when we see the imported_unit tag))
> 
> This is new in Draft 6, right?

Yes, but it's been brewing for a while (it was just taking time to
review).

> 
>> >  Does GCC generate these inter-CU
>> > references now? 
>> 
>> No, but only because GDB can't handle them
>> Let me rephrase that. It *can* generate them, it just defaults to not doing so.
>> -feliminate-dwarf2-dups is the flag to turn it on.
> 
> Okay.
> 
>> >   Do you have a test case?
>> I can easily generate them, if you like. It's pretty trivial.
> 
> That would be really helpful.
Okay.

> 
>> The inter-die cu references is what started the need the rewrite.
>> We assume everywhere we are only processing the current CU, and
>> never have to move between them.
> 
> That's right.
> 
>> So i rewrote the whole damn thing to pass around a structure that
>> contains the compilation unit context to do whatever in.  When we need
>> to switch between compilation units, we just change the context we are
>> passing around, reading in the abbrevs/whatever for that cu if
>> necessary.
> 
> But once we've called read_comp_unit, we're not going any more I/O ---
> we've turned the whole CU into a linked list (which should be a tree)
> of `struct die_info' nodes.  So if we find a reference to another
> compilation unit, it seems to me we could just call read_comp_unit
> again, and extend our die list (and reference table).

You'd think, but you'd be as wrong as i was when i started. :)
We throw away the hash tables we need, and would otherwise need a
stack of abbrev tables around (since we need to use a seperate abbrev
table for the other CU, and they all start numbering at 1), etc.
And, dwarf2.1 will have provisions for saying that debug info is
located in another file, so there is no guarantee we won't do I/O.

Also, the section pointers and such giving us info about *where* the
info is located is currently in global variables, so they'd have to be
made into some stackish structure too.
The die reference tables also are not set up to expect anything but a
current cu based offset, so you'd need to change the key there too.
And with 2.1 saying things could be in another file, you can't just
use an absolute offset. (I use the offset and the bfd involved)

> 
> We'd need a map of where each compilation unit starts and ends, so we
> could tell which CU to read when we find a reference that falls
> outside of our own.  But that's easily produced when we build the
> partial symbols.
Which is how I do it.  And when it falls outside the current CU, I go
lookup the "CU" context for it, which gives me the abbrev tables and
whatnot I need to be able to process that DIE.
I don't reread the entire compilation unit.
Though I could if necessary.

> 
> Suppose further that we restructured `struct die_info' to be a real
> tree, not a linked list whose tree structure is only apparent from the
> presence of null dies.  Each die could have a parent pointer, which
> would allow us to produce scoped names.
> This *sounds* a bit simpler than your changes.  But I haven't actually
> done the work, so I can't be sure.
If you look at my changes, you'll notice I added such a parent
pointer, specifically to produce scoped names.
:)

My changes are based on the same general ideas you are
proposing. 

The CU context I speak of is just an encapsulation of the necessary
info to be able to process DIE's in a given CU.
I deglobalized the CU specific variables we have right now.
The other changes are either related to
1. The parent pointer to scope the names
2. Cleaning up the functions so they don't depend on global
   variables/magic to know where they are (I.E. expect to only ever be
   called by some certain caller. This was done so that necessary to
   reread entire compilation units, we can do it. 

The work you are proposing is pretty much exactly the work i've
accomplished. I thought it was as easy as you suggest as well.  It
would have been, had we not had all these fun global variables to deal
with, and assumptions.

We had a ton of both, so it took a lot of changes.
It's not like i changed the actual basic data structures in use,
except in a few small cases (hash tables->splay trees, since these
were structures where it was hard to choose a good number of fixed
buckets).


-- 
"Ever notice how irons have a setting for *permanent* press?  I
don't get it...
"-Steven Wright