Branch created for inter-compilation-unit references

Branch created for inter-compilation-unit references

[Daniel Jacobowitz]

I've just created "drow_intercu-20040221-branch".  I'm going to post and
commit the work I've done over the past two days, which starts at the
opposite end of the problem from several approaches I've seen here in the
past.

At present we could safely ignore the case of partial symbol tables.  The
only inter-die reference that we would even examine during partial symbol
table building is DW_AT_specification (and similar), for the names of
DIEs whose specification is in a namespace.  And generally we have a mangled
name so the fallback that David checked in some weeks ago works OK.

But one of the long-term goals for inter-CU support is to decrease the total
size of debug information.  One way to do that is separating header DIEs
into separate compilation units; but another important way is to scrap
DW_AT_MIPS_linkage_name.  I don't want to add a new dependence on the
mangled names that I would just have to clean up later.

So I've begun with the partial symbol tables.  I haven't actually added any
inter-CU support yet; only support within a single CU for following
DW_AT_specification.  I load all necessary DIEs into memory before parsing
any, with a couple of exceptions for speed.  I build sibling and child
chains like we do for full DIEs.  This is where the performance cost comes
in: the two big hitters are adding the DIEs I want to save to the hash
table, and data cache misses from separating the loading and processing
of the DIE.

My focus so far has been on not increasing startup time for C (primarily) or
C++ (secondarily) applications; I am within about 4% for C and about 8% for
C++. I think that will be acceptable for now - there's room for about 10%
improvement in other parts of GDB for C and 30% for C++, based on my
profiling, and I will pursue it later.  Probably before merging this code.

Next will be real inter-CU support for partial symbols.  Then for full
symbols.  My hope is that I can do both of those without perturbing the
non-inter-CU path too much.

GCC HEAD can already generate the necessary debug information if you want to
play with it; use -gdwarf-2 -feliminate-dwarf2-dups.  I expect it will be at
least a few more days before GDB can read that output.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Branch created for inter-compilation-unit references

[Daniel Jacobowitz]

On Sat, Feb 21, 2004 at 03:08:15PM -0500, Daniel Jacobowitz wrote:
> I've just created "drow_intercu-20040221-branch".  I'm going to post and
> commit the work I've done over the past two days, which starts at the
> opposite end of the problem from several approaches I've seen here in the
> past.

> GCC HEAD can already generate the necessary debug information if you want to
> play with it; use -gdwarf-2 -feliminate-dwarf2-dups.  I expect it will be at
> least a few more days before GDB can read that output.

The branch is at this point, I hope, complete.  Michael, if you have
the chance, I would appreciate it if you could run it through your test
harness.  I would be flabbergasted if it affected any stabs target
(there's only one trivial patch outside of dwarf2read.c), and the new
code will not be heavily exercised by current GCC output, but the risk
for the latter is higher.

If you're really feeling adventurous, you can test the new code when
using DWARF-2 and GCC 3.3-branch, 3.4-branch, HEAD, tree-ssa, et
cetera: just add "--target_board unix/-feliminate-dwarf2-dups" to
RUNTESTFLAGS.  HEAD GDB will choke on this, the branch does not in my
testing.

Anyone else with a compiler that can generate this sort of data is
invited to test the branch also.  Please let me know how it goes.


I will start submitting preliminary patches, those which can also act
as bug-fixes or optimizations to the current dwarf2 code, tonight or
tomorrow.  Merging the branch may have to wait until after GDB 6.1.
Personally, I'd love for it to be included, since the sooner we release
a GDB that can understand this data the sooner GCC can emit it by
default; but I want at least some more testing on the branch before
I suggest that.  My timing for this project was somewhat unfortunate.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Branch created for inter-compilation-unit references

[Andrew Cagney]

> I will start submitting preliminary patches, those which can also act
> as bug-fixes or optimizations to the current dwarf2 code, tonight or
> tomorrow.  Merging the branch may have to wait until after GDB 6.1.
> Personally, I'd love for it to be included, since the sooner we release
> a GDB that can understand this data the sooner GCC can emit it by
> default; but I want at least some more testing on the branch before
> I suggest that.  My timing for this project was somewhat unfortunate.

Er, given that GCC 3.4 branch is emiting DW_OP_piece, isn't that more 
urgent?

Andrew

Re: Branch created for inter-compilation-unit references

[Daniel Jacobowitz]

On Tue, Feb 24, 2004 at 07:35:15PM -0500, Andrew Cagney wrote:
> >I will start submitting preliminary patches, those which can also act
> >as bug-fixes or optimizations to the current dwarf2 code, tonight or
> >tomorrow.  Merging the branch may have to wait until after GDB 6.1.
> >Personally, I'd love for it to be included, since the sooner we release
> >a GDB that can understand this data the sooner GCC can emit it by
> >default; but I want at least some more testing on the branch before
> >I suggest that.  My timing for this project was somewhat unfortunate.
> 
> Er, given that GCC 3.4 branch is emiting DW_OP_piece, isn't that more 
> urgent?

Probably.  As I said, I try to do large projects roughly FIFO, and this
has been on my TODO list a lot longer than DW_OP_piece has.

If you're interested in DW_OP_piece support for 6.1, I can certainly
look into it after this is done.  It will probably be less work.  I
don't want to try to juggle two large dwarf2 projects at the same time,
though.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Branch created for inter-compilation-unit references

[Andrew Cagney]

[I dropped the nameless r6k, they bounced my e-mail]

> On Tue, Feb 24, 2004 at 07:35:15PM -0500, Andrew Cagney wrote:
> 
>>>> >I will start submitting preliminary patches, those which can also act
>>>> >as bug-fixes or optimizations to the current dwarf2 code, tonight or
>>>> >tomorrow.  Merging the branch may have to wait until after GDB 6.1.
>>>> >Personally, I'd love for it to be included, since the sooner we release
>>>> >a GDB that can understand this data the sooner GCC can emit it by
>>>> >default; but I want at least some more testing on the branch before
>>>> >I suggest that.  My timing for this project was somewhat unfortunate.
>>
>>> 
>>> Er, given that GCC 3.4 branch is emiting DW_OP_piece, isn't that more 
>>> urgent?
> 
> 
> Probably.  As I said, I try to do large projects roughly FIFO, and this
> has been on my TODO list a lot longer than DW_OP_piece has.

> If you're interested in DW_OP_piece support for 6.1, I can certainly
> look into it after this is done.  It will probably be less work.  I
> don't want to try to juggle two large dwarf2 projects at the same time,
> though.

In terms of getting ICU "done", talk about bad timing.  I suspect 
everyone's concentration is as directed towards 6.1 and other more 
urgent issues - this right now is far from peoples minds (I've, for 
instance, been simply deleting your patches).

Andrew

Re: Branch created for inter-compilation-unit references

[Daniel Jacobowitz]

On Tue, Feb 24, 2004 at 09:23:44PM -0500, Andrew Cagney wrote:
> In terms of getting ICU "done", talk about bad timing.  I suspect 
> everyone's concentration is as directed towards 6.1 and other more 
> urgent issues - this right now is far from peoples minds (I've, for 
> instance, been simply deleting your patches).

This is an important issue to me.  You know the drill: the release made
me look over my lists and see what had slipped, and this had slipped.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Branch created for inter-compilation-unit references

[Andrew Cagney]

> This is an important issue to me.  You know the drill: the release made
> me look over my lists and see what had slipped, and this had slipped.

I wish, It's certainly not my drill.  Prior to a release I'll drop all 
the things I'd like to work on, and instead concentrate on finishing 
stuff that's already started.

Andrew

Re: Branch created for inter-compilation-unit references

[Michael Elizabeth Chastain]

> Michael, if you have the chance, I would appreciate it if you could run
> it through your test harness.

Okay, I'll add this branch to the next spin.  Results in about 72 hours.

> If you're really feeling adventurous, you can test the new code when
> using DWARF-2 and GCC 3.3-branch, 3.4-branch, HEAD, tree-ssa, et
> cetera: just add "--target_board unix/-feliminate-dwarf2-dups" to
> RUNTESTFLAGS.  HEAD GDB will choke on this, the branch does not in my
> testing.

I can do a special spin with -feliminate-dwarf2-dups.

> Merging the branch may have to wait until after GDB 6.1.

This process is looking like gcc, which is probably an improvement.
Develop on the branch; verify no regressions; then merge.

Michael C

Re: Branch created for inter-compilation-unit references

[Andrew Cagney]

>>Merging the branch may have to wait until after GDB 6.1.
> 
> 
> This process is looking like gcc, which is probably an improvement.
> Develop on the branch; verify no regressions; then merge.

Careful, GCC is currently faceing an SSA mega-merge.  A strategy, 
reminiscent of the HP merge, is not one we want to encourage here. 
Haveing said that short lived branches for experimentation are a good idea.

Andrew

Re: Branch created for inter-compilation-unit references

[Daniel Berlin]

On Feb 25, 2004, at 12:11 AM, Andrew Cagney wrote:

>>> Merging the branch may have to wait until after GDB 6.1.
>> This process is looking like gcc, which is probably an improvement.
>> Develop on the branch; verify no regressions; then merge.
>
> Careful, GCC is currently faceing an SSA mega-merge.

Um, except again, we are verifying no regressions in test results, plus 
no serious (>5%) regressions in compile time or execution time.
You also forgot that the code has already been reviewed by global 
maintainers who were working on the branch, and will again be design 
reviewed before committing to the main branch.
Plus it includes both high-level design, and user-level documentation
Finally, the merge in question, plus the document describing the merge 
and it's criteria, was explicitly approved by the GCC Steering 
Committee.

>   A strategy, reminiscent of the HP merge, is not one we want to 
> encourage here.
The HP merge was completely different than the above.
It seems like you are trying to degrade gcc here by comparing the SSA 
merge to the HP merge, which is clearly a dumb comparison.
Just because certain gdb people screwed that merge up by not requiring 
more doesn't mean GCC will do the same, as evidenced by the above.


>  Haveing said that short lived branches for experimentation are a good 
> idea.
>
> Andrew
>
>

Re: Branch created for inter-compilation-unit references

[Andrew Cagney]

> On Feb 25, 2004, at 12:11 AM, Andrew Cagney wrote:
> 
>>>> Merging the branch may have to wait until after GDB 6.1.
>>>
>>> This process is looking like gcc, which is probably an improvement.
>>> Develop on the branch; verify no regressions; then merge.
>>
>>
>> Careful, GCC is currently faceing an SSA mega-merge.
> 
> 
> Um, except again, we are verifying no regressions in test results, plus no serious (>5%) regressions in compile time or execution time.
> You also forgot that the code has already been reviewed by global maintainers who were working on the branch, and will again be design reviewed before committing to the main branch.
> Plus it includes both high-level design, and user-level documentation
> Finally, the merge in question, plus the document describing the merge and it's criteria, was explicitly approved by the GCC Steering Committee.

Yes, I know, and it is all good news.  However, that doesn't diminish 
the projects problems: the shear size of the branch, the number of 
dedicated full time resources currently been consumed, the constant 
schedule slip, ...

>>   A strategy, reminiscent of the HP merge, is not one we want to encourage here.
> 
> The HP merge was completely different than the above.
> It seems like you are trying to degrade gcc here by comparing the SSA merge to the HP merge, which is clearly a dumb comparison.
> Just because certain gdb people screwed that merge up by not requiring more doesn't mean GCC will do the same, as evidenced by the above.

A comparison is reasonable (and it isn't ment to degrate SSA or GCC). 
If the HP merge were to have been handled correctly it would have turned 
into a project of size and logistics comparable to SSA.  That, I think, 
is getting out of control.

As I said:

>>  Haveing said that short lived branches for experimentation are a good idea.

Andrew

Re: Branch created for inter-compilation-unit references

[Daniel Berlin]

On Feb 25, 2004, at 12:01 PM, Andrew Cagney wrote:

>> On Feb 25, 2004, at 12:11 AM, Andrew Cagney wrote:
>>>>> Merging the branch may have to wait until after GDB 6.1.
>>>>
>>>> This process is looking like gcc, which is probably an improvement.
>>>> Develop on the branch; verify no regressions; then merge.
>>>
>>>
>>> Careful, GCC is currently faceing an SSA mega-merge.
>> Um, except again, we are verifying no regressions in test results, 
>> plus no serious (>5%) regressions in compile time or execution time.
>> You also forgot that the code has already been reviewed by global 
>> maintainers who were working on the branch, and will again be design 
>> reviewed before committing to the main branch.
>> Plus it includes both high-level design, and user-level documentation
>> Finally, the merge in question, plus the document describing the 
>> merge and it's criteria, was explicitly approved by the GCC Steering 
>> Committee.
>
> Yes, I know, and it is all good news.  However, that doesn't diminish 
> the projects problems: the shear size of the branch,
Size?
It's necessary to get the goals of the branch accomplished.
> the number of dedicated full time resources currently been consumed,

This is not a problem, it's a good thing.
People *want* to work on the branch. How is that bad?

>  the constant schedule slip, ...
>

tree-ssa was never on a schedule to begin with, so what the heck are 
you talking about?
If you really want to play that card, it wasn't even supposed to be 
ready before 3.6
The fact that it is ready for 3.5 means it certainly hasn't *slipped*.

>>>   A strategy, reminiscent of the HP merge, is not one we want to 
>>> encourage here.
>> The HP merge was completely different than the above.
>> It seems like you are trying to degrade gcc here by comparing the SSA 
>> merge to the HP merge, which is clearly a dumb comparison.
>> Just because certain gdb people screwed that merge up by not 
>> requiring more doesn't mean GCC will do the same, as evidenced by the 
>> above.
>
> A comparison is reasonable (and it isn't ment to degrate SSA or GCC). 
> If the HP merge were to have been handled correctly it would have 
> turned into a project of size and logistics comparable to SSA.  That, 
> I think, is getting out of control.
>

Then you are sorely mistaken.
If you think the SSA branch is an example of a  bad thing, i fear for 
gdb development.

Re: Branch created for inter-compilation-unit references

[Andrew Cagney]

> 
>>  the constant schedule slip, ...
>>
> 
> tree-ssa was never on a schedule to begin with, so what the heck are you talking about?
> If you really want to play that card, it wasn't even supposed to be ready before 3.6
> The fact that it is ready for 3.5 means it certainly hasn't *slipped*. 

Er, it was originally, lets say, "hopeing" for 3.4 (or was it 3.3). 
Fortunatly the needed resources have now been dedicated to the task.

Andrew

Re: Branch created for inter-compilation-unit references

[Daniel Berlin]

On Feb 25, 2004, at 1:50 PM, Andrew Cagney wrote:

>>>  the constant schedule slip, ...
>>>
>> tree-ssa was never on a schedule to begin with, so what the heck are 
>> you talking about?
>> If you really want to play that card, it wasn't even supposed to be 
>> ready before 3.6
>> The fact that it is ready for 3.5 means it certainly hasn't *slipped*.
>
> Er, it was originally, lets say, "hopeing" for 3.4 (or was it 3.3).
Sorry.
This simply isn't true.
Nobody involved thought it would be ready for 3.4.


>  Fortunatly the needed resources have now been dedicated to the task.
>
But i thought you said that the branch was consuming too many resources?

You == very bad troll.

--Dan

Re: Branch created for inter-compilation-unit references

[Andrew Cagney]

> 
> On Feb 25, 2004, at 1:50 PM, Andrew Cagney wrote:
> 
>>>>  the constant schedule slip, ...
>>>>
>>> tree-ssa was never on a schedule to begin with, so what the heck are you talking about?
>>> If you really want to play that card, it wasn't even supposed to be ready before 3.6
>>> The fact that it is ready for 3.5 means it certainly hasn't *slipped*.
>>
>>
>> Er, it was originally, lets say, "hopeing" for 3.4 (or was it 3.3).
> 
> Sorry.
> This simply isn't true.
> Nobody involved thought it would be ready for 3.4. 

I guess we just talk to different people :-/

Andrew

Re: Branch created for inter-compilation-unit references

[Eli Zaretskii]

> From: Daniel Berlin <dberlin@dberlin.org>
> Date: Wed, 25 Feb 2004 13:53:54 -0500
> 
> You == very bad troll.

Kindly stop this kind of ``reasoning'' here.  It hurts the discussions
by making them a childish exchange of ``did not'', ``did, too'' type
of arguments.  This issue is too serious for us to allow ourselves to
do that.

Re: Branch created for inter-compilation-unit references

[Daniel Berlin]

On Feb 26, 2004, at 12:45 AM, Eli Zaretskii wrote:

>> From: Daniel Berlin <dberlin@dberlin.org>
>> Date: Wed, 25 Feb 2004 13:53:54 -0500
>>
>> You == very bad troll.
>
> Kindly stop this kind of ``reasoning'' here.
> It hurts the discussions
> by making them a childish exchange of ``did not'', ``did, too'' type
> of arguments.

Andrew continually claimed provably untrue things that could do nothing 
but piss off people involved in the hard work that went on in the gcc 
tree-ssa branch (schedule slippage, too many resources consumed, etc)
It did, in fact, do just that, and i'm not referring to me, even though 
i did work on that branch. There was absolutely no reason for doing 
this other than to piss off people, since they were, as i said, 
provably untrue.
This is generally known as trolling.
This is why i replied.
Because others are too kind to call Andrew on some of the bullshit he 
spews.
I am not so kind.
If GDB actually had a real steering committee, i would seriously 
recommend that they reprimand him publicly for those comments (which 
GCC's steering committee has done for certain people in the past).
It's crap like that that strains relationships between GCC and GDB.
For a head maintainer, he should be being more careful about what he 
says.
> This issue is too serious for us to allow ourselves to
> do that.
>
I'm seriously concerned that if he thinks the tree-ssa branch is 
somehow an example of a bad development plan, that gdb development is 
going down the wrong path.
I'm also seriously concerned that if he somehow thinks DW_OP_piece 
support is more important than the intercu-branch, that he is also 
going down the wrong path.  GCC has emitted DW_OP_piece for a while. 
GCC has been able to emit inter-cu references for a while. Every user 
i've talked with is much more concerned with being able to reduce the 
size of their debugging info, which requires intercu support, than they 
are with debugging some variable that uses DW_OP_piece.
They've learned to work around DW_OP_piece issues through various 
hacks, but large debugging info and high memory usage prevents some of 
them from debugging at *all*.

But again, i'll just unsubscribe from gdb.
I came back after i was asked by various people to do so.
I won't make such a mistake again.

--Dan

Re: Branch created for inter-compilation-unit references

[Eli Zaretskii]

> From: Daniel Berlin <dberlin@dberlin.org>
> Date: Thu, 26 Feb 2004 01:05:58 -0500
> 
> Andrew continually claimed provably untrue things that could do nothing 
> but piss off people involved in the hard work that went on in the gcc 
> tree-ssa branch (schedule slippage, too many resources consumed, etc)

I think you are exaggerating.  Andrew expressed concerns about
technical issues based on whatever information he had.  I don't know
whom he was talking to, and I'm ignorant of the subject matter because
I don't track GCC lists, but my outsider's judgement is that his
argument was of a technical nature, and I don't see why someone would
spot any sign of malice in what he wrote.  He might be mistaken, as we
all are sometimes, but his is a genuine concern, not a wish to piss
off.

> This is generally known as trolling.

I think you know Andrew all too well to suspect that he is trollying.

> I'm seriously concerned that if he thinks the tree-ssa branch is 
> somehow an example of a bad development plan, that gdb development is 
> going down the wrong path.

I understand his comments differently: that he fears that merging
large branches _could_ have adverse side effects if things get out of
control.  In other words, it was a general comment on large merges,
not something too specific about the specific case of tree-ssa.

> I'm also seriously concerned that if he somehow thinks DW_OP_piece 
> support is more important than the intercu-branch, that he is also 
> going down the wrong path.

It's not more important in general, but since we are preparing to cut
the 6.1 branch in a few days, DW_OP_piece might be a good thing to do
now, while delaying intercu-branch merge till after the release.
It's a question of timing, not of an abstract importance.

Re: Branch created for inter-compilation-unit references

[Daniel Jacobowitz]

On Thu, Feb 26, 2004 at 08:30:56AM +0200, Eli Zaretskii wrote:
> It's not more important in general, but since we are preparing to cut
> the 6.1 branch in a few days, DW_OP_piece might be a good thing to do
> now, while delaying intercu-branch merge till after the release.
> It's a question of timing, not of an abstract importance.

Then both you and Andrew underestimate the intrusiveness of DW_OP_piece
support.  I estimate it will be less total code but considerably more
disruptive, since it will require changes outside of the dwarf2 reader.
Ergo require more testing, more settling time, more review time, et
cetera, and less workable for the branch.

Both of the distributions I maintain GDB for (one for my employer, and
Debian) are facing problems with the size and load time of debug
information.  For them, the intercu branch is both more useful and more
urgent.  Whether or not it makes GDB 6.1 through the gdb-patches review
process, the GDB 6.1 packages for both of those distributions will
include it.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Branch created for inter-compilation-unit references

[Elena Zannoni]

Daniel Jacobowitz writes:

 > Both of the distributions I maintain GDB for (one for my employer, and
 > Debian) are facing problems with the size and load time of debug
 > information.  For them, the intercu branch is both more useful and more
 > urgent.  Whether or not it makes GDB 6.1 through the gdb-patches review
 > process, the GDB 6.1 packages for both of those distributions will
 > include it.
 
I don't feel comfortable including it in FSF gdb6.1.  One strong
reason is that there has been no design review (as opposed to
tree-ssa).  If those distros give you more latitude (i.e. no review
process, availability of 'unstable and 'stable' versions, etc), it is
not a good reason for FSF to do the same.  I am not saying no a
priori.  It's just unfortunate timing.

Re: Branch created for inter-compilation-unit references

[Daniel Jacobowitz]

On Thu, Feb 26, 2004 at 11:14:43AM -0500, Elena Zannoni wrote:
> Daniel Jacobowitz writes:
> 
>  > Both of the distributions I maintain GDB for (one for my employer, and
>  > Debian) are facing problems with the size and load time of debug
>  > information.  For them, the intercu branch is both more useful and more
>  > urgent.  Whether or not it makes GDB 6.1 through the gdb-patches review
>  > process, the GDB 6.1 packages for both of those distributions will
>  > include it.
>  
> I don't feel comfortable including it in FSF gdb6.1.  One strong
> reason is that there has been no design review (as opposed to
> tree-ssa).  If those distros give you more latitude (i.e. no review
> process, availability of 'unstable and 'stable' versions, etc), it is
> not a good reason for FSF to do the same.  I am not saying no a
> priori.  It's just unfortunate timing.

OK, this is perfectly reasonable.  It's a lot of new code.

The design is actually - I can't really say based on - but pretty much
follows suggestions given to me by Jim Blandy last year.  There was
definitely no design review of this code; that's because I just sat
down and did it.  The design is part of what I have to ask the
maintainers of the dwarf2 reader to review.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Branch created for inter-compilation-unit references

[Elena Zannoni]

Daniel Jacobowitz writes:
 > On Thu, Feb 26, 2004 at 11:14:43AM -0500, Elena Zannoni wrote:
 > > Daniel Jacobowitz writes:
 > > 
 > >  > Both of the distributions I maintain GDB for (one for my employer, and
 > >  > Debian) are facing problems with the size and load time of debug
 > >  > information.  For them, the intercu branch is both more useful and more
 > >  > urgent.  Whether or not it makes GDB 6.1 through the gdb-patches review
 > >  > process, the GDB 6.1 packages for both of those distributions will
 > >  > include it.
 > >  
 > > I don't feel comfortable including it in FSF gdb6.1.  One strong
 > > reason is that there has been no design review (as opposed to
 > > tree-ssa).  If those distros give you more latitude (i.e. no review
 > > process, availability of 'unstable and 'stable' versions, etc), it is
 > > not a good reason for FSF to do the same.  I am not saying no a
 > > priori.  It's just unfortunate timing.
 > 
 > OK, this is perfectly reasonable.  It's a lot of new code.
 > 
 > The design is actually - I can't really say based on - but pretty much
 > follows suggestions given to me by Jim Blandy last year.  There was
 > definitely no design review of this code; that's because I just sat
 > down and did it.  The design is part of what I have to ask the
 > maintainers of the dwarf2 reader to review.
 > 


Thanks for understanding. To help streamline this, can I ask you to
post a little summary of the major changes you needed to do?  I am
thinking we should strat having a dwarf2 handling section in the doco
and this would be a starting point. 

Re: Branch created for inter-compilation-unit references

[Daniel Jacobowitz]

On Thu, Feb 26, 2004 at 11:32:00AM -0500, Elena Zannoni wrote:
> Thanks for understanding. To help streamline this, can I ask you to
> post a little summary of the major changes you needed to do?  I am
> thinking we should strat having a dwarf2 handling section in the doco
> and this would be a starting point. 

Sure.  Here's the bird's-eye view of the new code.


Partial symbol handling

Our existing partial symbol handling is one-pass; read in one DIE at a
time, grab whatever information we need from it, and then go on to
either its children or its sibling.  As David found out some time ago,
this is incompatible with DW_AT_specification/DW_AT_abstract_origin.
For C++, this means that it becomes very awkward to determine a class's
fully qualified name; it may have a non-defining declaration as the
child of a namespace and a defining declaration with
DW_AT_specification.  We have limited partial DIE support for these
attributes, which we can use to find the DW_AT_name in the referenced
DIE, but it is incompatible with processing multiple compilation units
at a time.

So to solve both of these problems, I implemented a two-pass algorithm
much like the one currently used for full symbol reading.  It's a
little slower, but I took pains to keep the impact minimal for code
that does not use namespaces and specifications.

Normally we process one compilation unit at a time.  When we first
encounter an inter-compilation-unit reference, we switch gears
(somewhat inelegantly): we build a table of all compilation unit
lengths and offsets, and store it in a splay tree.  Then we load
compilation units into memory as needed and free them when they've been
unused for a tunable amount of time.


Full symbol handling

The full symbol handling (with the bug fix I've posted for HEAD this
week) was much closer to supporting inter-compilation-unit references
than the partial symbol handling.  We know (from the partial symbol
table reading) whether or not we will need to support references.  If
we will, then for each compilation unit we cache die->type in a hash
table.  This allows us to recreate the type information for a
compilation unit without keeping the enter DIE tree in memory.

If we're in reference-supporting mode, we build full symbols using a
queue.  We add the current psymtab's CU to the queue; load full DIEs
into memory for everything on the queue; recreate any already read in
comp units using the saved die->type hash table; and process any not
yet read in comp units.  While loading full DIEs we queue any
referenced compilation units for reading.  All queued compilation units
will be read in before any are turned into symtabs.  I did it this way
because making the symtab-building code re-entrant (so that we could
build up two psymtabs at the same time) would have been very
challenging, but the DIE loading code is much simpler.  Also, it turns
out that we can have circular type references between compilation
units, which requires that we be able to build types in one pass and
symbols in another.

A couple of functions that return DIE pointers were adjusted to also
return the pointer to the compilation unit containing the DIE, so that
relative references could be followed.


I think that's pretty much everything.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Branch created for inter-compilation-unit references

[Eli Zaretskii]

> From: Elena Zannoni <ezannoni@redhat.com>
> Date: Thu, 26 Feb 2004 11:32:00 -0500
> 
> I am thinking we should strat having a dwarf2 handling section in
> the doco and this would be a starting point.

I will certainly applaud that!

Re: Branch created for inter-compilation-unit references

[Eli Zaretskii]

> Date: Thu, 26 Feb 2004 10:05:26 -0500
> From: Daniel Jacobowitz <drow@false.org>
> 
> Then both you and Andrew underestimate the intrusiveness of DW_OP_piece
> support.

I don't underestimate it, I simply don't know anything about it.
Frankly, I was relying on Andrew's opinion that it's good to be
included in 6.1.  That is why I said:

	 [...] DW_OP_piece might be a good thing [...]

Note the ``might'' part.

I do agree that if DW_OP_piece is intrusive, we should think hard
whether we want to include it at this time.

Re: Branch created for inter-compilation-unit references

[Andrew Cagney]

> On Thu, Feb 26, 2004 at 08:30:56AM +0200, Eli Zaretskii wrote:
> 
>>> It's not more important in general, but since we are preparing to cut
>>> the 6.1 branch in a few days, DW_OP_piece might be a good thing to do
>>> now, while delaying intercu-branch merge till after the release.
>>> It's a question of timing, not of an abstract importance.
> 
> 
> Then both you and Andrew underestimate the intrusiveness of DW_OP_piece
> support.

(I can't speak for Elena)   I think I've got a pretty good feel for how 
much work is involved in finishing (rather than prototyping) something 
like this.  Thats why I'm making noises about a DW_OP_piece hack for 6.1.

Andrew

Re: Branch created for inter-compilation-unit references

[Daniel Jacobowitz]

On Thu, Feb 26, 2004 at 02:24:46PM -0500, Andrew Cagney wrote:
> >On Thu, Feb 26, 2004 at 08:30:56AM +0200, Eli Zaretskii wrote:
> >
> >>>It's not more important in general, but since we are preparing to cut
> >>>the 6.1 branch in a few days, DW_OP_piece might be a good thing to do
> >>>now, while delaying intercu-branch merge till after the release.
> >>>It's a question of timing, not of an abstract importance.
> >
> >
> >Then both you and Andrew underestimate the intrusiveness of DW_OP_piece
> >support.
> 
> (I can't speak for Elena)   I think I've got a pretty good feel for how 
> much work is involved in finishing (rather than prototyping) something 
> like this.  Thats why I'm making noises about a DW_OP_piece hack for 6.1.

I haven't heard any of these noises?  In any case I don't think it's a
particularly good idea, unless all you mean is handling the resulting
error() call so that it doesn't abort symbol reading.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Branch created for inter-compilation-unit references

[Andrew Cagney]

> On Thu, Feb 26, 2004 at 02:24:46PM -0500, Andrew Cagney wrote:
> 
>>>> >On Thu, Feb 26, 2004 at 08:30:56AM +0200, Eli Zaretskii wrote:
>>>> >
>>>
>>>>>> >>>It's not more important in general, but since we are preparing to cut
>>>>>> >>>the 6.1 branch in a few days, DW_OP_piece might be a good thing to do
>>>>>> >>>now, while delaying intercu-branch merge till after the release.
>>>>>> >>>It's a question of timing, not of an abstract importance.
>>>
>>>> >
>>>> >
>>>> >Then both you and Andrew underestimate the intrusiveness of DW_OP_piece
>>>> >support.
>>
>>> 
>>> (I can't speak for Elena)   I think I've got a pretty good feel for how 
>>> much work is involved in finishing (rather than prototyping) something 
>>> like this.  Thats why I'm making noises about a DW_OP_piece hack for 6.1.
> 
> 
> I haven't heard any of these noises?  In any case I don't think it's a
> particularly good idea, unless all you mean is handling the resulting
> error() call so that it doesn't abort symbol reading.

Lets say hints with others making various noises
http://sources.redhat.com/ml/gdb/2004-02/msg00170.html
If we don't do something, how useful will GDB 6.1 be when GCC 3.4 is 
released?  Spending resources on adding ICU to the branch certainly 
won't fix that problem, sigh.

I'm thinking of something that at least stops GDB falling on its sword 
when this happens.

Andrew

Re: Branch created for inter-compilation-unit references

[Eli Zaretskii]

> From: Daniel Berlin <dberlin@dberlin.org>
> Date: Wed, 25 Feb 2004 12:07:16 -0500
> 
> >
> > A comparison is reasonable (and it isn't ment to degrate SSA or GCC). 
> > If the HP merge were to have been handled correctly it would have 
> > turned into a project of size and logistics comparable to SSA.  That, 
> > I think, is getting out of control.
> 
> Then you are sorely mistaken.
> If you think the SSA branch is an example of a  bad thing, i fear for 
> gdb development.

Don't fear.

I think you are reading too much into Andrew's comments, and therefore
miss his point.  As I understand what Andrew says, he simply is afraid
that merging large modifications brings a danger of repeating past
mistakes such as the HP merge.  I think the point is valid, and we
should be aware of that danger so that the problems don't repeat.