Re: [Dwarf-Discuss] RFC: DWARF Extensions for Separate Debug Info Files ("Fission")

John DelSignore
2011-09-23T21:35:39Z

Hi Jason,

Jason Molenda wrote:
> On Sep 23, 2011, at 10:58 AM, Cary Coutant wrote:
>
>>> The compiler puts DWARF in the .o file, the linker adds some records in the executable which help us to understand where files/function/symbols landed in the final executable[1].
>> Did you intend to add a footnote?
>
> Yeah, I realized after I sent the email - it didn't seem interesting enough to warrant a separate followup.
>
> The records that our linker puts in the executable are in the form of stabs entries. There are a handful of stabs records created - file start, file end, function start, function end, symbol, pointer to a .o file, maybe one or two others. We chose that format because it was trivial to support and we already had tools for stripping these records out of the executable once the dSYM had been created.

I don't remember the exact details, but the problem I recall with the Darwin scheme is that it builds an incomplete index in the Mach-O symbol table. IIRC, it was missing things that a user might want to lookup by-name in the debugger, like static functions or variables, and type names with external linkage. Without a reasonably complete index, the debugger can't know where to find the definitions of certain things, and that forces the user to navigate using other information, like source file name or global function definitions to force the debug information in the object to be read.

Of course, the current DWARF indexes (like pubnames/pubtypes) have the same problem, and some compilers do a really bad job at generating those sections. But at least when there's a single .debug_info section, the debugger can decide to ignore the indexes and "skim" the full debug information. The compilers on IRIX did a better job at generating indexes, so the debugger could find by-name static functions/objects.

> Once a dSYM has been created with all of the DWARF collected in a single file, our DWARF is parseable by any debug info consumer with minimal changes -- they need to know to look in a separate file for the DWARF from the main executable, but the format itself is unchanged. Supporting the debug-information-in-.o-files is more involved, I don't know if any of the third-party debuggers on our platform work with it.

TotalView supports debug information in .o files on Darwin, and has since day one. Perhaps you recall all those email exchanges you and I had several years back. It was a modest amount of work, given that we already supported debug information in .o files on the Sun and HP platforms.

I seem to recall one of the sore spots for us on Dawrin was getting good address information for certain DWARF location operations, like DW_OP_addr. Fortran was a particularly messy because some compilers didn't supply a linkage name attribute, so the debugger had to make several guesses at the name, and look things up by trial and error.

Cheers, John D.

>> We're trying to achieve something very similar, but we have the
>> additional goal of separating the info from the .o files because of
>> our distributed build environment. I also wanted to attempt to
>> standardize the approach, instead of having each vendor go in separate
>> directions.
>
>
> Yeah, if your regular build environment involves distributed compilation, and the .o files need to be copied to a central system for the linker, then I can see why you're pursuing this approach. For us, the most common usage is single-computer compilation & linking -- where the linker never pages in the debug info sections from the .o files so their size is not particular important.
>
> J
> _______________________________________________
> Dwarf-Discuss mailing list
> Dwarf-Discuss@lists.dwarfstd.org
> http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org
>