Re: [patch] build-id .debug files load (like .gnu_debuglink)

Eli Zaretskii
2007-09-01T10:31:39Z

> Date: Sat, 1 Sep 2007 10:19:34 +0200
> From: Jan Kratochvil <jan.kratochvil@redhat.com>
>
> On Fri, 31 Aug 2007 11:38:37 +0200, Eli Zaretskii wrote:
> > > Date: Sun, 26 Aug 2007 11:40:53 +0200
> > > From: Jan Kratochvil <jan.kratochvil@redhat.com>
> > > Cc: gdb-patches@sourceware.org, Roland McGrath <roland@redhat.com>
> > >
> > > 2007-08-26 Jan Kratochvil <jan.kratochvil@redhat.com>
> > >
> > > * gdb.texinfo (Separate Debug Files): Included a BUILD ID description.
> > > Enlisted BUILD ID to the debug file searching example.
> > > Included a BUILD ID `.note.gnu.build-id' section description.
> > > Updated/added the debug files splitting instructions for OBJCOPY.
> >
> > Thanks. The patch for the manual needs some work on the wording.
> > Please commit it (assuming that the code patch is approved), and I
> > will then improve it and post the changes here so you will see what
> > I've done.
>
> Just a notification the patch has been committed.

Thanks.

I attach below the changes I committed to the manual, to fix the
description of how GDB looks for debug info files when debug ID is
supported. Because the rewording is so extensive, I also attach below
the entire section in its new form, to facilitate reading.

I ended up to be a bit confused about several issues related to debug
ID; if you (or someone else, maybe Roland) can shed some light on
them, I might be able to improve the section in a couple of additional
areas where for now I left things a bit vague or even inaccurate:

. The Fedora site that describes the build ID features seems to say
that there are TWO files (actually symlinks) in the global debug
directory for each executable: ab/cdef1234 and ab/cdef1234.debug.
By contrast, you only talk about a single file. What is the
truth, and if the second file exists, how, if at all, does it
affect GDB?

. The Fedora site says that the build ID symlinks are created only
in the global debug directory. However, your patch seems to say
that GDB looks for these symlinks in all the other possible
locations as well:

@value{GDBN} checks under each of these names for a debugging
+information file with build id content matching the build id content of the
+executable file - or - whose checksum matches the one given in the link in the
+debug link case.

Which one is true?

. The objcopy commands shown in your patch seem to be relevant only
to the ``debug link'' method (at least that's what I understand
from the last objcopy command). If so, I think we should say what
are the corresponding commands for the ``debug ID'' method.
Likewise, if there are (or going to be) features in elfutils that
support ``debug ID'', I think we should mention them, as we do for
``debug links'' now.

. I'm confused by your mentioning of ``all files'' in this passage:

+@dfn{build id} is present in all the files [...]

as opposed to ``only the executable'' in the debug link case:

+@dfn{debug link} is present only in the executable [...]

I don't understand this distinction. As far as I could glean from
the build ID description on the Fedora site, the difference is
that the build ID is present both in the executable and in the
debug info file, whereas the debug link is present only in the
executable. Thus we have two files (maybe 3, if core dumps are
counted), vs one, not ``only one'' vs ``all''. Did I miss
something?

Finally, here are the patch followed by the full section.

Please feel free to ask about any of the changes I made.

Btw, I think this change warrants an entry in NEWS.

Thanks again for working on this.

---------------- The patch -------------------------

Index: gdb.texinfo
===================================================================
RCS file: /cvs/src/src/gdb/doc/gdb.texinfo,v
retrieving revision 1.425
diff -u -r1.425 gdb.texinfo
--- gdb.texinfo 1 Sep 2007 08:17:13 -0000 1.425
+++ gdb.texinfo 1 Sep 2007 09:57:16 -0000
@@ -11893,66 +11893,79 @@
@cindex @file{.debug} subdirectories
@cindex debugging information directory, global
@cindex global debugging information directory
+@cindex build ID, and separate debugging files
+@cindex @file{.build-id} directory

@value{GDBN} allows you to put a program's debugging information in a
file separate from the executable itself, in a way that allows
@value{GDBN} to find and load the debugging information automatically.
-Since debugging information can be very large --- sometimes larger
-than the executable code itself --- some systems distribute debugging
+Since debugging information can be very large---sometimes larger
+than the executable code itself---some systems distribute debugging
information for their executables in separate files, which users can
install only when they need to debug a problem.

-There are two identificators how the separate debug file may be found:
+@value{GDBN} supports two ways of specifying the separate debug info
+file:

@itemize @bullet
@item
-@dfn{debug link} is present only in the executable if its debug information has
-been split out. It is not present in the separate debug file. It provides the
-separate debug file filename, usually as @file{executable.debug}.
-@item
-@dfn{build id} is present in all the files (if the operating system supports
-it). The executable file and its separate debug file have the same unique
-@dfn{build id} content.
+The executable contains a @dfn{debug link} that specifies the name of
+the separate debug info file. The separate debug file's name is
+usually @file{@var{executable}.debug}, where @var{executable} is the
+name of the corresponding executable file without leading directories
+(e.g., @file{ls.debug} for @file{/usr/bin/ls}). In addition, the
+debug link specifies a CRC32 checksum for the debug file, which
+@value{GDBN} uses to validate that the executable and the debug file
+came from the same build.
+
+@item
+The executable contains a @dfn{build ID}, a unique signature that is
+also present in the corresponding debug info file. (This is supported
+only on some operating systems, notably on @sc{gnu}/Linux. For more
+details about this feature, see
+@uref{http://fedoraproject.org/wiki/Releases/FeatureBuildId, the
+Fedora Project's description of the buid ID feature}.) The debug info
+file's name is not specified explicitly by the debug ID, but can be
+computed from the build ID, see below.
@end itemize

-If the full name of the directory containing the executable is @var{execdir},
-the executable has a debug link that specifies the name @var{debugfile},
-@var{bu} is the first byte (two hexadecimal characters) of the build id
-content, @var{ild-id} are the remaining bytes / hexadecimal characters and
-@var{globaldebugdir} is the global debug file directory then @value{GDBN} will
-automatically search for the debugging information file in four places:
+Depending on the way the debug info file is specified, @value{GDBN}
+uses two different methods of looking for the debug file:

@itemize @bullet
@item
-a specific file in the subdirectory of the global debug file directory
-according to the @dfn{build id} content (if present), the file tried is
-@file{@var{globaldebugdir}/.debug-id/@var{bu}/@var{ild-id}.debug}.
-@item
-the directory containing the executable file (that is, it will look
-for a file named @file{@var{execdir}/@var{debugfile}},
-@item
-a subdirectory of that directory named @file{.debug} (that is, the
-file @file{@var{execdir}/.debug/@var{debugfile}}, and
-@item
-a subdirectory of the global debug file directory that includes the
-executable's full path, and the name from the link (that is, the file
-@file{@var{globaldebugdir}/@var{execdir}/@var{debugfile}}, where
-@var{globaldebugdir} is the global debug file directory, and
-@var{execdir} has been turned into a relative path).
+For the ``debug link'' method, @value{GDBN} looks up the named file in
+the directory of the executable file, then in a subdirectory of that
+directory named @file{.debug}, and finally under the global debug
+directory, in a subdirectory whose name is identical to the leading
+directories of the executable's absolute file name.
+
+@item
+For the ``debug ID'' method, @value{GDBN} looks in the
+@file{.build-id} subdirectory of the global debug directory for a file
+named @file{@var{nn}/@var{nnnnnnnn}.debug}, where @var{nn} are the
+first 2 hex characters of the debug ID signature, and @var{nnnnnnnn}
+are the rest of the signature. (Real signatures are 32 or more
+characters, not 10.)
+@end itemize
+
+So, for example, suppose you ask @value{GDBN} to debug
+@file{/usr/bin/ls}, which has a @dfn{debug link} that specifies the
+file @file{ls.debug}, and a @dfn{build id} whose value in hex is
+@code{abcdef1234}. If the global debug directory is
+@file{/usr/lib/debug}, then @value{GDBN} will look for the following
+debug information files, in the indicated order:
+
+@itemize @minus
+@item
+@file{/usr/lib/debug/.build-id/ab/cdef1234.debug}
+@item
+@file{/usr/bin/ls.debug}
+@item
+@file{/usr/bin/.debug/ls.debug}
+@item
+@file{/usr/lib/debug/usr/bin/ls.debug}.
@end itemize
-@noindent
-@value{GDBN} checks under each of these names for a debugging
-information file with build id content matching the build id content of the
-executable file - or - whose checksum matches the one given in the link in the
-debug link case. In each case @value{GDBN} reads the debugging information
-from the first debug file it finds.
-
-So, for example, if you ask @value{GDBN} to debug @file{/usr/bin/ls}, which has
-a @dfn{debug link} containing the name @file{ls.debug}, its @dfn{build id}
-value in hexadecimal is @code{abcdef} and the global debug directory is
-@file{/usr/lib/debug}, then @value{GDBN} will look for debug information in
-@file{/usr/lib/debug/.build-id/ab/cdef.debug}, @file{/usr/bin/ls.debug},
-@file{/usr/bin/.debug/ls.debug}, and @file{/usr/lib/debug/usr/bin/ls.debug}.

You can set the global debugging info directory's name, and view the
name @value{GDBN} is currently using.
@@ -11972,7 +11985,7 @@
@end table

@cindex @code{.gnu_debuglink} sections
-@cindex debug links
+@cindex debug link sections
A debug link is a special section of the executable file named
@code{.gnu_debuglink}. The section must contain:

@@ -11995,37 +12008,46 @@
described above.

@cindex @code{.note.gnu.build-id} sections
-@cindex build id
-Build id is a special section of the executable file named
-@code{.note.gnu.build-id}. The section contains unique identification hash
-derived from the built files - it remains the same across multiple builds of
-the same build tree. The default algorithm SHA1 produces 160 bits (40
-hexadecimal characters) of the content. The same section and value is present
-in the original built binary with symbols, in its stripped variant and in the
-separate debug information file.
+@cindex build ID sections
+A build ID is a special section of the executable file named
+@code{.note.gnu.build-id}. This section contains unique
+identification for the built files---it remains the same across
+multiple builds of the same build tree. The default algorithm SHA1
+produces 160 bits (40 hexadecimal characters) of the content. The
+same section with an identical value is present in the original built
+binary with symbols, in its stripped variant, and in the separate
+debugging information file.

The debugging information file itself should be an ordinary
executable, containing a full set of linker symbols, sections, and
debugging information. The sections of the debugging information file
-should have the same names, addresses and sizes as the original file,
-but they need not contain any data --- much like a @code{.bss} section
+should have the same names, addresses, and sizes as the original file,
+but they need not contain any data---much like a @code{.bss} section
in an ordinary executable.

-@sc{gnu} binary utilities contain the @samp{objcopy} utility able to produce
-the separated executable / debugging information file pairs by commands
-@kbd{objcopy --only-keep-debug foo foo.debug; strip -g foo; objcopy
---add-gnu-debuglink="foo.debug" "foo"}. These commands remove the debugging
+@sc{gnu} binary utilities (Binutils) package includes the
+@samp{objcopy} utility that can produce
+the separated executable / debugging information file pairs using the
+following commands:
+
+@example
+@kbd{objcopy --only-keep-debug foo foo.debug}
+@kbd{strip -g foo}
+@kbd{objcopy --add-gnu-debuglink="foo.debug" "foo"}
+@end example
+
+@noindent
+These commands remove the debugging
information from the executable file @file{foo}, place it in the file
@file{foo.debug}, and leave behind a debug link in @file{foo}. Ulrich
Drepper's @file{elfutils} package, starting with version 0.53, contains
a version of the @code{strip} command such that the command @kbd{strip foo -f
foo.debug} has the same functionality as the three commands above.

-Since there are many different ways to compute CRC's for the debug link
-(different polynomials, reversals, byte ordering, etc.). This computation does
-not apply to the build id section. The simplest way to describe the CRC used
-in @code{.gnu_debuglink} sections is to give the complete code for a function
-that computes it:
+Since there are many different ways to compute CRC's for the debug
+link (different polynomials, reversals, byte ordering, etc.), the
+simplest way to describe the CRC used in @code{.gnu_debuglink}
+sections is to give the complete code for a function that computes it:

@kindex gnu_debuglink_crc32
@smallexample
@@ -12097,6 +12119,9 @@
@}
@end smallexample

+@noindent
+This computation does not apply to the ``build ID'' method.
+

@node Symbol Errors
@section Errors Reading Symbol Files

----------------- The full text of the section ------------------

@node Separate Debug Files
@section Debugging Information in Separate Files
@cindex separate debugging information files
@cindex debugging information in separate files
@cindex @file{.debug} subdirectories
@cindex debugging information directory, global
@cindex global debugging information directory
@cindex build ID, and separate debugging files
@cindex @file{.build-id} directory

@value{GDBN} allows you to put a program's debugging information in a
file separate from the executable itself, in a way that allows
@value{GDBN} to find and load the debugging information automatically.
Since debugging information can be very large---sometimes larger
than the executable code itself---some systems distribute debugging
information for their executables in separate files, which users can
install only when they need to debug a problem.

@value{GDBN} supports two ways of specifying the separate debug info
file:

@itemize @bullet
@item
The executable contains a @dfn{debug link} that specifies the name of
the separate debug info file. The separate debug file's name is
usually @file{@var{executable}.debug}, where @var{executable} is the
name of the corresponding executable file without leading directories
(e.g., @file{ls.debug} for @file{/usr/bin/ls}). In addition, the
debug link specifies a CRC32 checksum for the debug file, which
@value{GDBN} uses to validate that the executable and the debug file
came from the same build.

@item
The executable contains a @dfn{build ID}, a unique signature that is
also present in the corresponding debug info file. (This is supported
only on some operating systems, notably on @sc{gnu}/Linux. For more
details about this feature, see
@uref{http://fedoraproject.org/wiki/Releases/FeatureBuildId, the
Fedora Project's description of the buid ID feature}.) The debug info
file's name is not specified explicitly by the debug ID, but can be
computed from the build ID, see below.
@end itemize

Depending on the way the debug info file is specified, @value{GDBN}
uses two different methods of looking for the debug file:

@itemize @bullet
@item
For the ``debug link'' method, @value{GDBN} looks up the named file in
the directory of the executable file, then in a subdirectory of that
directory named @file{.debug}, and finally under the global debug
directory, in a subdirectory whose name is identical to the leading
directories of the executable's absolute file name.

@item
For the ``debug ID'' method, @value{GDBN} looks in the
@file{.build-id} subdirectory of the global debug directory for a file
named @file{@var{nn}/@var{nnnnnnnn}.debug}, where @var{nn} are the
first 2 hex characters of the debug ID signature, and @var{nnnnnnnn}
are the rest of the signature. (Real signatures are 32 or more
characters, not 10.)
@end itemize

So, for example, suppose you ask @value{GDBN} to debug
@file{/usr/bin/ls}, which has a @dfn{debug link} that specifies the
file @file{ls.debug}, and a @dfn{build id} whose value in hex is
@code{abcdef1234}. If the global debug directory is
@file{/usr/lib/debug}, then @value{GDBN} will look for the following
debug information files, in the indicated order:

@itemize @minus
@item
@file{/usr/lib/debug/.build-id/ab/cdef1234.debug}
@item
@file{/usr/bin/ls.debug}
@item
@file{/usr/bin/.debug/ls.debug}
@item
@file{/usr/lib/debug/usr/bin/ls.debug}.
@end itemize

You can set the global debugging info directory's name, and view the
name @value{GDBN} is currently using.

@table @code

@kindex set debug-file-directory
@item set debug-file-directory @var{directory}
Set the directory which @value{GDBN} searches for separate debugging
information files to @var{directory}.

@kindex show debug-file-directory
@item show debug-file-directory
Show the directory @value{GDBN} searches for separate debugging
information files.

@end table

@cindex @code{.gnu_debuglink} sections
@cindex debug link sections
A debug link is a special section of the executable file named
@code{.gnu_debuglink}. The section must contain:

@itemize
@item
A filename, with any leading directory components removed, followed by
a zero byte,
@item
zero to three bytes of padding, as needed to reach the next four-byte
boundary within the section, and
@item
a four-byte CRC checksum, stored in the same endianness used for the
executable file itself. The checksum is computed on the debugging
information file's full contents by the function given below, passing
zero as the @var{crc} argument.
@end itemize

Any executable file format can carry a debug link, as long as it can
contain a section named @code{.gnu_debuglink} with the contents
described above.

@cindex @code{.note.gnu.build-id} sections
@cindex build ID sections
A build ID is a special section of the executable file named
@code{.note.gnu.build-id}. This section contains unique
identification for the built files---it remains the same across
multiple builds of the same build tree. The default algorithm SHA1
produces 160 bits (40 hexadecimal characters) of the content. The
same section with an identical value is present in the original built
binary with symbols, in its stripped variant, and in the separate
debugging information file.

The debugging information file itself should be an ordinary
executable, containing a full set of linker symbols, sections, and
debugging information. The sections of the debugging information file
should have the same names, addresses, and sizes as the original file,
but they need not contain any data---much like a @code{.bss} section
in an ordinary executable.

@sc{gnu} binary utilities (Binutils) package includes the
@samp{objcopy} utility that can produce
the separated executable / debugging information file pairs using the
following commands:

@example
@kbd{objcopy --only-keep-debug foo foo.debug}
@kbd{strip -g foo}
@kbd{objcopy --add-gnu-debuglink="foo.debug" "foo"}
@end example

@noindent
These commands remove the debugging
information from the executable file @file{foo}, place it in the file
@file{foo.debug}, and leave behind a debug link in @file{foo}. Ulrich
Drepper's @file{elfutils} package, starting with version 0.53, contains
a version of the @code{strip} command such that the command @kbd{strip foo -f
foo.debug} has the same functionality as the three commands above.

Since there are many different ways to compute CRC's for the debug
link (different polynomials, reversals, byte ordering, etc.), the
simplest way to describe the CRC used in @code{.gnu_debuglink}
sections is to give the complete code for a function that computes it:

@kindex gnu_debuglink_crc32
@smallexample
unsigned long
gnu_debuglink_crc32 (unsigned long crc,
unsigned char *buf, size_t len)
@{
static const unsigned long crc32_table[256] =
@{
0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419,
0x706af48f, 0xe963a535, 0x9e6495a3, 0x0edb8832, 0x79dcb8a4,
0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07,
0x90bf1d91, 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de,
0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, 0x136c9856,
0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9,
0xfa0f3d63, 0x8d080df5, 0x3b6e20c8, 0x4c69105e, 0xd56041e4,
0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3,
0x45df5c75, 0xdcd60dcf, 0xabd13d59, 0x26d930ac, 0x51de003a,
0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599,
0xb8bda50f, 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924,
0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, 0x76dc4190,
0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f,
0x9fbfe4a5, 0xe8b8d433, 0x7807c9a2, 0x0f00f934, 0x9609a88e,
0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01,
0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed,
0x1b01a57b, 0x8208f4c1, 0xf50fc457, 0x65b0d9c6, 0x12b7e950,
0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3,
0xfbd44c65, 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2,
0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, 0x4369e96a,
0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5,
0xaa0a4c5f, 0xdd0d7cc9, 0x5005713c, 0x270241aa, 0xbe0b1010,
0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17,
0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, 0xedb88320, 0x9abfb3b6,
0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615,
0x73dc1683, 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8,
0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, 0xf00f9344,
0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb,
0x196c3671, 0x6e6b06e7, 0xfed41b76, 0x89d32be0, 0x10da7a5a,
0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5,
0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1,
0xa6bc5767, 0x3fb506dd, 0x48b2364b, 0xd80d2bda, 0xaf0a1b4c,
0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef,
0x4669be79, 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236,
0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, 0xc5ba3bbe,
0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31,
0x2cd99e8b, 0x5bdeae1d, 0x9b64c2b0, 0xec63f226, 0x756aa39c,
0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713,
0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b,
0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, 0x86d3d2d4, 0xf1d4e242,
0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1,
0x18b74777, 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c,
0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, 0xa00ae278,
0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7,
0x4969474d, 0x3e6e77db, 0xaed16a4a, 0xd9d65adc, 0x40df0b66,
0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605,
0xcdd70693, 0x54de5729, 0x23d967bf, 0xb3667a2e, 0xc4614ab8,
0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b,
0x2d02ef8d
@};
unsigned char *end;

crc = ~crc & 0xffffffff;
for (end = buf + len; buf < end; ++buf)
crc = crc32_table[(crc ^ *buf) & 0xff] ^ (crc >> 8);
return ~crc & 0xffffffff;
@}
@end smallexample

@noindent
This computation does not apply to the ``build ID'' method.