GDB not able to debug files(dwarf2.0) loaded using add-symbol-file

GDB not able to debug files(dwarf2.0) loaded using add-symbol-file

[Sandeep Joshi]

PROBLEM :
----------------
When we use add-symbol-file option to debug more than one executables,
the sources are not shown correctly. This happens if we build with
dwarf2.0 debug format. Although GDB is able the debug correctly, it
doesn't show correct sources.


HOW :
-------
I have tried to remotely debug an application running on arm board
using the add-symbol-file command. We have a gdb stub on the target
which gets control on bootup. We have changed the trap handler and
using a specific undefined instruction(Hard Breakpoint) to pass
control to stub on target. After trap initialization, we have put Hard
Breakpoint in the kernel code. Below are the steps :

1) reboot the board. GDB specific undefined instruction is hit and the
stub waits for GDB to connect.
2) start gdb on the host and connect to target.
3) load vmlinux for kernel debugging
4) give 'continue' command so that the kernel is loaded and we get the prompt.
5) put a Hard Breakpoint in the executable and run it. The hard
breakpoint is hit and gdb on the host has control.
6) use add-symbol-file command to load the application symbol table.

WHY :
-------
The problem is in the way GDB performs symbol lookup using PC. In
function 'find_pc_sect_psymtab' gdb is not performing symbol lookup in
all the objfiles. If it finds the PC in the range of any partial
symtab, it tries to find the partial symtab that contains a symbol
whose address is closest to the PC address in that object file only
and returns that partial symbol table.

Now if the user puts a breakpoint in the file loaded using
add-symbol-file, then in function 'find_pc_sect_psymtab' the PC
Address might in the range of some psymtab in the first objfile
(vmlinux) and the best match in that file is returned. This is
happening because the code range of partial symtabs often overlap,
mostly if the functions are reordered.

SOLUTION :
----------------
Function 'find_pc_sect_psymtab' has to be modified to traverse all
objfiles before returning the best match. Below is the patch for the
same. This patch was made with
insight-weekly-6.30.50.20051128.tar.bz2. Kindly review the patch and
let me know if it can be added to gdb distribution.

diff -ru -P src_original/gdb/symtab.c src/gdb/symtab.c

--- src_original/gdb/symtab.c       2006-12-21 17:51:24.000000000 +0530

+++ src/gdb/symtab.c    2006-12-21 17:32:48.000000000 +0530

@@ -717,6 +717,11 @@

   struct partial_symtab *pst;

   struct objfile *objfile;

   struct minimal_symbol *msymbol;

+  struct partial_symtab *tpst;

+  struct partial_symtab *best_pst = NULL;

+  struct partial_symbol *best_psym = NULL;

+  int match_found_in_prev_objfile = 0;

+



   /* If we know that this is not a text address, return failure.  This is

      necessary because we loop based on texthigh and textlow, which do

@@ -730,13 +735,18 @@

              || msymbol->type == mst_file_bss))

     return NULL;



-  ALL_PSYMTABS (objfile, pst)

+  ALL_OBJFILES (objfile)

   {

+   ALL_OBJFILE_PSYMTABS (objfile, pst)

+   {

     if (pc >= pst->textlow && pc < pst->texthigh)

       {

-           struct partial_symtab *tpst;

-           struct partial_symtab *best_pst = pst;

-           struct partial_symbol *best_psym = NULL;

+                      if(!match_found_in_prev_objfile)

+                      {

+                                  best_pst = pst;

+                                  best_psym = NULL;

+                                  match_found_in_prev_objfile = 1;

+                      }



            /* An objfile that has its functions reordered might have

               many partial symbol tables containing the PC, but

@@ -793,10 +803,17 @@



                  }

              }

-           return (best_pst);

-      }

+            /* all partial symtabs in the current objfile traversed,

+               look for psymtabs of next objfile, if any. */

+            break;

+            }

+          }

   }

-  return (NULL);

+

+  if(best_pst)

+            return (best_pst);

+  else

+            return (NULL);

}



/* Find which partial symtab contains PC.  Return 0 if none.


REFERENCES
----------------------
Below are some references which were of great help while investigating
this problem. Thanks to all.
http://www.cygwin.com/ml/gdb-patches/2001-10/msg00304.html
http://sourceware.org/ml/gdb-patches/2006-12/msg00005.html
http://www.informatik.uni-frankfurt.de/doc/texi/gdbint_11.html


Thanks and Regards
Sandeep Joshi

Re: GDB not able to debug files(dwarf2.0) loaded using add-symbol-file

[Daniel Jacobowitz]

On Fri, Dec 22, 2006 at 09:51:20AM +0530, Sandeep Joshi wrote:
> The problem is in the way GDB performs symbol lookup using PC. In
> function 'find_pc_sect_psymtab' gdb is not performing symbol lookup in
> all the objfiles. If it finds the PC in the range of any partial
> symtab, it tries to find the partial symtab that contains a symbol
> whose address is closest to the PC address in that object file only
> and returns that partial symbol table.
> 
> Now if the user puts a breakpoint in the file loaded using
> add-symbol-file, then in function 'find_pc_sect_psymtab' the PC
> Address might in the range of some psymtab in the first objfile
> (vmlinux) and the best match in that file is returned. This is
> happening because the code range of partial symtabs often overlap,
> mostly if the functions are reordered.

That would slow this already slow function down even further.  What
does your memory layout look like to create this problem?

-- 
Daniel Jacobowitz
CodeSourcery

Re: GDB not able to debug files(dwarf2.0) loaded using add-symbol-file

[Sandeep Joshi]

Memory layout:
        Entry point: 0x40018000
        0x40018000 - 0x40060000 is .init
        0x40060000 - 0x40259644 is .text
        0x40259650 - 0x40259f78 is __ex_table
        0x40259f78 - 0x4025a278 is .pci_fixup
        0x4025a278 - 0x4025d8f0 is __ksymtab
        0x4025d8f0 - 0x4025ddf0 is __ksymtab_gpl
        0x4025ddf0 - 0x40266070 is __ksymtab_strings
        0x40266070 - 0x40266174 is __param
        0x40268000 - 0x402b8c10 is .data
        0x402b8c20 - 0x402d995c is .bss

add-symbol-file ~/arm-linux-gdb/bin/test_trap4.out 0x83d8



On 12/24/06, Daniel Jacobowitz <drow@false.org> wrote:
> On Fri, Dec 22, 2006 at 09:51:20AM +0530, Sandeep Joshi wrote:
> > The problem is in the way GDB performs symbol lookup using PC. In
> > function 'find_pc_sect_psymtab' gdb is not performing symbol lookup in
> > all the objfiles. If it finds the PC in the range of any partial
> > symtab, it tries to find the partial symtab that contains a symbol
> > whose address is closest to the PC address in that object file only
> > and returns that partial symbol table.
> >
> > Now if the user puts a breakpoint in the file loaded using
> > add-symbol-file, then in function 'find_pc_sect_psymtab' the PC
> > Address might in the range of some psymtab in the first objfile
> > (vmlinux) and the best match in that file is returned. This is
> > happening because the code range of partial symtabs often overlap,
> > mostly if the functions are reordered.
>
> That would slow this already slow function down even further.  What
> does your memory layout look like to create this problem?
>
> --
> Daniel Jacobowitz
> CodeSourcery
>

Re: GDB not able to debug files(dwarf2.0) loaded using add-symbol-file

[Daniel Jacobowitz]

On Wed, Jan 03, 2007 at 03:14:30PM +0530, Sandeep Joshi wrote:
> Memory layout:
>        Entry point: 0x40018000
>        0x40018000 - 0x40060000 is .init
>        0x40060000 - 0x40259644 is .text
>        0x40259650 - 0x40259f78 is __ex_table
>        0x40259f78 - 0x4025a278 is .pci_fixup
>        0x4025a278 - 0x4025d8f0 is __ksymtab
>        0x4025d8f0 - 0x4025ddf0 is __ksymtab_gpl
>        0x4025ddf0 - 0x40266070 is __ksymtab_strings
>        0x40266070 - 0x40266174 is __param
>        0x40268000 - 0x402b8c10 is .data
>        0x402b8c20 - 0x402d995c is .bss
> 
> add-symbol-file ~/arm-linux-gdb/bin/test_trap4.out 0x83d8

Then how does that overlap with the symbols for the kernel?  It
shouldn't.

-- 
Daniel Jacobowitz
CodeSourcery

Re: GDB not able to debug files(dwarf2.0) loaded using add-symbol-file

[Sandeep Joshi]

Below is what 'find_pc_sect_psymtab' returned when i tried to set
breakpoint in the file added with 'add-symbol-file' command.

******************************************
Breakpoint 5, find_pc_sect_psymtab (pc=34148, section=0x9cd4324) at symtab.c:796
796             return (best_pst);

(gdb) p *best_pst
$2 = {next = 0x9ad046c, filename = 0x9ad0518
"net/sunrpc/auth_gss/gss_krb5_mech.c", fullname = 0x0,
  dirname = 0x99f1e78
"/home/joshis/sdk/usr/src/kernel/linux-2.6.11.12", objfile =
0x994b650,
  section_offsets = 0x9951800, textlow = 288, texthigh = 1076066628,
dependencies = 0x0, number_of_dependencies = 0,
  globals_offset = 4596, n_global_syms = 0, statics_offset = 149919,
n_static_syms = 259, symtab = 0x9ccdc50,
  read_symtab = 0x80fbf84 <dwarf2_psymtab_to_symtab>,
read_symtab_private = 0x9a01b44 "óvø", readin = 1 '\001'}

******************************************
Here the pc to be looked is is '34148' . It starts with first object
file i.e vmlinux and checks all the psymtabs for their code range. Now
for the above psymtab, Code range is 'textlow = 288, texthigh =
1076066628' and our PC lies in this. So this is set as best_pst and
function returns without checking the second objfile, which has the
perfect match for this PC. That is why gdb is not able to show the
sources correct.



On 1/4/07, Daniel Jacobowitz <drow@false.org> wrote:
> On Wed, Jan 03, 2007 at 03:14:30PM +0530, Sandeep Joshi wrote:
> > Memory layout:
> >        Entry point: 0x40018000
> >        0x40018000 - 0x40060000 is .init
> >        0x40060000 - 0x40259644 is .text
> >        0x40259650 - 0x40259f78 is __ex_table
> >        0x40259f78 - 0x4025a278 is .pci_fixup
> >        0x4025a278 - 0x4025d8f0 is __ksymtab
> >        0x4025d8f0 - 0x4025ddf0 is __ksymtab_gpl
> >        0x4025ddf0 - 0x40266070 is __ksymtab_strings
> >        0x40266070 - 0x40266174 is __param
> >        0x40268000 - 0x402b8c10 is .data
> >        0x402b8c20 - 0x402d995c is .bss
> >
> > add-symbol-file ~/arm-linux-gdb/bin/test_trap4.out 0x83d8
>
> Then how does that overlap with the symbols for the kernel?  It
> shouldn't.
>
> --
> Daniel Jacobowitz
> CodeSourcery
>

Re: GDB not able to debug files(dwarf2.0) loaded using add-symbol-file

[Daniel Jacobowitz]

On Thu, Jan 04, 2007 at 12:26:51PM +0530, Sandeep Joshi wrote:
> Here the pc to be looked is is '34148' . It starts with first object
> file i.e vmlinux and checks all the psymtabs for their code range. Now
> for the above psymtab, Code range is 'textlow = 288, texthigh =
> 1076066628' and our PC lies in this. So this is set as best_pst and
> function returns without checking the second objfile, which has the
> perfect match for this PC. That is why gdb is not able to show the
> sources correct.

Aha.  OK, this is actually a different bug - the range should be set
that way.  Unfortunately, this is a very hard bug to fix in GDB.  I
recently patched GNU ld to prevent this from happening in new releases;
the patch is here:

  http://sourceware.org/ml/binutils/2006-11/msg00019.html

If you can apply that to your linker, I believe it will fix up the
problem, which comes from /DISCARD/ { *(.text.exit) } in the kernel's
linker scripts.

If you can't change your linker, it should be possible to detect the
problem in GDB.  Where we currently check for has_section_at_zero,
in dwarf2read.c, check instead that the address is in some section in
the file.

-- 
Daniel Jacobowitz
CodeSourcery