remote debugging symbol problem

remote debugging symbol problem

[David Antliff]

Hello,

We use gdb-6.0 and a PowerPC port of gdbserver to remotely debug our 
target application on it's PowerPC platform. Our application forks several 
(20 or so) child processes shortly after starting up, and a few of these 
child processes create threads.

Until quite recently, this has all worked fine. I was able to invoke:

# gdbserver 0.0.0.0:2345 --attach <pid>  for any of the child processes, 
and then run on my development host (x86 linux):

$ cd <application-src-dir>
$ gdb <application-binary>

I am 100% sure the same application binary is running on the host as on 
the target (I'm using NFS).

Normally, once GDB is pointed at the target, the process is interrupted 
and GDB can show a stack trace at the point of interruption.

However, now, I just get this:

(gdb) target remote 10.16.10.237:2345
Remote debugging using 10.16.10.237:2345
0x0fe896a0 in ?? ()
warning: Unable to find dynamic linker breakpoint function.
GDB will be unable to debug shared library initializers
and track explicitly loaded dynamic code.

My understanding is that the warning is simply because the standard 
libraries aren't compiled with GDB support. This used to happen even when 
it worked so I'm not concerned about it (should I be?).

That address 0x0fe896a0 is always the same regardless of which child 
process I attach to. There is no backtrace available (just prints out that 
address and ?? again). I used the info commands to check the symbol table 
is correct, and it is. Also:


(gdb) info target
Symbols from "<application-src-dir>".
Remote serial target in gdb-specific protocol:
Debugging a target over a serial line.
         While running this, GDB does not access memory from...
Local exec file:
         `<application-src-dir>/<application-binary>', 
file type 
elf32-powerpc.
         Entry point: 0x10002aa8
         0x10000114 - 0x10000121 is .interp
         0x10000130 - 0x10000150 is .note.ABI-tag
         0x10000150 - 0x1000081c is .hash
         0x1000081c - 0x100016dc is .dynsym
         0x100016dc - 0x10001ec9 is .dynstr
         0x10001eca - 0x100020a2 is .gnu.version
         0x100020a4 - 0x10002144 is .gnu.version_r
         0x10002144 - 0x10002174 is .rela.sbss
         0x10002174 - 0x10002a80 is .rela.plt
         0x10002a80 - 0x10002aa8 is .init
         0x10002aa8 - 0x1014ba9c is .text
         0x1014ba9c - 0x1014babc is .fini
         0x1014bac0 - 0x101be660 is .rodata
         0x101ce660 - 0x102172bc is .data
         0x102172bc - 0x102172c0 is .eh_frame
         0x102172c0 - 0x102172d8 is .got2
         0x102172d8 - 0x102172e0 is .ctors
         0x102172e0 - 0x102172e8 is .dtors
         0x102172e8 - 0x102172f8 is .got
         0x102172f8 - 0x102173b8 is .dynamic
         0x102173b8 - 0x10217dac is .sdata
         0x10217db0 - 0x10218308 is .sbss
         0x10218308 - 0x10218c5c is .plt
         0x10218c60 - 0x104d4660 is .bss

'info sources' and 'info functions' show the expected output.

I've spent a few hours reading the GDB manual but I cannot find any clues 
as to why this is happening. If I write a short main() loop and try 
remotely debugging that, it works fine.

I have disabled all optimisations, and gcc is using -g3 (I've tried just 
-g as well).


Does anyone have any ideas for further things I could try please? What 
sort of things that can go wrong would cause this sort of problem or 
these symptoms?


Kind regards,
David Antliff
NZ

Re: remote debugging symbol problem

[Daniel Jacobowitz]

On Thu, May 19, 2005 at 12:26:33PM +1200, David Antliff wrote:
> 
> Hello,
> 
> We use gdb-6.0 and a PowerPC port of gdbserver to remotely debug our 
> target application on it's PowerPC platform. Our application forks several 
> (20 or so) child processes shortly after starting up, and a few of these 
> child processes create threads.
> 
> Until quite recently, this has all worked fine. I was able to invoke:
> 
> # gdbserver 0.0.0.0:2345 --attach <pid>  for any of the child processes, 
> and then run on my development host (x86 linux):
> 
> $ cd <application-src-dir>
> $ gdb <application-binary>
> 
> I am 100% sure the same application binary is running on the host as on 
> the target (I'm using NFS).
> 
> Normally, once GDB is pointed at the target, the process is interrupted 
> and GDB can show a stack trace at the point of interruption.
> 
> However, now, I just get this:
> 
> (gdb) target remote 10.16.10.237:2345
> Remote debugging using 10.16.10.237:2345
> 0x0fe896a0 in ?? ()
> warning: Unable to find dynamic linker breakpoint function.
> GDB will be unable to debug shared library initializers
> and track explicitly loaded dynamic code.
> 
> My understanding is that the warning is simply because the standard 
> libraries aren't compiled with GDB support. This used to happen even when 
> it worked so I'm not concerned about it (should I be?).

You should be.  This means that GDB has not found your target
libraries.  Is solib-absolute-prefix set in your cross GDB?


-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: remote debugging symbol problem

[David Antliff]

On Wed, 18 May 2005, Daniel Jacobowitz wrote:

> On Thu, May 19, 2005 at 12:26:33PM +1200, David Antliff wrote:
>> However, now, I just get this:
>>
>> (gdb) target remote 10.16.10.237:2345
>> Remote debugging using 10.16.10.237:2345
>> 0x0fe896a0 in ?? ()
>> warning: Unable to find dynamic linker breakpoint function.
>> GDB will be unable to debug shared library initializers
>> and track explicitly loaded dynamic code.
>>
>> My understanding is that the warning is simply because the standard
>> libraries aren't compiled with GDB support. This used to happen even when
>> it worked so I'm not concerned about it (should I be?).
>
> You should be.  This means that GDB has not found your target
> libraries.  Is solib-absolute-prefix set in your cross GDB?


Absolutely brilliant - that solved the problem. Thank you! :)

As you suggested, solib-absolute-prefix wasn't set so I set it to the root 
of my NFS directory and it all came to life.

Many thanks,

-- 
David.

Re: remote debugging symbol problem

[Eli Zaretskii]

> Date: Wed, 18 May 2005 21:01:33 -0400
> From: Daniel Jacobowitz <drow@false.org>
> Cc: gdb@sources.redhat.com
> 
> > (gdb) target remote 10.16.10.237:2345
> > Remote debugging using 10.16.10.237:2345
> > 0x0fe896a0 in ?? ()
> > warning: Unable to find dynamic linker breakpoint function.
> > GDB will be unable to debug shared library initializers
> > and track explicitly loaded dynamic code.
> > 
> > My understanding is that the warning is simply because the standard 
> > libraries aren't compiled with GDB support. This used to happen even when 
> > it worked so I'm not concerned about it (should I be?).
> 
> You should be.  This means that GDB has not found your target
> libraries.

If this is the cause, shouldn't GDB tell that explicitly?  I mean,
it's quite hard, given ``unable to find dynamic linker breakpoint
function'' to figure out that the reason is a failed search for the
libraries, except perhaps for wizards such as Daniel ;-)

Should we teach GDB be more explicit here?