Cross-platform, multithreaded debugging (x86 to ARM) with gdb and gdbserver not recognizing threads

Jason Machacek
2009-12-16T23:01:20Z

Hello everyone,

I recently ran into an issue where my embedded application became too
large to run natively under GDB. Unfortunately, my SBC provider doesn't
supply a precompiled version of GDB, so I've been trying to build a
GDB/gdbserver combination myself. My application is a C++ app with
several threads running.

I can successfully build both GDB (native x86 with --target=arm-linux
specified) and gdbserver (ARM-native). However, when I connect to
gdbserver on my target with gdb, it doesn't understand that it's a
multithreaded application--it breaks on each context switch (SIG32) and
can't figure out what instruction it's looking at.

I found a 2005 thread from this mailing list
(http://www.cygwin.com/ml/gdb/2005-11/msg00364.html) and several other
messages from other places that set me on the right track, but I'm still
not quite there.

Here are the commands I used to build GDB (x86):

# cd gdb-6.7.1/
# PATH=$PATH:/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/bin/
./configure --target=arm-linux --disable-sim --disable-tcl
--enable-threads --enable-shared
# make

Here are the commands I used to build gdbserver (ARM):

# cd gdb/gdbserver/
# PATH=$PATH:/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/bin
CC=/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/bin/arm-linux-gcc
./configure --host=arm-linux --target=arm-linux --enable-threads
--enable-shared
# make

I execute the gdbserver binary on the target:

$ ./gdbserver :12345 app
Process app created; pid = 270
Listening on port 12345

I then connect to the target from the host like so:

# ../gdb-6.7.1/gdb/gdb app
GNU gdb 6.7.1
Copyright (C) 2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show
copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-pc-linux-gnu
--target=arm-linux"...
(gdb) set solib-absolute-prefix
/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/arm-linux/
(gdb) file app
Load new symbol table from "/home/jmachacek/juice/juice/src/app"? (y or
n) y
Reading symbols from /home/jmachacek/juice/juice/src/app...done.
(gdb) target remote 10.0.150.55:12345
Remote debugging using 10.0.150.55:12345
0x2aaaba70 in _start () from
/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/arm-linux/lib/ld-linux.so
.2
(gdb)

So far so good. "info threads" and "info shared" look okay:

(gdb) info threads
* 1 Thread 270 0x2aaaba70 in _start ()
from
/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/arm-linux/lib/ld-linux.so
.2
(gdb) info shared
From To Syms Read Shared Object Library
0x2aaaba70 0x2aabd888 Yes
/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/arm-linux/lib/ld-linux.so
.2
(gdb)

I can debug to main() with no problem:

(gdb) b main
Breakpoint 1 at 0x20808: file Main/Main.cpp, line 41.
(gdb) c
Continuing.

Breakpoint 1, main (argc=1, argv=0x7ffffec4) at Main/Main.cpp:41
41 BSP_init(argc, argv);
(gdb) bt
#0 main (argc=1, argv=0x7ffffec4) at Main/Main.cpp:41
(gdb)

At this point, I use 'c' to start running and GDB stops when it receives
SIG32 (which appears to be generated when the CPU switches between
threads):

(gdb) c
Continuing.

Program received signal SIG32, Real-time event 32.
0x2abd49e8 in ?? ()
(gdb) bt
#0 0x2abd49e8 in ?? ()
(gdb) info threads
* 1 Thread 270 0x2abd49e8 in ?? ()
warning: Couldn't restore frame in current thread, at frame 0
0x2abd49e8 in ?? ()
(gdb) info shared
warning: .dynamic section for
"/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/arm-linux/lib/libm.so.6"
is not at the expected address (wrong library or version mismatch?)
warning: .dynamic section for
"/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/arm-linux/lib/libgcc_s.s
o.1" is not at the expected address (wrong library or version mismatch?)
warning: .dynamic section for
"/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/arm-linux/lib/libc.so.6"
is not at the expected address (wrong library or version mismatch?)
From To Syms Read Shared Object Library
0x2aaaba70 0x2aabd888 Yes
/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/arm-linux/lib/ld-linux.so
.2
0x2aacc050 0x2aad5e1c No
/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/arm-linux/lib/libpthread.
so.0
0x2ab254c4 0x2ab5d5d0 No
/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/arm-linux/lib/libm.so.6
0x2ab9ab84 0x2aba12d8 No
/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/arm-linux/lib/libgcc_s.so
.1
0x2abc0f80 0x2acbd404 No
/opt/crosstool/gcc-3.4.4-glibc-2.3.2/arm-linux/arm-linux/lib/libc.so.6
(gdb)

The warnings in "info shared" are due to libc being statically linked in
my application. Can anyone help me figure out why I'm unable to get
gdb/gdbserver to work properly with my target?

Thanks very much,
Jason

Jason Machacek | Stratos Product Development | Software Engineer |
(206) 576-7312