Triggering qSymbol packets

Triggering qSymbol packets

[John Williams]

Hi,

The short version:
Is there a way to trigger GDB to issue a remote qSymbol packet, other
than the "symbol" command?

The long version:
I am working on multithreaded gdbserver support for a noMMU
embedded Linux target (uClinux).  The thread library is linuxthreads
(and linuxthreads_db) in uClibc 0.9.x.  gdbserver has all of the
multithreading hooks and proc-service etc.

The cross-gdb is version 5.3, which should have the necessary remote
thread code in it.

gdbserver initialises linuxthreads_db hooks in response to a qSymbol
packet (generated by the gdb "sym" command).  This necessarily must come
after initial remote target connection.

Here's the catch - in uClinux we use qOffset to find out where
in the flat memory space is the application loaded - qOffset is
triggered by gdb/remote.c:remote_start_remote()

To trigger the qSymbol packet (and thus initialise gdbserver's
linuxthreads_db stuff), requires the gdb "sym" command.

From my searching, it seems the recommended sequence for gdbserver +
pthreads should be

1. gdb% target remote host:port
(connect to gdbserver, send qOffset packet, relocate the objfile
accordingly)
2. gdb% sym myapp.elf
(send qSymbol packet to gdbserver, to initialise the linuxthreads_db)

Here's the problem - the sym command resets gdb's symbol table, thus
undoing the relocation done in response to the qOffset packet.  This
means that when linuxthreads_db sends symbol query packets, gdb replies
with *unrelocated* symbol addresses, and everything falls apart.

This is a pretty specific issue - not sure if gdbserver+threads+noMMU
Linux has ever been done before.  So, I suppose my question(s) are:

1. has this been reported as an issue before, maybe resolved in a newer
gdb? (I've googled for 2 days and found no direct answer, just many
different pieces of the puzzle)

2. Am I correct thinking that for gdbserver+threads the user must
manually do the gdb "sym" command to trigger the qSymbol packet, and
thus initialise linuxthreads_db?

I'm torn on the cleanest way to fix this, if (1) and (2) are true.  For
now I've added a call to remote_check_symbols() inside
remote_start_remote(), after get_offsets(), and that seems to do what I
expect.  At least it gets further this time, but still doesn't quite work.

Do my ramblings above ring any bells with anyone?  Any suggestions or
advice greatfully appreciated.

Thanks,

John

Re: Triggering qSymbol packets

[Daniel Jacobowitz]

On Thu, Jan 19, 2006 at 09:58:44AM +1000, John Williams wrote:
> Hi,
> 
> The short version:
> Is there a way to trigger GDB to issue a remote qSymbol packet, other
> than the "symbol" command?
> 
> The long version:
> I am working on multithreaded gdbserver support for a noMMU
> embedded Linux target (uClinux).  The thread library is linuxthreads
> (and linuxthreads_db) in uClibc 0.9.x.  gdbserver has all of the
> multithreading hooks and proc-service etc.
> 
> The cross-gdb is version 5.3, which should have the necessary remote
> thread code in it.

I strongly recommend an upgrade, as a matter of principle.  The current
version is GDB 6.4 and any answers you get off this list may not apply
well to 5.3.

> gdbserver initialises linuxthreads_db hooks in response to a qSymbol
> packet (generated by the gdb "sym" command).  This necessarily must come
> after initial remote target connection.
> 
> Here's the catch - in uClinux we use qOffset to find out where
> in the flat memory space is the application loaded - qOffset is
> triggered by gdb/remote.c:remote_start_remote()

Right so far...

> To trigger the qSymbol packet (and thus initialise gdbserver's
> linuxthreads_db stuff), requires the gdb "sym" command.

Wrong.  It's triggered (A) on connection to a remote target, and (B)
whenever a new file is loaded.

(A) may be broken or missing in 5.3.

> From my searching, it seems the recommended sequence for gdbserver +
> pthreads should be
> 
> 1. gdb% target remote host:port
> (connect to gdbserver, send qOffset packet, relocate the objfile
> accordingly)
> 2. gdb% sym myapp.elf
> (send qSymbol packet to gdbserver, to initialise the linuxthreads_db)

Incorrect.  Please tell us where you found anything suggesting this.
People keep doing it, and so far, not a one has been able to point to
an incorrect reference (that I remember at the moment, anyway).

The correct sequence is "file" followed by "target remote".  Please try
that instead.

> I'm torn on the cleanest way to fix this, if (1) and (2) are true.  For
> now I've added a call to remote_check_symbols() inside
> remote_start_remote(), after get_offsets(), and that seems to do what I
> expect.  At least it gets further this time, but still doesn't quite work.

Please take a look at the call site for remote_start_remote, in
remote_open_1.

-- 
Daniel Jacobowitz
CodeSourcery

Re: Triggering qSymbol packets

[John Williams]

Hi Daniel,

Thanks for quick reply, comments inline below.

Daniel Jacobowitz wrote:
> On Thu, Jan 19, 2006 at 09:58:44AM +1000, John Williams wrote:
> 
>>The short version:
>>Is there a way to trigger GDB to issue a remote qSymbol packet, other
>>than the "symbol" command?
>>
>>The long version:
>>I am working on multithreaded gdbserver support for a noMMU
>>embedded Linux target (uClinux).  The thread library is linuxthreads
>>(and linuxthreads_db) in uClibc 0.9.x.  gdbserver has all of the
>>multithreading hooks and proc-service etc.
>>
>>The cross-gdb is version 5.3, which should have the necessary remote
>>thread code in it.
> 
> I strongly recommend an upgrade, as a matter of principle.  The current
> version is GDB 6.4 and any answers you get off this list may not apply
> well to 5.3.

I agree 100%, and understand statute of limitations re: list support for
5.3.  However a gdb arch forward-port is not something I'm ready to take
on right now.  The CPU vendor (Xilinx, MicroBlaze softCPU) does their
toolchain maintenance in-house, and are settled on gdb5.3 + gcc3.4 for now.

>>To trigger the qSymbol packet (and thus initialise gdbserver's
>>linuxthreads_db stuff), requires the gdb "sym" command.
> 
> Wrong.  It's triggered (A) on connection to a remote target, and (B)
> whenever a new file is loaded.
> 
> (A) may be broken or missing in 5.3.

Ah ha, good to know.

>>From my searching, it seems the recommended sequence for gdbserver +
>>pthreads should be
>>
>>1. gdb% target remote host:port
>>(connect to gdbserver, send qOffset packet, relocate the objfile
>>accordingly)
>>2. gdb% sym myapp.elf
>>(send qSymbol packet to gdbserver, to initialise the linuxthreads_db)
> 
> 
> Incorrect.  Please tell us where you found anything suggesting this.
> People keep doing it, and so far, not a one has been able to point to
> an incorrect reference (that I remember at the moment, anyway).

I got it from here:

http://www.cygwin.com/ml/gdb/2002-02/msg00098.html

I looked but didn't find any current references that describe gdbserver
+ pthread debugging - if there's something obvious I'm missing please
let me know.

> The correct sequence is "file" followed by "target remote".  Please try
> that instead.

Will do.

>>I'm torn on the cleanest way to fix this, if (1) and (2) are true.  For
>>now I've added a call to remote_check_symbols() inside
>>remote_start_remote(), after get_offsets(), and that seems to do what I
>>expect.  At least it gets further this time, but still doesn't quite work.
> 
> 
> Please take a look at the call site for remote_start_remote, in
> remote_open_1.

OK.  I'm guessing a back-port of the fixup will be much easer (short
term) than a forward port of the entire arch.    Am I maybe
over-estimating the work involved in migrating an arch from gdb 5.3 to 6.4?

Thanks again,

John

Re: Triggering qSymbol packets

[Jim Blandy]

On 1/18/06, John Williams <jwilliams@itee.uq.edu.au> wrote:
> OK.  I'm guessing a back-port of the fixup will be much easer (short
> term) than a forward port of the entire arch.    Am I maybe
> over-estimating the work involved in migrating an arch from gdb 5.3 to 6.4?

A lot has changed.  The frame unwinding interface is eccentric in a
different way than it used to be.  :)  Actually, it's much more
well-defined, and well-behaved.  The gdbarch methods for calling
functions and extracting their return values have changed.  So there's
a lot of work entailed in bringing a port forward.

Re: Triggering qSymbol packets

[Daniel Jacobowitz]

On Thu, Jan 19, 2006 at 10:45:34AM +1000, John Williams wrote:
> I got it from here:
> 
> http://www.cygwin.com/ml/gdb/2002-02/msg00098.html
> 
> I looked but didn't find any current references that describe gdbserver
> + pthread debugging - if there's something obvious I'm missing please
> let me know.

Well, that's a shame.  The implementation eventually merged wasn't that
one, and his advice is not useful today.

There's inadequate documentation for this process and it could use a
good FAQ; it comes up on this list about twice a month.

-- 
Daniel Jacobowitz
CodeSourcery

Re: Triggering qSymbol packets

[John Williams]

Hi again,

An update on yesterday's emails - I've sorted this one out.  In gdb5.3
the trigger of the qOffset packet, in remote_start_remote, was nested
inside an #ifdef SOLIB... structure.  In 6.4 (but interestingly, not
6.3), this #ifdef is more discriminating, and does permit the qSymbol on
target connect, even if SOLIB is not in use.

It's still not working yet, but certainly gets a lot further than before.

Thanks for your help,

John

Daniel Jacobowitz wrote:
> On Thu, Jan 19, 2006 at 10:45:34AM +1000, John Williams wrote:
> 
>>I got it from here:
>>
>>http://www.cygwin.com/ml/gdb/2002-02/msg00098.html
>>
>>I looked but didn't find any current references that describe gdbserver
>>+ pthread debugging - if there's something obvious I'm missing please
>>let me know.
> 
> 
> Well, that's a shame.  The implementation eventually merged wasn't that
> one, and his advice is not useful today.
> 
> There's inadequate documentation for this process and it could use a
> good FAQ; it comes up on this list about twice a month.
> 


-- 
Dr John Williams, Research Fellow,
Embedded Systems Group / Reconfigurable Computing
School of ITEE, The University of Queensland, Brisbane, Australia
(p) +61 7 33652185  (f) +61 7 33654999 (m) +61 403969243

Re: Triggering qSymbol packets

[Jie Zhang]

On 1/19/06, John Williams <jwilliams@itee.uq.edu.au> wrote:
>
> This is a pretty specific issue - not sure if gdbserver+threads+noMMU
> Linux has ever been done before.  So, I suppose my question(s) are:
>
Yes. Debugging multithreading application using GDB/gdbserver works
pretty well on uClinux for Blackfin. You can find our test result in
file "toolchain_test_results_2005R4.tar.gz" on this page:
<http://blackfin.uclinux.org/frs/?group_id=18>.

> 1. has this been reported as an issue before, maybe resolved in a newer
> gdb? (I've googled for 2 days and found no direct answer, just many
> different pieces of the puzzle)
>
It's better to upgrade to the newest version. Our test is done on GDB
6.3 with some backports from 6.4.

> 2. Am I correct thinking that for gdbserver+threads the user must
> manually do the gdb "sym" command to trigger the qSymbol packet, and
> thus initialise linuxthreads_db?
>
I think "file" followed by "target remote" should work.

Jie