Re: howto debug libthread_db

["Carmelo Amoroso" <carmelo73@gmail.com>]

Well,
after a few days of debugging my gdbserver (with uClibc-nptl on sh4),
I found the bug into uClibc,
that was causing a stack frame corruption due to a buffer overrun.
This was the reason for which the td_ta_map_lwp2thr was never
returning to the caller
(the stack location where the PR had been saved was corupted).

It was a very interesting debugging session. I successfully used a
native gdb (glibc base)on target
to debug a gdbserver (uClinc-nptl based) I connect to with a gdb
client from a linux host.

At the end I did not need to use a double remote gdb session.

Thanks to Michael for his very usefull tips.

Carmelo


On 9/27/06, Carmelo Amoroso <carmelo73@gmail.com> wrote:
> Michael Snyder wrote:
> > On Wed, 2006-09-27 at 08:23 +0200, Carmelo Amoroso wrote:
> >> Michael Snyder wrote:
> >>> On Tue, 2006-09-26 at 16:06 +0200, Carmelo Amoroso wrote:
> >>>> Hi All,
> >>>> I'm trying to debug a simple multithread application on a remote target,
> >>>> and I like to debug the libthread_db itself... is it possible to do it?
> >>>> is it possible to set some breakpoints and stepping through?
> >>>> Is there anyone already played with it?
> >>>>
> >>>> I tried to use symbol-file to gdb client, and I successfully added a
> >>>> breakpoint, but I cannot reach it after I connected to the target and
> >>>> issued 'continue'.
> >>>>
> >>>> Any help will be appreciated
> >>> Interesting.  I haven't done it (or not much anyway).
> >>>
> >>> The first thing you must bear in mind is that libthread_db is not
> >>> part of the target application -- it is part of gdb (or in the
> >>> remote debugging context, part of gdbserver).  So to debug it,
> >>> you must debug gdb, or in your case gdbserver.
> >>>
> >> Yes. it's what I want to do
> >
> >>> It might be easier if you try this out first using a native gdb,
> >>> debugging itself.
> >>>
> >>> When you debug gdb with gdb, it is easier if you change the prompt.
> >>> I usually do this in the 'parent' gdb, so I can tell it apart from
> >>> the 'child' gdb:
> >>>
> >>>     (gdb) set prompt (GDB)
> >>>     (GDB)
> >>>
> >>> Now, in the parent GDB, you should be able to set your
> >>> breakpoints in libthread_db, then start the child gdb,
> >>> load the multi-threaded program, and begin debugging it.
> >>>
> >> Does the native gdb use the thread_db as the gdbserver does?
> >
> > Yes.
> >
> >> I was thinking to run gdb --args my-gdbserver <application>
> >> on the target, and connect to the my-gdbserver from the host
> >> just to trigger the thread_db initialization process.
> >> Comments?
> >
> > Oh, so you can actually run gdb on the target?
> Yes, I'm using it after I read your reply. So I'm running
> a working gdb (glibc/sh4 based) to debug a half-working gdbserver
> (uClibc-nptl/sh4 based).
> I was able to set breakpoint on gdbserver itself
> (thread_db_find_new_threads function for example) and to step into
> my thread_db library. But up to now I could not reach the
> td_ta_map_lp2thr function due to some failures in packets transmission
> between gdb remote client and target gdbserver, so the tbread_db
> initializaition is failing before entering in that function.
> I'm still working on it.
>
> I'd like to describe the problem I have in case some one may have an idea:
> The stack is as follows:
>
> td_ta_map_lwp2thr
> iterate_list_thread
> td_ta_thr_iter
> thread_db_find_new_threads
> thread_db_init
>
> The td_ta_map_lwp2thr funtcion reach its end successfully
> (performing any symbol lookup; I also printed out the thread pointer
> and its value is just the same as printed by the pthread_self function
> from the libpthread), but when returns to the caller it produces a
> SIGSEV due to a misaligned access (as reported by dmesg command).
> Any printf after the function's invocation is never executed!
>
> The thread_db code is he same as the glibc, so I cannot understand
> where it's failing. The gdbserver version is 6.4 (built using a
> sh4-linux-gcc configured for uClibc) and it is the same code used to
> build the glibc based version.
>
>
> > Yes, that would make things simpler, and what you
> > suggest above is pretty much what I would recommend.
> >
> >
> >>> When you get ready to try this with gdbserver, you will
> >>> have to start one gdbserver and attach to it with another
> >>> gdbserver.  Then you'll again have two gdbs running on the
> >>> host side, which you can again think of as parent and child,
> >>> only now the parent is debugging the child's gdbserver, not
> >>> the child gdb itself.
> >>>
> >> Well, just before reading your reply, I tried with this:
> >>
> >> target:
> >> gdbserver localhost:999 my-gdbserver localhost:998 <applcation>
> >>
> >> host:
> >> xxx-gdb my-gdbserver (connecting to port 999)
> >>
> >> xxx-gdb application (connecting to port 998)
> >>
> >> With the first gdb session I was able to set a breakpoint into
> >> thread_db, but not able to stepping through the code.
> >>
> >> Is this approach as well as the gdb native session.
> >
> > Hmmm, that's pretty much what I would have tried.
> > What happened when you tried to step?  How did it fail?
> >
>
> Difficult to describe; I'm trying to follow the native approach for now,
> but I'd like trying this again later; I'll report any failure or success
> with more details.
>
> Thanks a lot again
> Carmelo
>