Re: howto debug libthread_db

Michael Snyder
2006-09-29T18:12:57Z

On Fri, 2006-09-29 at 11:08 +0200, Carmelo Amoroso wrote:
> Well,
> after a few days of debugging my gdbserver (with uClibc-nptl on sh4),
> I found the bug into uClibc,
> that was causing a stack frame corruption due to a buffer overrun.
> This was the reason for which the td_ta_map_lwp2thr was never
> returning to the caller
> (the stack location where the PR had been saved was corupted).
>
> It was a very interesting debugging session. I successfully used a
> native gdb (glibc base)on target
> to debug a gdbserver (uClinc-nptl based) I connect to with a gdb
> client from a linux host.
>
> At the end I did not need to use a double remote gdb session.
>
> Thanks to Michael for his very usefull tips.

So, does that mean that there was no actual problem with gdb,
with respect to stepping in libthread_db? This works now?

Thanks,
Michael

>
> On 9/27/06, Carmelo Amoroso <carmelo73@gmail.com> wrote:
> > Michael Snyder wrote:
> > > On Wed, 2006-09-27 at 08:23 +0200, Carmelo Amoroso wrote:
> > >> Michael Snyder wrote:
> > >>> On Tue, 2006-09-26 at 16:06 +0200, Carmelo Amoroso wrote:
> > >>>> Hi All,
> > >>>> I'm trying to debug a simple multithread application on a remote target,
> > >>>> and I like to debug the libthread_db itself... is it possible to do it?
> > >>>> is it possible to set some breakpoints and stepping through?
> > >>>> Is there anyone already played with it?
> > >>>>
> > >>>> I tried to use symbol-file to gdb client, and I successfully added a
> > >>>> breakpoint, but I cannot reach it after I connected to the target and
> > >>>> issued 'continue'.
> > >>>>
> > >>>> Any help will be appreciated
> > >>> Interesting. I haven't done it (or not much anyway).
> > >>>
> > >>> The first thing you must bear in mind is that libthread_db is not
> > >>> part of the target application -- it is part of gdb (or in the
> > >>> remote debugging context, part of gdbserver). So to debug it,
> > >>> you must debug gdb, or in your case gdbserver.
> > >>>
> > >> Yes. it's what I want to do
> > >
> > >>> It might be easier if you try this out first using a native gdb,
> > >>> debugging itself.
> > >>>
> > >>> When you debug gdb with gdb, it is easier if you change the prompt.
> > >>> I usually do this in the 'parent' gdb, so I can tell it apart from
> > >>> the 'child' gdb:
> > >>>
> > >>> (gdb) set prompt (GDB)
> > >>> (GDB)
> > >>>
> > >>> Now, in the parent GDB, you should be able to set your
> > >>> breakpoints in libthread_db, then start the child gdb,
> > >>> load the multi-threaded program, and begin debugging it.
> > >>>
> > >> Does the native gdb use the thread_db as the gdbserver does?
> > >
> > > Yes.
> > >
> > >> I was thinking to run gdb --args my-gdbserver <application>
> > >> on the target, and connect to the my-gdbserver from the host
> > >> just to trigger the thread_db initialization process.
> > >> Comments?
> > >
> > > Oh, so you can actually run gdb on the target?
> > Yes, I'm using it after I read your reply. So I'm running
> > a working gdb (glibc/sh4 based) to debug a half-working gdbserver
> > (uClibc-nptl/sh4 based).
> > I was able to set breakpoint on gdbserver itself
> > (thread_db_find_new_threads function for example) and to step into
> > my thread_db library. But up to now I could not reach the
> > td_ta_map_lp2thr function due to some failures in packets transmission
> > between gdb remote client and target gdbserver, so the tbread_db
> > initializaition is failing before entering in that function.
> > I'm still working on it.
> >
> > I'd like to describe the problem I have in case some one may have an idea:
> > The stack is as follows:
> >
> > td_ta_map_lwp2thr
> > iterate_list_thread
> > td_ta_thr_iter
> > thread_db_find_new_threads
> > thread_db_init
> >
> > The td_ta_map_lwp2thr funtcion reach its end successfully
> > (performing any symbol lookup; I also printed out the thread pointer
> > and its value is just the same as printed by the pthread_self function
> > from the libpthread), but when returns to the caller it produces a
> > SIGSEV due to a misaligned access (as reported by dmesg command).
> > Any printf after the function's invocation is never executed!
> >
> > The thread_db code is he same as the glibc, so I cannot understand
> > where it's failing. The gdbserver version is 6.4 (built using a
> > sh4-linux-gcc configured for uClibc) and it is the same code used to
> > build the glibc based version.
> >
> >
> > > Yes, that would make things simpler, and what you
> > > suggest above is pretty much what I would recommend.
> > >
> > >
> > >>> When you get ready to try this with gdbserver, you will
> > >>> have to start one gdbserver and attach to it with another
> > >>> gdbserver. Then you'll again have two gdbs running on the
> > >>> host side, which you can again think of as parent and child,
> > >>> only now the parent is debugging the child's gdbserver, not
> > >>> the child gdb itself.
> > >>>
> > >> Well, just before reading your reply, I tried with this:
> > >>
> > >> target:
> > >> gdbserver localhost:999 my-gdbserver localhost:998 <applcation>
> > >>
> > >> host:
> > >> xxx-gdb my-gdbserver (connecting to port 999)
> > >>
> > >> xxx-gdb application (connecting to port 998)
> > >>
> > >> With the first gdb session I was able to set a breakpoint into
> > >> thread_db, but not able to stepping through the code.
> > >>
> > >> Is this approach as well as the gdb native session.
> > >
> > > Hmmm, that's pretty much what I would have tried.
> > > What happened when you tried to step? How did it fail?
> > >
> >
> > Difficult to describe; I'm trying to follow the native approach for now,
> > but I'd like trying this again later; I'll report any failure or success
> > with more details.
> >
> > Thanks a lot again
> > Carmelo
> >