Re: howto debug libthread_db

[Carmelo Amoroso <carmelo73@gmail.com>]

Michael Snyder wrote:
> On Fri, 2006-09-29 at 11:08 +0200, Carmelo Amoroso wrote:
>> Well,
>> after a few days of debugging my gdbserver (with uClibc-nptl on sh4),
>> I found the bug into uClibc,
>> that was causing a stack frame corruption due to a buffer overrun.
>> This was the reason for which the td_ta_map_lwp2thr was never
>> returning to the caller
>> (the stack location where the PR had been saved was corupted).
>>
>> It was a very interesting debugging session. I successfully used a
>> native gdb (glibc base)on target
>> to debug a gdbserver (uClinc-nptl based) I connect to with a gdb
>> client from a linux host.
>>
>> At the end I did not need to use a double remote gdb session.
>>
>> Thanks to Michael for his very usefull tips.
> 
> 
> So, does that mean that there was no actual problem with gdb, 
> with respect to stepping in libthread_db?  
Yes, it was my thread_db that was failing killing the gdbserver
This works now?
Yes, it works perfectly, even with multithreadrd applications.

Carmelo
> 
> Thanks,
> Michael
> 
> 
>> On 9/27/06, Carmelo Amoroso <carmelo73@gmail.com> wrote:
>>> Michael Snyder wrote:
>>>> On Wed, 2006-09-27 at 08:23 +0200, Carmelo Amoroso wrote:
>>>>> Michael Snyder wrote:
>>>>>> On Tue, 2006-09-26 at 16:06 +0200, Carmelo Amoroso wrote:
>>>>>>> Hi All,
>>>>>>> I'm trying to debug a simple multithread application on a remote target,
>>>>>>> and I like to debug the libthread_db itself... is it possible to do it?
>>>>>>> is it possible to set some breakpoints and stepping through?
>>>>>>> Is there anyone already played with it?
>>>>>>>
>>>>>>> I tried to use symbol-file to gdb client, and I successfully added a
>>>>>>> breakpoint, but I cannot reach it after I connected to the target and
>>>>>>> issued 'continue'.
>>>>>>>
>>>>>>> Any help will be appreciated
>>>>>> Interesting.  I haven't done it (or not much anyway).
>>>>>>
>>>>>> The first thing you must bear in mind is that libthread_db is not
>>>>>> part of the target application -- it is part of gdb (or in the
>>>>>> remote debugging context, part of gdbserver).  So to debug it,
>>>>>> you must debug gdb, or in your case gdbserver.
>>>>>>
>>>>> Yes. it's what I want to do
>>>>>> It might be easier if you try this out first using a native gdb,
>>>>>> debugging itself.
>>>>>>
>>>>>> When you debug gdb with gdb, it is easier if you change the prompt.
>>>>>> I usually do this in the 'parent' gdb, so I can tell it apart from
>>>>>> the 'child' gdb:
>>>>>>
>>>>>>     (gdb) set prompt (GDB)
>>>>>>     (GDB)
>>>>>>
>>>>>> Now, in the parent GDB, you should be able to set your
>>>>>> breakpoints in libthread_db, then start the child gdb,
>>>>>> load the multi-threaded program, and begin debugging it.
>>>>>>
>>>>> Does the native gdb use the thread_db as the gdbserver does?
>>>> Yes.
>>>>
>>>>> I was thinking to run gdb --args my-gdbserver <application>
>>>>> on the target, and connect to the my-gdbserver from the host
>>>>> just to trigger the thread_db initialization process.
>>>>> Comments?
>>>> Oh, so you can actually run gdb on the target?
>>> Yes, I'm using it after I read your reply. So I'm running
>>> a working gdb (glibc/sh4 based) to debug a half-working gdbserver
>>> (uClibc-nptl/sh4 based).
>>> I was able to set breakpoint on gdbserver itself
>>> (thread_db_find_new_threads function for example) and to step into
>>> my thread_db library. But up to now I could not reach the
>>> td_ta_map_lp2thr function due to some failures in packets transmission
>>> between gdb remote client and target gdbserver, so the tbread_db
>>> initializaition is failing before entering in that function.
>>> I'm still working on it.
>>>
>>> I'd like to describe the problem I have in case some one may have an idea:
>>> The stack is as follows:
>>>
>>> td_ta_map_lwp2thr
>>> iterate_list_thread
>>> td_ta_thr_iter
>>> thread_db_find_new_threads
>>> thread_db_init
>>>
>>> The td_ta_map_lwp2thr funtcion reach its end successfully
>>> (performing any symbol lookup; I also printed out the thread pointer
>>> and its value is just the same as printed by the pthread_self function
>>> from the libpthread), but when returns to the caller it produces a
>>> SIGSEV due to a misaligned access (as reported by dmesg command).
>>> Any printf after the function's invocation is never executed!
>>>
>>> The thread_db code is he same as the glibc, so I cannot understand
>>> where it's failing. The gdbserver version is 6.4 (built using a
>>> sh4-linux-gcc configured for uClibc) and it is the same code used to
>>> build the glibc based version.
>>>
>>>
>>>> Yes, that would make things simpler, and what you
>>>> suggest above is pretty much what I would recommend.
>>>>
>>>>
>>>>>> When you get ready to try this with gdbserver, you will
>>>>>> have to start one gdbserver and attach to it with another
>>>>>> gdbserver.  Then you'll again have two gdbs running on the
>>>>>> host side, which you can again think of as parent and child,
>>>>>> only now the parent is debugging the child's gdbserver, not
>>>>>> the child gdb itself.
>>>>>>
>>>>> Well, just before reading your reply, I tried with this:
>>>>>
>>>>> target:
>>>>> gdbserver localhost:999 my-gdbserver localhost:998 <applcation>
>>>>>
>>>>> host:
>>>>> xxx-gdb my-gdbserver (connecting to port 999)
>>>>>
>>>>> xxx-gdb application (connecting to port 998)
>>>>>
>>>>> With the first gdb session I was able to set a breakpoint into
>>>>> thread_db, but not able to stepping through the code.
>>>>>
>>>>> Is this approach as well as the gdb native session.
>>>> Hmmm, that's pretty much what I would have tried.
>>>> What happened when you tried to step?  How did it fail?
>>>>
>>> Difficult to describe; I'm trying to follow the native approach for now,
>>> but I'd like trying this again later; I'll report any failure or success
>>> with more details.
>>>
>>> Thanks a lot again
>>> Carmelo
>>>
>