Re: Behaviour of default all-stop mode -- Why no one has replied ?

suresh ds
2009-06-15T15:10:04Z

Thanks for the reply.
I think I've to explain a bit more here to indicate what problems I face and how I'm stuck.

I'm developing a stub for a multi-core mips-based processor, with each core containing a certain no. of hardware threads. Assuming 'n' core, with each core consisting of 'm' threads, totally it'll be n*m threads. The host is gdb-6.8 cross-compiled for mips64.

Each of these n*m threads can run any image. These images, will get compiled along with the stub (which means each of these n*m threads will run an image which consists of application + stub). The stub is a kind of library which the images link to while getting compiled. I don't have the luxury of keeping one thread aside only for the stub.
When these images come up, they execute a break instruction (as recommended by gdb) and go into exception mode waiting for gdb commands. When gdb connects, it connects to a default (hardware) thread, say thread 1. Of course, gdb does not know anything about hardware threads, I'm mapping the gdb's software threads to hardware threads in the chip. (This is the way in general, multi-core debugging is achieved in gdb since gdb is not yet multi-core aware...).

As gdb connects to the default hardware thread 1 (shortly, ht1), the stub sitting in ht1 responds to queries (including the '?' packet). At this stage, note that all hardware threads are stopped. When gdb queries with a '?' to ht1, should ht1 apart from sending its status, also report the status of other hardware threads, something like "$T05;thread:1;thread:2;blah blah". It is another matter that whether I reported initially the status of all the hardware threads or not, didn't solve the problem. Initially, I tried only "$T05:thread:1", which means only that default thread sent its status. Then I did "info threads" to make gdb aware that there are other threads too. Then I switched to thread 1. I really a have a big requirement from the gdb developers here. Once I switch to a thread, ideally I expect all further commands to apply only to that thread; It makes life simple, isn't it ? (Of course, I tried "thread apply", it didn't work as expected. I don't know if this is also post-6.8 !!).Here, switching to a thread really doesn't make you switch to that thread. Because if you step or continue, it can still step or continue other threads. Even an "info reg" is not guaranteed to fetch only THAT thread's registers. With this background, let's go into the log where my comments also are given at appropriate places prefixed with a ==>>

(gdb)target remote 10.7.8.68:5000
Remote debugging using 10.7.8.68:5000
Sending packet: $qSupported#37...Ack
Packet received:
Packet qSupported (supported-packets) is NOT supported
Sending packet: $g#67...Ack
Packet received: 00000000000000000000000050000001ffffffff800b4000ffffffff800b40200000000000000001000000000080000000000000018066a80000000000000001000000000180683800000000018067280000000000000001ffffffff800b00000000000000000001000000000087170c000000000000006000000000000000700000000000000010000000000087172cffffffff900ddeed0000000000000004000000000080000000000000002d000000000000009538780000000000800000ffffffff800b4000ffffffff800b404000000000000000000000000000000000000000000080ac5000000000008f2e800000000000000001000000000023b2fc0000000050000003000000000121eac00000000000f4240000000000008000000000000000008024000000000023741000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Sending packet: $?#3f...Ack
Packet received: T05thread:4;thread:5;thread:6;

==>> Well, the stub sitting on thread 4 (default hardware thread) corresponds to gdb here. It also sends the status of other (hardware) threads. Initially, I made thread 4 pass only its status as "$T05thread:4", but no difference.

Sending packet: $Hc-1#09...Ack

==>> Here, gdb asks to resume all threads, '-1' indicating 'all threads'. But I don't resume here, all the threads are still in stopped state. If I resume all the threads here in fact, I don't think it'll matter ?

Packet received: OK
Sending packet: $qC#b4...Ack
Packet received: QC 4
Sending packet: $qOffsets#4b...Ack
Packet received: Text=0;Data=0;Bss=0
[New Thread 4]
[New Thread 5]
[New Thread 6]
Sending packet: $g#67...Ack
Packet received: 00000000000000000000000050000001ffffffff800b4000ffffffff800b40200000000000000001000000000080000000000000018066a80000000000000001000000000180683800000000018067280000000000000001ffffffff800b00000000000000000001000000000087170c000000000000006000000000000000700000000000000010000000000087172cffffffff900ddeed0000000000000004000000000080000000000000002d000000000000009538780000000000800000ffffffff800b4000ffffffff800b404000000000000000000000000000000000000000000080ac5000000000008f2e800000000000000001000000000023b2fc0000000050000003000000000121eac00000000000f4240000000000008000000000000000008024000000000023741000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Sending packet: $g#67...Ack
Packet received: 00000000000000000000000050000001ffffffff800b4000ffffffff800b40200000000000000001000000000080000000000000018066a80000000000000001000000000180683800000000018067280000000000000001ffffffff800b00000000000000000001000000000087170c000000000000006000000000000000700000000000000010000000000087172cffffffff900ddeed0000000000000004000000000080000000000000002d000000000000009538780000000000800000ffffffff800b4000ffffffff800b404000000000000000000000000000000000000000000080ac5000000000008f2e800000000000000001000000000023b2fc0000000050000003000000000121eac00000000000f4240000000000008000000000000000008024000000000023741000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
[Switching to Thread 6]

==>> The stub on ht4 (hardware thread 4), has reported three threads, 4, 5, and 6 to gdb; And, gdb has switched to the last reported thread here.

0x00237410 in gdb_break ()
at blah blah....
123

Sending packet: $qSymbol::#5b...Ack
Packet received:
Packet qSymbol (symbol-lookup) is NOT supported
(gdb)info threads
Sending packet: $T00000006#da...Ack
Packet received: OK
Sending packet: $T00000005#d9...Ack
Packet received: OK
Sending packet: $T00000004#d8...Ack
Packet received: OK
Sending packet: $qfThreadInfo#bb...Ack
Packet received: m 4,5,6,l
Sending packet: $qsThreadInfo#c8...Ack
Packet received: l
Sending packet: $qThreadExtraInfo,6#bb...Ack
Packet received: 72756e6e696e67
* 3 Thread 6 (running) 0x00237410 in gdb_break ()
at blah blah...
Sending packet: $qThreadExtraInfo,5#ba...Ack
Packet received: 72756e6e696e67
Sending packet: $Hg5#e4...Ack
Packet received: OK
Sending packet: $g#67...Ack
Packet received: 00000000000000000000000050000001ffffffff800b4000ffffffff800b40200000000000000001000000000080000000000000018066a80000000000000001000000000180683c00000000018067280000000000000001ffffffff800b00000000000000000001000000000087170d000000000000000000000000000000200000000000000020000000000087172dffffffff900ddeed0000000000000005000000000080000000000000002d00000000000000953a780000000000800000ffffffff800b4000ffffffff800b404000000000000000000000000000000000000000000080ac5000000000008f2e800000000000000001000000000023b2fc0000000050000003000000000121eac00000000000f4240000000000008000000000000000008024000000000023741000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
2 Thread 5 (running) 0x00237410 in gdb_break ()
at blah blah...
Sending packet: $qThreadExtraInfo,4#b9...Ack
Packet received: 72756e6e696e67
Sending packet: $Hg4#e3...Ack
Packet received: OK
Sending packet: $g#67...Ack
Packet received: 00000000000000000000000050000001ffffffff800b4000ffffffff800b40200000000000000001000000000080000000000000018066a80000000000000001000000000180683800000000018067280000000000000001ffffffff800b00000000000000000001000000000087170c000000000000006000000000000000700000000000000010000000000087172cffffffff900ddeed0000000000000004000000000080000000000000002d000000000000009538780000000000800000ffffffff800b4000ffffffff800b404000000000000000000000000000000000000000000080ac5000000000008f2e800000000000000001000000000023b2fc0000000050000003000000000121eac00000000000f4240000000000008000000000000000008024000000000023741000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
1 Thread 4 (running) 0x00237410 in gdb_break ()
at blah blah...
Sending packet: $Hg6#e5...Ack
Packet received: OK
Sending packet: $g#67...Ack
Packet received: 00000000000000000000000050000001ffffffff800b4000ffffffff800b40200000000000000001000000000080000000000000018066a80000000000000001000000000180684000000000018067280000000000000001ffffffff800b00000000000000000001000000000087170e000000000000002000000000000000600000000000000040000000000087172effffffff900ddeed0000000000000006000000000080000000000000002d00000000000000953c780000000000800000ffffffff800b4000ffffffff800b404000000000000000000000000000000000000000000080ac5000000000008f2e800000000000000001000000000023b2fc0000000050000003000000000121eac00000000000f4240000000000008000000000000000008024000000000023741000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

==>> At this point, gdb is made aware that there are three threads waiting to be debugged; Already, the presence of these threads were also indicated when the stub on the default thread sent the status of all the threads.

(rmios-gdb)c
Continuing.
Sending packet: $vCont?#49...Ack
Packet received: vCont;c;C;s;S;t;T
Packet vCont (verbose-resume) is supported
Sending packet: $vCont;c#a8...Ack

==>> In default all-stop mode, when you do continue, all threads should continue, correct ? According to our discussions so far (read mails below), gdb will indicate through vCont, which threads it wants to resume. But here, it does not indicate. So, which thread should I resume ? Actually, I resumed the default thread (4). Should the stub resume all the threads even though it has not received any thread no. ?

Now, this is without any breakpoints set. Even if I set a breakpoint and continue, it only continues the 'current' thread. I typically expect something like this from gdb. "$vCont;c:4;c:5;blahblah". After thread 4 hits the breakpoint, then I switch to some other thread,say 5, and continue, gdb still sends packets only to continue thread 4; Thread 5 does not move at all. In other words, switching to any thread does not make gdb to send packets in that thread context (which I've complained earlier).

I tried 'thread-specific breakpoints', and 'thread apply', but to no avail. Forget breakpoint setting; First of all, when I continue from one thread, why gdb is not issuing packets to resume all the threads? Is this not the expected behaviour in default all-stop mode ? In our earlier discussions, you have mentioned that that stub itself should not resume all the threads and that the gdb will indicate that through vCont packets, which is not happening here.

Please help.

Thanks,
Suresh

----- Original Message -----
From: "Pedro Alves"
To: "suresh ds"
Cc: gdb@sourceware.org
Subject: Re: Behaviour of default all-stop mode -- Why no one has replied ?
Date: Mon, 15 Jun 2009 12:01:43 +0100

On Monday 15 June 2009 11:37:53, suresh ds wrote:

> > Suppose three threads are running, say 1, 2, & 3.
> > Suppose thread 1 hits the breakpoint; It'll send the status to
> > gdb; At this point, gdb itself will send (further) packets to >
> stop other threads, or the stub itself should take the initiative
> > to stop other threads ?
>
> The stub.
>
> ==>> So, the stub take the initiative to stop other threads. In
> this case, the other threads' status should also be reported to
> gdb ?

Nope. The other threads are supposedly stopped without
anything interesting to report to GDB. Only one event is
reported to GDB at a time.

If while your stub tries to stop all threads to report a stop
event to GDB (e.g., breakpoint hits in thread 1), one of those
threads hits another event (e.g., thread 2 gets signal)
then you have to deal with it. E.g., leaving the extra events pending
until the next time GDB tells the threads to resume, or, by tweaking
the thread state so that the event would naturally trigger again
the next time you resume the threads. If your stub is in-kernel, assuming
one core only, then usually you don't have to care for these complications,
as all threads are halted while the stub's code is executing.

>
>
> > 2) The document says, "whenever you restart the program, all >
> threads start executing".
> > -- Again, the gdb takes initiative to continue all the threads
> or > the stub should have a mechanism to do this ?
> > Suppose three threads, 1, 2, & 3 are in a stopped state.
>
> See the vCont packet description. That packet explicitly says
> what to resume,
> there's no all-stop vs non-stop magic in that aspect.
>
> > Now, "continue" from thread 1 will continue all the threads ?
> > When I checked the packets, I found that it does only "Hc1" to
> > set the further continue packets only for thread 1.
>
> Implement vCont support instead, s,c,Hc are deprecated for
> multithreaded stubs.
>
> ==>> But I get indication from gdb only to start one thread;
> Maybe, this will change if the stub reports the status of all the
> threads initially ??

I think you're not reading the packet right, but this this is just
guessing, since now at this point, I don't know if you're talking
about vCont or c or s... Please be specific.

`vCont[;action[:thread-id]]...'
Resume the inferior, specifying different actions for each thread.
If an action is specified with no thread-id, then it is applied to any
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
threads that don't have a specific action specified;
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

> I suggest studying the trafic between gdb and gdbserver.
>
> ==>> I tried debugging a pthread program through gdbserver on
> intel-i386. But info threads was not recognizing the threads; I
> don't know if I should compile/configure gdbserver separately for
> thread mode ...

It should just work, if you run gdbserver on the same machine
as gdb. Otherwise, you may be stumbling on point 6
at http://sourceware.org/gdb/wiki/FAQ.

--
Pedro Alves

--
Be Yourself @ mail.com!
Choose From 200+ Email Addresses
Get a Free Account at www.mail.com