gdbserver and "terminated with signal SIGTRAP"

Atsushi Nemoto
2007-11-29T14:53:01Z

I got "Child terminated with signal = 5" message when debugging this
tiny program with gdbserver. Setting breakpoint on "func" (which is
an entry point of thread) and doing "continue" can reproduce this
problem. I'm running gdb and gdbserver on same host (linux/x86).

--- foo.c ---
#include <stdlib.h>
#include <pthread.h>

void *func(void *data)
{
return data;
}

int main(int argc, char **argv)
{
pthread_t id;
int i;

for (i = 0; i < 4; i++) {
pthread_create(&id, NULL, func, NULL);
pthread_join(id, NULL);
}
return 0;
}
--- --- ---

I tried gdb 6.4.90 (Debian 4.0), 6.5 (FC6), 6.6 (FC7), vanilla 6.7.1
on i386 and got same results on all of them. The current cvs snapshot
(gdb-6.7.50.20071129) did not cause this problem, but I want to know
how to fix older versions.

Here is a session log (both side).

--- host (gdb) side ---
$ gdb ./foo
GNU gdb Red Hat Linux (6.5-15.fc6rh)
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1".

(gdb) target remote localhost:50000
Remote debugging using localhost:50000
0x00b8f810 in ?? ()
(gdb) b func
Breakpoint 1 at 0x8048437: file foo.c, line 6.
(gdb) c
Continuing.
[New thread 19500]
[New thread 19502]
[Switching to thread 19502]

Breakpoint 1, func (data=0x0) at foo.c:6
6 return data;
(gdb) c
Continuing.

Program terminated with signal SIGTRAP, Trace/breakpoint trap.
The program no longer exists.
--- --- ---

--- target (gdbserver) side ---
$ gdbserver localhost:50000 ./foo
Process ./foo created; pid = 19500
Listening on port 50000
Remote debugging from host 127.0.0.1

Child terminated with signal = 5

Child terminated with signal = 0x5
GDBserver exiting
--- --- ---

And here is a session log on gdbserver side (gdb-6.7.1) with
debug_threads=1.

--- target (gdbserver) side (debug_thread=1) ---
Process ./foo created; pid = 10926
Got an event from 10926 (57f)
stop pc (before any decrement) is 00b8f810
stop pc (before any decrement) is 00b8f810
Hit a (non-reinsert) breakpoint.
Listening on port 50000
Remote debugging from host 127.0.0.1
Writing 24 to 00b9d7a0
Writing cc to 00b9d7a0
Writing cc to 08048437
Resuming, no pending status
Resuming process 10926 (continue, signal 0, stop not expected)
stop pc (before any decrement) is 00b8f810
Got an event from 10926 (57f)
stop pc (before any decrement) is 00b9d7a1
stop pc (before any decrement) is 00b9d7a1
Hit a (non-reinsert) breakpoint.
Writing 55 to 00b9d7a0
Writing 8b to 08048437
Resuming, no pending status
Resuming process 10926 (step, signal 0, stop not expected)
Checking for breakpoint in process 10926.
stop pc (before any decrement) is 00b9d7a0
Ignoring, PC was changed. Old PC was 0x00b9d7a0
stop pc (before any decrement) is 00b9d7a0
Got an event from 10926 (57f)
stop pc (before any decrement) is 00b9d7a1
stop pc (before any decrement) is 00b9d7a1
Hit a (non-reinsert) breakpoint.
Writing cc to 00b9d7a0
Writing cc to 08048437
Resuming, no pending status
Resuming process 10926 (continue, signal 0, stop not expected)
stop pc (before any decrement) is 00b9d7a1
Got an event from 10926 (57f)
stop pc (before any decrement) is 00b9d7a1
stop pc (before any decrement) is 00b9d7a1
Hit a (non-reinsert) breakpoint.
Writing 55 to 00b9d7a0
Writing 8b to 08048437
Writing 80 to 00dfa16c
Writing cc to 00de87d0
Found first thread -1208305984 (LWP 10926)
Writing 01 to b7fab8a9
Resuming, no pending status
Resuming process 10926 (step, signal 0, stop not expected)
Checking for breakpoint in process 10926.
stop pc (before any decrement) is 00b9d7a1
Removed breakpoint.
set pc to 00b9d7a0
stop pc (before any decrement) is 00b9d7a0
Got an event from 10926 (57f)
stop pc (before any decrement) is 00b9d7a1
stop pc (before any decrement) is 00b9d7a1
Hit a (non-reinsert) breakpoint.
Writing resume reply for -1208305984

Writing cc to 00b9d7a0
Writing cc to 08048437
Resuming, no pending status
Resuming process 10926 (continue, signal 0, stop not expected)
stop pc (before any decrement) is 00b9d7a1
Got an event from 10926 (57f)
stop pc (before any decrement) is 00de87d1
stop pc (before any decrement) is 00de87d1
Hit a (non-reinsert) breakpoint.
Thread creation event. # The 1st thread creation
Writing 00 to b7faadac
Writing 00 to 00dfa174
Attaching to thread -1208308848 (LWP 10927) # 1st attach
Writing 01 to b7faad79
Writing 55 to 00de87d0
Resuming process 10926 (step, signal 0, stop not expected)
pending reinsert at 00de87d0Checking for breakpoint in process 10926.
stop pc (before any decrement) is 00de87d1
Removed breakpoint.
set pc to 00de87d0
stop pc (before any decrement) is 00de87d0
Got an event from 10926 (57f)
stop pc (before any decrement) is 00de87d1
stop pc (before any decrement) is 00de87d1
Reinserted breakpoint.
Writing cc to 00de87d0
Resuming process 10926 (continue, signal 0, stop not expected)
stop pc (before any decrement) is 00de87d1
Got an event from 10927 (137f)
stop pc (before any decrement) is 00110402
Expected stop.
Resuming process 10927 (continue, signal 0, stop not expected)
stop pc (before any decrement) is 00110402
Got an event from 10927 (57f)
stop pc (before any decrement) is 08048438
stop pc (before any decrement) is 08048438
Hit a (non-reinsert) breakpoint.
Sending sigstop to process 10926
Got an event from 10926 (137f)
stop pc (before any decrement) is 00110402
Writing resume reply for -1208308848

Writing 55 to 00b9d7a0
Writing 8b to 08048437
---
# stop at first breakpoint here.
---
Resuming, no pending status
Resuming process 10927 (step, signal 0, stop not expected)
Checking for breakpoint in process 10927.
stop pc (before any decrement) is 08048437
Ignoring, PC was changed. Old PC was 0x08048437
stop pc (before any decrement) is 08048437
Got an event from 10927 (57f)
stop pc (before any decrement) is 0804843a
stop pc (before any decrement) is 0804843a
Hit a (non-reinsert) breakpoint.
Writing resume reply for -1208308848

Writing cc to 00b9d7a0
Writing cc to 08048437
Resuming, no pending status
Resuming process 10926 (continue, signal 0, stop not expected)
stop pc (before any decrement) is 00110402
Resuming process 10927 (continue, signal 0, stop not expected)
stop pc (before any decrement) is 0804843a
Got an event from 10926 (57f)
stop pc (before any decrement) is 00de87d1
stop pc (before any decrement) is 00de87d1
Hit a (non-reinsert) breakpoint.
Thread creation event. # The 2nd thread creation
Writing 00 to b7faadac
Writing 00 to 00dfa174
Writing 55 to 00de87d0 # no attach here...
Resuming process 10926 (step, signal 0, stop not expected)
pending reinsert at 00de87d0Checking for breakpoint in process 10926.
stop pc (before any decrement) is 00de87d1
Removed breakpoint.
set pc to 00de87d0
stop pc (before any decrement) is 00de87d0
Got an event from 10927 (0)
Thread -1208308848 (LWP 10927) exiting # detect death of 1st thread here???
Got an event from 10926 (57f)
stop pc (before any decrement) is 00de87d1
stop pc (before any decrement) is 00de87d1
Reinserted breakpoint.
Writing cc to 00de87d0
Resuming process 10926 (continue, signal 0, stop not expected)
stop pc (before any decrement) is 00de87d1
Got an event from 10926 (5) # terminated (not stopped) by bp
Thread -1208305984 (LWP 10926) exiting

Child terminated with signal = 5

Child terminated with signal = 0x5 (SIGTRAP)
GDBserver exiting

Program exited normally.
--- --- ---

After second "Thread creation event." message, there is no "Attaching
to thread ..." message. So the new thread did not attached, then
breakpoint just terminate (not stop) the program.

I confirmed that maybe_attach_thread() in thread-db.c did not call
linux_attach_lwp() on the second time. That was because
find_inferior_id() returned non NULL value. It seems
find_inferior_id() returned the first thread (which should be dead
before the second thread creation).

Any suggestion to how to fix this (or how this is fixed on current
CVS) are welcome. Thank you.