From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 1032 invoked by alias); 29 Aug 2006 10:42:49 -0000 Received: (qmail 1021 invoked by uid 22791); 29 Aug 2006 10:42:47 -0000 X-Spam-Check-By: sourceware.org Received: from smtpsrv20.tdc.dk (HELO smtpsrv20.tdc.dk) (192.66.25.157) by sourceware.org (qpsmtpd/0.31) with ESMTP; Tue, 29 Aug 2006 10:42:44 +0000 Received: from conbr321.tdk.dk ([10.74.47.33]) by smtpsrv20.tdc.dk with Microsoft SMTPSVC(5.0.2195.6982); Tue, 29 Aug 2006 12:45:18 +0200 Received: from conbr102.tdk.dk ([10.74.17.55]) by conbr321.tdk.dk with InterScan Messaging Security Suite; Tue, 29 Aug 2006 12:42:40 +0200 Received: from VESTMB403A.tdk.dk ([10.106.170.54]) by conbr102.tdk.dk with Microsoft SMTPSVC(5.0.2195.6713); Tue, 29 Aug 2006 12:42:40 +0200 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: Which thread caused dump? Date: Tue, 29 Aug 2006 10:42:00 -0000 Message-ID: <2B1831460E7C6B46A19066A0E9147A4E0C173F@VESTMB403A.tdk.dk> In-Reply-To: <1156847520.29200.ezmlm@sourceware.org> From: "Steffen Schumacher" To: X-IsSubscribed: yes Mailing-List: contact gdb-help@sourceware.org; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-owner@sourceware.org X-SW-Source: 2006-08/txt/msg00207.txt.bz2 Hi! I have a multithreaded program which coredumps after ~20 hours of running. Is it possible to see which thread caused the crash? Usually I use thr x and bt for seeing if some exception was thrown, but I have a case where no thread is throwing any exceptions. I've enclosed output from the actual core dump.. any other cmd's I should know of? Any help on discovering why it is crashing would be greatly appreciated.. /Steffen testnxq2# uname -a FreeBSD testnxq2.xxx.dk 6.1-RELEASE FreeBSD 6.1-RELEASE #0: Sun May 7 04:32:43 UTC 2006 root@opus.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 testnxq2# gdb measuring measuring.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"... Core was generated by `measuring'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/libpcap.so.4...done. Loaded symbols for /usr/lib/libpcap.so.4 Reading symbols from /usr/local/lib/libqosmysql.so.1...done. Loaded symbols for /usr/local/lib/libqosmysql.so.1 Reading symbols from /usr/local/lib/libqosthreads.so.1...done. Loaded symbols for /usr/local/lib/libqosthreads.so.1 Reading symbols from /usr/local/lib/libqoslog.so.1...done. Loaded symbols for /usr/local/lib/libqoslog.so.1 Reading symbols from /usr/local/lib/libqoslegacynetstuff.so.1...done. Loaded symbols for /usr/local/lib/libqoslegacynetstuff.so.1 Reading symbols from /usr/local/lib/libqoscommon.so.1...done. Loaded symbols for /usr/local/lib/libqoscommon.so.1 Reading symbols from /usr/local/lib/libqosmeasuring.so.1...done. Loaded symbols for /usr/local/lib/libqosmeasuring.so.1 Reading symbols from /usr/local/lib/libqosdb.so.1...done. Loaded symbols for /usr/local/lib/libqosdb.so.1 Reading symbols from /usr/local/lib/libqospacket.so.1...done. Loaded symbols for /usr/local/lib/libqospacket.so.1 Reading symbols from /usr/local/lib/libqosaggregation.so.1...done. Loaded symbols for /usr/local/lib/libqosaggregation.so.1 Reading symbols from /usr/lib/libstdc++.so.5...done. Loaded symbols for /usr/lib/libstdc++.so.5 Reading symbols from /lib/libm.so.4...done. Loaded symbols for /lib/libm.so.4 Reading symbols from /usr/lib/libpthread.so.2...done. Loaded symbols for /usr/lib/libpthread.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /usr/local/lib/libmysqlpp.so.2...done. Loaded symbols for /usr/local/lib/libmysqlpp.so.2 Reading symbols from /usr/local/lib/libqoscmd.so.1...done. Loaded symbols for /usr/local/lib/libqoscmd.so.1 Reading symbols from /lib/libz.so.3...done. Loaded symbols for /lib/libz.so.3 Reading symbols from /usr/local/lib/mysql/libmysqlclient.so.15...done. Loaded symbols for /usr/local/lib/mysql/libmysqlclient.so.15 Reading symbols from /lib/libcrypt.so.3...done. Loaded symbols for /lib/libcrypt.so.3 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x2834f4ab in pthread_testcancel () from /usr/lib/libpthread.so.2 [New Thread 0x806bc00 (sleeping)] [New Thread 0x806ba00 (runnable)] [New Thread 0x806b800 (sleeping)] [New Thread 0x806b600 (runnable)] [New Thread 0x8057800 (runnable)] [New Thread 0x8057600 (runnable)] [New Thread 0x8057400 (runnable)] [New Thread 0x8057200 (LWP 100041)] [New Thread 0x8057000 (sleeping)] [New LWP 100080] (gdb) info threads * 10 LWP 100080 0x2834f4ab in pthread_testcancel () from /usr/lib/libpthread.so.2 9 Thread 0x8057000 (sleeping) 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 8 Thread 0x8057200 (LWP 100041) 0x2834f46b in pthread_testcancel () from /usr/lib/libpthread.so.2 7 Thread 0x8057400 (runnable) 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 6 Thread 0x8057600 (runnable) 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 5 Thread 0x8057800 (runnable) 0x282960de in std::_Rb_tree_increment () from /usr/lib/libstdc++.so.5 4 Thread 0x806b600 (runnable) 0x2840c7df in read () from /lib/libc.so.6 3 Thread 0x806b800 (sleeping) 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 2 Thread 0x806ba00 (runnable) 0x283ffacb in gettimeofday () from /lib/libc.so.6 1 Thread 0x806bc00 (sleeping) 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 (gdb) thr 1 [Switching to thread 1 (Thread 0x806bc00 (sleeping))]#0 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 (gdb) bt #0 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 #1 0x283419ae in _nanosleep () from /usr/lib/libpthread.so.2 #2 0x283d7669 in sleep () from /lib/libc.so.6 #3 0x283373aa in sleep () from /usr/lib/libpthread.so.2 #4 0x280c4b26 in WatchDog::execute (this=3D0x805a2c0) at WatchDog.cpp:138 #5 0x2813d015 in IThread::entrypoint (pthis=3D0x805a2c0) at IThread.h:175 #6 0x28340319 in pthread_create () from /usr/lib/libpthread.so.2 #7 0x283f4637 in _ctx_start () from /lib/libc.so.6 (gdb) thr 2 [Switching to thread 2 (Thread 0x806ba00 (runnable))]#0 0x283ffacb in gettimeofday () from /lib/libc.so.6 (gdb) bt #0 0x283ffacb in gettimeofday () from /lib/libc.so.6 #1 0x283f31de in time () from /lib/libc.so.6 #2 0x280d4161 in LogSender::lsWrite (this=3D0x8051d00, message=3D {static npos =3D 4294967295, _M_dataplus =3D {> =3D {<__gnu_cxx::new_allocator> =3D {}, }, _M_p =3D 0x8071a2c "sending 3 results"}}, level=3D7 '\a') at LogSender.cpp:25 #3 0x280d40a5 in LogSender::lsWrite (this=3D0x8051d00, message=3D {static npos =3D 4294967295, _M_dataplus =3D {> =3D {<__gnu_cxx::new_allocator> =3D {}, }, _M_p =3D 0x7
}}) at LogSender.cpp:17 #4 0x280d43aa in LogSender::bufferChar (this=3D0x8051d00, c=3D10 '\n') at LogSender.cpp:63 #5 0x280d4e74 in LogSender::LogStreambuf::overflow (this=3D0x8051d18, c=3D10) at LogSender.h:119 #6 0x282c13ff in std::basic_streambuf >::xsputn () from /usr/lib/libstdc++.so.5 #7 0x282b78eb in std::operator<< > () from /usr/lib/libstdc++.so.5 #8 0x2820a415 in ResultHandler::execute (this=3D0x806c400) at ResultHandler.cpp:85 #9 0x2813cf65 in IMonitoredThread::entrypoint (pthis=3D0x806c400) at IMonitoredThread.h:217 #10 0x28340319 in pthread_create () from /usr/lib/libpthread.so.2 #11 0x283f4637 in _ctx_start () from /lib/libc.so.6 (gdb) thr 3 [Switching to thread 3 (Thread 0x806b800 (sleeping))]#0 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 (gdb) bt #0 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 #1 0x283419ae in _nanosleep () from /usr/lib/libpthread.so.2 #2 0x283f31b7 in usleep () from /lib/libc.so.6 #3 0x283373f6 in usleep () from /usr/lib/libpthread.so.2 #4 0x2812fb04 in PacketScheduler::execute (this=3D0x806c100) at PacketScheduler.cpp:104 #5 0x2813ce81 in IMonitoredThread::entrypoint (pthis=3D0x806c100) at IMonitoredThread.h:217 #6 0x28340319 in pthread_create () from /usr/lib/libpthread.so.2 #7 0x283f4637 in _ctx_start () from /lib/libc.so.6 (gdb) thr 4 [Switching to thread 4 (Thread 0x806b600 (runnable))]#0 0x2840c7df in read () from /lib/libc.so.6 (gdb) bt #0 0x2840c7df in read () from /lib/libc.so.6 #1 0x2833819a in read () from /usr/lib/libpthread.so.2 #2 0x2808a5ac in pcap_lookupnet () from /usr/lib/libpcap.so.4 #3 0x2808b57e in pcap_dispatch () from /usr/lib/libpcap.so.4 #4 0x2808b666 in pcap_next () from /usr/lib/libpcap.so.4 #5 0x2814cc60 in Receiver::getPacket (this=3D0x8088000, pac=3D0xbf5faf2c, t=3D@0xbf5faf10) at Receiver.cpp:94 #6 0x2814cdd2 in Receiver::serviceQueue (this=3D0x8088000) at Receiver.cpp:144 #7 0x2814cb2c in Receiver::execute (this=3D0x8088000) at Receiver.cpp:72 #8 0x2813cd9d in IMonitoredThread::entrypoint (pthis=3D0x8088000) at IMonitoredThread.h:217 #9 0x28340319 in pthread_create () from /usr/lib/libpthread.so.2 #10 0x283f4637 in _ctx_start () from /lib/libc.so.6 (gdb) thr 5 [Switching to thread 5 (Thread 0x8057800 (runnable))]#0 0x282960de in std::_Rb_tree_increment () from /usr/lib/libstdc++.so.5 (gdb) bt #0 0x282960de in std::_Rb_tree_increment () from /usr/lib/libstdc++.so.5 #1 0x2813c0e6 in std::_Rb_tree_iterator > > >::operator++ ( this=3D0xbf6fbf30) at stl_tree.h:189 #2 0x28135ba8 in Measuring::execute (this=3D0x805d480) at Measuring.cpp:114 #3 0x0804c13a in IMonitoredThread::entrypoint (pthis=3D0x805d480) at IMonitoredThread.h:217 #4 0x28340319 in pthread_create () from /usr/lib/libpthread.so.2 #5 0x283f4637 in _ctx_start () from /lib/libc.so.6 (gdb) thr 6 [Switching to thread 6 (Thread 0x8057600 (runnable))]#0 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 (gdb) bt #0 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 #1 0x283419ae in _nanosleep () from /usr/lib/libpthread.so.2 #2 0x283d7669 in sleep () from /lib/libc.so.6 #3 0x283373aa in sleep () from /usr/lib/libpthread.so.2 #4 0x280d5fa2 in LogPostOffice::execute (this=3D0x805a040) at LogPostOffice.cpp:75 #5 0x280d9bb9 in IMonitoredThread::entrypoint (pthis=3D0x805a040) at IMonitoredThread.h:217 #6 0x28340319 in pthread_create () from /usr/lib/libpthread.so.2 #7 0x283f4637 in _ctx_start () from /lib/libc.so.6 (gdb) tht 7 Undefined command: "tht". Try "help". (gdb) thr 7 [Switching to thread 7 (Thread 0x8057400 (runnable))]#0 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 (gdb) bt #0 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 #1 0x283419ae in _nanosleep () from /usr/lib/libpthread.so.2 #2 0x28337fce in select () from /usr/lib/libpthread.so.2 #3 0x00000000 in ?? () (gdb) thr 8 [Switching to thread 8 (Thread 0x8057200 (LWP 100041))]#0 0x2834f46b in pthread_testcancel () from /usr/lib/libpthread.so.2 (gdb) bt #0 0x2834f46b in pthread_testcancel () from /usr/lib/libpthread.so.2 #1 0x28347e3c in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 #2 0x284fa450 in ?? () (gdb) thr 9 [Switching to thread 9 (Thread 0x8057000 (sleeping))]#0 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 (gdb) bt #0 0x28347f0f in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 #1 0x283419ae in _nanosleep () from /usr/lib/libpthread.so.2 #2 0x283d7669 in sleep () from /lib/libc.so.6 #3 0x283373aa in sleep () from /usr/lib/libpthread.so.2 #4 0x0804a7ae in debug () at main.cpp:92 #5 0x0804a365 in main () at main.cpp:54 (gdb) thr 10 [Switching to thread 10 (LWP 100080)]#0 0x2834f46b in pthread_testcancel () from /usr/lib/libpthread.so.2 (gdb) bt #0 0x2834f46b in pthread_testcancel () from /usr/lib/libpthread.so.2 #1 0x28347e3c in pthread_mutexattr_init () from /usr/lib/libpthread.so.2 #2 0x284fa450 in ?? () (gdb)