From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 15706 invoked by alias); 30 Apr 2009 16:20:08 -0000 Received: (qmail 15691 invoked by uid 22791); 30 Apr 2009 16:20:06 -0000 X-SWARE-Spam-Status: No, hits=-2.6 required=5.0 tests=BAYES_00,WEIRD_PORT X-Spam-Check-By: sourceware.org Received: from web38104.mail.mud.yahoo.com (HELO web38104.mail.mud.yahoo.com) (209.191.124.131) by sourceware.org (qpsmtpd/0.43rc1) with SMTP; Thu, 30 Apr 2009 16:19:58 +0000 Received: (qmail 21640 invoked by uid 60001); 30 Apr 2009 16:19:56 -0000 Message-ID: <481435.21529.qm@web38104.mail.mud.yahoo.com> Received: from [86.144.250.116] by web38104.mail.mud.yahoo.com via HTTP; Thu, 30 Apr 2009 09:19:55 PDT Date: Thu, 30 Apr 2009 16:30:00 -0000 From: x x Subject: Re: gdb stack trace unreadable for a pthreads application To: Paul Pluzhnikov Cc: gdb@sourceware.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-IsSubscribed: yes Mailing-List: contact gdb-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-owner@sourceware.org X-SW-Source: 2009-04/txt/msg00233.txt.bz2 Paul, Thank you for your reply. I am building and running mfsrv exactly from the same machine and same dire= ctory. And I am analyzing the core also from the same machine and same user= account. I am running on a chrooted system. Could this be the cause of the= path mismatch? How can I view which libraries it is using when running, an= d make sure that gdb uses the same? Ian > De: Paul Pluzhnikov > Assunto: Re: gdb stack trace unreadable for a pthreads application > Para: "x x" > Cc: gdb@sourceware.org > Data: Segunda-feira, 27 de Abril de 2009, 1:25 > On Sat, Apr 25, 2009 at 6:56 AM, x x > > wrote: >=20 > > I have this server app that creates a pool of 24 > threads. I had a core dump the other day and tried to > investigate the cause. >=20 > You didn't say whether SIGSEGV happened on the same host on > which you > are trying to analyze the core, but all clues indicate that > it didn't. >=20 > > I can view the stack trace of the thread that caused > the segmentation fault, but I cannot view the stack trace of > the other threads. >=20 > This is a typical symptom of libc.so.6 mismatch between > when core was > produced (on target?) and when it is being analyzed (on > host). >=20 > > The stack trace of the other threads is important for > me to determine the cause of the crash. Except for thread 2 > these other threads are blocked inside a pthread_mutex_lock > call. But the stack trace doesn't make any sense. > > Please view the gdb output in the bottom. > > Should I upgrade my gdb version? Can you help? > > > > Thanks > > > > Ian > > > > > > > > $ uname -a > > Linux ds5632 2.6.22-8-server #1 SMP Thu Jul 12 > 16:28:57 GMT 2007 i686 GNU/Linux > > > > $lsb_release -a > > No LSB modules are available. > > Distributor ID: Ubuntu > > Description: =A0 =A0Ubuntu 6.06 LTS > > Release: =A0 =A0 =A0 =A06.06 > > Codename: =A0 =A0 =A0 dapper > > > > $gdb mfsrv core.31589 > > > > GNU gdb 6.4-debian > > Copyright 2005 Free Software Foundation, Inc. > > GDB is free software, covered by the GNU General > Public License, and you are > > welcome to change it and/or distribute copies of it > under certain conditions. > > Type "show copying" to see the conditions. > > There is absolutely no warranty for GDB. =A0Type "show > warranty" for details. > > This GDB was configured as "i486-linux-gnu"...Using > host libthread_db library > "/lib/tls/i686/cmov/libthread_db.so.1". > > >=20 > So is this: >=20 > > Failed to read a valid object file image from memory. > > Core was generated by `./mfsrv mf MundialFoot 1'. > > Program terminated with signal 11, Segmentation > fault. > > >=20 > And this: >=20 > > warning: Can't read pathname for load map: > Input/output error. > > Reading symbols from /usr/lib/libstdc++.so.6...done. > > Loaded symbols for /usr/lib/libstdc++.so.6 > > Reading symbols from > /lib/tls/i686/cmov/libpthread.so.0...done. > > Loaded symbols for /lib/tls/i686/cmov/libpthread.so.0 > > Reading symbols from > /usr/lib/libmysqlclient_r.so.15...done. > > Loaded symbols for /usr/lib/libmysqlclient_r.so.15 > > Reading symbols from /usr/lib/libz.so.1...done. > > Loaded symbols for /usr/lib/libz.so.1 > > Reading symbols from > /lib/tls/i686/cmov/libcrypt.so.1...done. > > Loaded symbols for /lib/tls/i686/cmov/libcrypt.so.1 > > Reading symbols from > /lib/tls/i686/cmov/libnsl.so.1...done. > > Loaded symbols for /lib/tls/i686/cmov/libnsl.so.1 > > Reading symbols from > /lib/tls/i686/cmov/libm.so.6...done. > > Loaded symbols for /lib/tls/i686/cmov/libm.so.6 > > Reading symbols from /lib/libgcc_s.so.1...done. > > Loaded symbols for /lib/libgcc_s.so.1 > > Reading symbols from > /lib/tls/i686/cmov/libc.so.6...done. > > Loaded symbols for /lib/tls/i686/cmov/libc.so.6 > > Reading symbols from /lib/ld-linux.so.2...done. > > Loaded symbols for /lib/ld-linux.so.2 > > #0 =A00xb7b9c443 in memmove () from > /lib/tls/i686/cmov/libc.so.6 > > (gdb) info threads >=20 > Threads 2 through 24 are blocked in a system call (in > kernel VDSO page): >=20 > > =A024 process 31589 =A00xffffe410 in ?? () > > =A023 process 31591 =A00xffffe410 in ?? () > > =A022 process 31592 =A00xffffe410 in ?? () > > =A021 process 31593 =A00xffffe410 in ?? () > > =A020 process 31594 =A00xffffe410 in ?? () > > =A019 process 31595 =A00xffffe410 in ?? () > > =A018 process 31600 =A00xffffe410 in ?? () > > =A017 process 31601 =A00xffffe410 in ?? () > > =A016 process 31602 =A00xffffe410 in ?? () > > =A015 process 31603 =A00xffffe410 in ?? () > > =A014 process 31608 =A00xffffe410 in ?? () > > =A013 process 31609 =A00xffffe410 in ?? () > > =A012 process 31610 =A00xffffe410 in ?? () > > =A011 process 31611 =A00xffffe410 in ?? () > > =A010 process 31616 =A00xffffe410 in ?? () > > =A09 process 31617 =A00xffffe410 in ?? () > > =A08 process 31618 =A00xffffe410 in ?? () > > =A07 process 31619 =A00xffffe410 in ?? () > > =A06 process 31624 =A00xffffe410 in ?? () > > =A05 process 31625 =A00xffffe410 in ?? () > > =A04 process 31626 =A00xffffe410 in ?? () > > =A03 process 31632 =A00xffffe410 in ?? () > > =A02 process 31633 =A00xffffe410 in ?? () > > * 1 process 31627 =A00xb7b9c443 in memmove () from > /lib/tls/i686/cmov/libc.so.6 > > (gdb) bt 12 > > #0 =A00xb7b9c443 in memmove () from > /lib/tls/i686/cmov/libc.so.6 > > #1 =A00x08066b62 in IE_Sock::Recv (this=3D0x8098268, > pch=3D0x8098264 "", cbToRecv=3D4, iSecs=3D-1) at IE_Sock.cc:284 > > #2 =A00x08066d51 in IE_Sock::RecvInt (this=3D0x8098268, > pi=3D0x8098264, iSecs=3D-1) at IE_Sock.cc:257 > > #3 =A00x08070b3f in Thread::InitConnection > (this=3D0x8097dfc) at commands.cc:45 > > #4 =A00x08062d73 in Thread::Start (this=3D0x8097dfc) at > SMthreads.cc:195 > > #5 =A00x08062ec7 in Thread::Thread_Start (pv=3D0x8097dfc) > at SMthreads.cc:37 > > #6 =A00xb7e3f341 in start_thread () from > /lib/tls/i686/cmov/libpthread.so.0 > > #7 =A00xb7bfa4ee in clone () from > /lib/tls/i686/cmov/libc.so.6 > > (gdb) thread 2 > > [Switching to thread 2 (process 31633)]#0 =A00xffffe410 > in ?? () > > (gdb) bt 12 >=20 > Typical stack trace when libc.so.6 and libpthread.so.0 are > different > at core dump and at analysis time: >=20 > > #0 =A00xffffe410 in ?? () > > #1 =A00xacaf8258 in ?? () > > #2 =A00xb7e493b4 in ?? () from > /lib/tls/i686/cmov/libpthread.so.0 > > #3 =A00xacaf8220 in ?? () > > #4 =A00xb7e44778 in accept () from > /lib/tls/i686/cmov/libpthread.so.0 > > #5 =A00x080661fd in IE_SockSrv::Accept (this=3D0x8099530, > psock=3D0x8099100, sz=3D0x0, sl=3D0) at IE_Sock.cc:74 > > #6 =A00x0808039c in admin_thread::thread_start (pv=3D0x0) > at admin_thread.cc:44 > > #7 =A00xb7e3f341 in start_thread () from > /lib/tls/i686/cmov/libpthread.so.0 > > #8 =A00xb7bfa4ee in clone () from > /lib/tls/i686/cmov/libc.so.6 > > (gdb) thread 3 > > [Switching to thread 3 (process 31632)]#0 =A00xffffe410 > in ?? () > > (gdb) bt 12 > > #0 =A00xffffe410 in ?? () > > #1 =A00xad2f940c in ?? () > > #2 =A00x00000021 in ?? () > > #3 =A00x00000000 in ?? () > > > > and all remaining the threads (4 to 24) have the same > screwed up stack > > > > > > > > =A0 =A0 =A0Veja quais s=E3o os assuntos do momento no > Yahoo! +Buscados > > http://br.maisbuscados.yahoo.com > > >=20 > Assuming my guess of host/target mismatch is correct, the > solution is: >=20 > gdb mfsrv > (gdb) set solib-absolute-prefix /path/to/target/root > =A0 =A0 # libc.so.6, libpthread.so.0 should be in > /path/to/target/root/lib/tls/i686/cmov > (gdb) core core.31589 >=20 > Cheers, > --=20 > Paul Pluzhnikov >=20 Veja quais s=E3o os assuntos do momento no Yahoo! +Buscados http://br.maisbuscados.yahoo.com