GDB receives SIGKILL

GDB receives SIGKILL

[Vadim Shmelev]

Good day.
I'm novice in Linux and during of application debugging that linked
statically I bump into the following problem: debugger receives SIGKILL and
exits. Application uses glibc 2.3.2, gcc 4.1.1 and Linux threads. I will
appreciate your comments.

Regards, Vadim

Re: GDB receives SIGKILL

[Baurzhan Ismagulov]

Hello Vadim,

On Mon, Apr 02, 2007 at 11:57:01AM +0400, Vadim Shmelev wrote:
> I'm novice in Linux and during of application debugging that linked
> statically I bump into the following problem: debugger receives SIGKILL and
> exits.

Are you using a front-end before gdb? If yes, you could strace it to see
whether it kills gdb. If not, check the kernel logs for the OOM killer.

With kind regards,
-- 
Baurzhan Ismagulov
http://www.kz-easy.com/

RE: GDB receives SIGKILL

[Vadim Shmelev]

Thanks for response, Baurzhan. I will try to describe the problem in
details. Application works satisfactorily, but some results are unexpected.
Strace works normally too. Problem occurs during debug process only. RAM is
2 Gb and there is no out of memory messages in log. There is output from gdb
session:

GNU gdb Red Hat Linux (6.3.0.0-1.84rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.

This GDB was configured as "i386-asplinux-linux-gnu"...Using host
libthread_db library "/lib/libthread_db.so.1".

(gdb) handle SIG32 nostop noprint
Signal        Stop      Print   Pass to program Description
SIG32         No        No      Yes             Real-time event 32
(gdb) handle SIG33 nostop noprint
Signal        Stop      Print   Pass to program Description
SIG33         No        No      Yes             Real-time event 33

(gdb) b
'ComputerManagement::CreateLocalLinuxHardwareInfo(Fomatik::Computer*)'
Breakpoint 1 at 0x9053fe6: file ../../../include/processor/ref/allocator.h,
line 23.
(gdb) r
Starting program: /mnt/afc_agent.exe
warning: shared library handler failed to enable breakpoint
Reading symbols from shared object read from target memory...done.
Loaded system supplied DSO at 0xffffe000
Detaching after fork from child process 2322.
Detaching after fork from child process 2325.

Program terminated with signal SIGKILL, Killed.
The program no longer exists.
Current language:  auto; currently c++

The next output string looks strange ' warning: shared library handler
failed to enable breakpoint' as application is linked statically. Any
advises?

With best regards, Vadim

-----Original Message-----
From: gdb-owner@sourceware.org [mailto:gdb-owner@sourceware.org] On Behalf
Of Baurzhan Ismagulov
Sent: Monday, April 02, 2007 2:01 PM
To: gdb@sourceware.org
Subject: Re: GDB receives SIGKILL

Hello Vadim,

On Mon, Apr 02, 2007 at 11:57:01AM +0400, Vadim Shmelev wrote:
> I'm novice in Linux and during of application debugging that linked
> statically I bump into the following problem: debugger receives SIGKILL
and
> exits.

Are you using a front-end before gdb? If yes, you could strace it to see
whether it kills gdb. If not, check the kernel logs for the OOM killer.

With kind regards,
-- 
Baurzhan Ismagulov
http://www.kz-easy.com/

RE: GDB receives SIGKILL

[Michael Snyder]

On Mon, 2007-04-02 at 17:47 +0400, Vadim Shmelev wrote:
> Thanks for response, Baurzhan. I will try to describe the problem in

> 
> (gdb) handle SIG32 nostop noprint
> Signal        Stop      Print   Pass to program Description
> SIG32         No        No      Yes             Real-time event 32
> (gdb) handle SIG33 nostop noprint
> Signal        Stop      Print   Pass to program Description
> SIG33         No        No      Yes             Real-time event 33

Why did you do the above?  Those signals are part of 
linuxthreads handling and debugging.  There should be 
no need to silence them -- if there is, it could be a 
sign of something wrong.

> 
> (gdb) b
> 'ComputerManagement::CreateLocalLinuxHardwareInfo(Fomatik::Computer*)'
> Breakpoint 1 at 0x9053fe6: file ../../../include/processor/ref/allocator.h,
> line 23.
> (gdb) r
> Starting program: /mnt/afc_agent.exe
> warning: shared library handler failed to enable breakpoint

This is odd, but may have to do with libthread_db being linked static.
I forget -- do we have linuxthreads debugging working with static link?
[question for the other maintainers]

> Reading symbols from shared object read from target memory...done.
> Loaded system supplied DSO at 0xffffe000

What's this?  System supplied DSO?

> Detaching after fork from child process 2322.
> Detaching after fork from child process 2325.

Ah... fork?  You know it is dangerous to mix threads and forks, don't
you?  If your process has spawned threads before it forks, you're 
almost guaranteed to be in bad trouble.

I didn't see any msgs from gdb saying a thread had been spawned, 
but the thread detection might be broken eg. by the fact that we
are linked statically.

Can you try dynamic linking, at least of the libthread_db library?

> 
> Program terminated with signal SIGKILL, Killed.
> The program no longer exists.
> Current language:  auto; currently c++
> 
> The next output string looks strange ' warning: shared library handler
> failed to enable breakpoint' as application is linked statically. Any
> advises?

RE: GDB receives SIGKILL

[Vadim Shmelev]

Thanks for comments, Michael. Below I try to comment your comments.

> 
> (gdb) handle SIG32 nostop noprint
> Signal        Stop      Print   Pass to program Description
> SIG32         No        No      Yes             Real-time event 32
> (gdb) handle SIG33 nostop noprint
> Signal        Stop      Print   Pass to program Description
> SIG33         No        No      Yes             Real-time event 33

Why did you do the above?  Those signals are part of 
linuxthreads handling and debugging.  There should be 
no need to silence them -- if there is, it could be a 
sign of something wrong.

-COMMENT-------------------------------------------------------------
If I enable SIG32 and SIG33 gdb will be stopped at every thread switch.
-END COMMENT---------------------------------------------------------

> 
> (gdb) b
> 'ComputerManagement::CreateLocalLinuxHardwareInfo(Fomatik::Computer*)'
> Breakpoint 1 at 0x9053fe6: file
../../../include/processor/ref/allocator.h,
> line 23.
> (gdb) r
> Starting program: /mnt/afc_agent.exe
> warning: shared library handler failed to enable breakpoint

This is odd, but may have to do with libthread_db being linked static.

-COMMENT-------------------------------------------------------------
Yes, that library is statically linked

#ldd my_prog
         not a dynamic executable
-END COMMENT---------------------------------------------------------

I forget -- do we have linuxthreads debugging working with static link?
[question for the other maintainers]

-COMMENT---------------------------------------------------------------
I'm also interested very much in answer.
-END COMMENT-----------------------------------------------------------


> Reading symbols from shared object read from target memory...done.
> Loaded system supplied DSO at 0xffffe000

What's this?  System supplied DSO?

-COMMENT---------------------------------------------------------------
I suspect that gdb doesn't correctly guess that it works with static
executable.
-END COMMENT-----------------------------------------------------------

> Detaching after fork from child process 2322.
> Detaching after fork from child process 2325.

Ah... fork?  You know it is dangerous to mix threads and forks, don't
you?  If your process has spawned threads before it forks, you're 
almost guaranteed to be in bad trouble.

-COMMENT---------------------------------------------------------------
Yes, I know. The forks are really used for "exec".
-END COMMENT-----------------------------------------------------------

I didn't see any msgs from gdb saying a thread had been spawned, 
but the thread detection might be broken eg. by the fact that we
are linked statically.

Can you try dynamic linking, at least of the libthread_db library?

-COMMENT---------------------------------------------------------------
The dynamically linked application works fine with gdb. The problem occurs
with statically linked one.
-END COMMENT-----------------------------------------------------------

> 
> Program terminated with signal SIGKILL, Killed.
> The program no longer exists.
> Current language:  auto; currently c++
> 
> The next output string looks strange ' warning: shared library handler
> failed to enable breakpoint' as application is linked statically. Any
> advises?

Re: GDB receives SIGKILL

[Daniel Jacobowitz]

On Tue, Apr 03, 2007 at 11:15:59AM +0400, Vadim Shmelev wrote:
> -COMMENT-------------------------------------------------------------
> If I enable SIG32 and SIG33 gdb will be stopped at every thread switch.
> -END COMMENT---------------------------------------------------------

Figure out why.  If this is happening, GDB thread support is not
working, and you can not expect anything to behave properly.

> I forget -- do we have linuxthreads debugging working with static link?
> [question for the other maintainers]
> 
> -COMMENT---------------------------------------------------------------
> I'm also interested very much in answer.
> -END COMMENT-----------------------------------------------------------

Sometimes yes, sometimes no.  Usually no.  There are patches for this
problem in the glibc bugzilla that have still not been applied.

> > Reading symbols from shared object read from target memory...done.
> > Loaded system supplied DSO at 0xffffe000
> 
> What's this?  System supplied DSO?

It's the vDSO.

-- 
Daniel Jacobowitz
CodeSourcery

RE: GDB receives SIGKILL

[Michael Snyder]

On Tue, 2007-04-03 at 11:15 +0400, Vadim Shmelev wrote:
> Thanks for comments, Michael. Below I try to comment your comments.
> 
> > 
> > (gdb) handle SIG32 nostop noprint
> > Signal        Stop      Print   Pass to program Description
> > SIG32         No        No      Yes             Real-time event 32
> > (gdb) handle SIG33 nostop noprint
> > Signal        Stop      Print   Pass to program Description
> > SIG33         No        No      Yes             Real-time event 33
> 
> Why did you do the above?  Those signals are part of 
> linuxthreads handling and debugging.  There should be 
> no need to silence them -- if there is, it could be a 
> sign of something wrong.
> 
> -COMMENT-------------------------------------------------------------
> If I enable SIG32 and SIG33 gdb will be stopped at every thread switch.
> -END COMMENT---------------------------------------------------------

Well, this indicates that something is wrong right there.
There is some mis-match between gdb, the target, and the  
target thread libraries.

I see that you have a modern gcc (4.1.1), but an older glibc (2.3.5)
and a fairly old gdb (6.3 something, circa 2004).  Are you on an older
Red Hat distribution, and have you perhaps added a newer gcc to it?

Are you sure that your glibc is configured for old-style linux 
threads, and not NPTL threads?

Have all of your glibc components (including libpthread and 
libthread_db) been configured and built together?  Or have
some of them been changed?



What about the possibility of linking most of your libraries
statically, but linking libpthread and/or libthread_db
dynamically?

RE: GDB receives SIGKILL

[Vadim Shmelev]

Thanks for comments, Daniel and Michael. I will try to describe my
environment.

For statically building of application I use gcc 4.1.1, glibc 2.3.2 and all
glibc components configured and built together. NPTL threads are not used.
Target platform is VMware ESX Server with 2.4.21 kernel, accordingly dynamic
linking doesn't match, as glibc's conflict is possible. How I can realize,
that problem doesn't have decision, but I should know possible reasons for
receiving SIG32 and SIG33 signals.

Best regards, Vadim

RE: GDB receives SIGKILL

[Michael Snyder]

On Wed, 2007-04-04 at 16:50 +0400, Vadim Shmelev wrote:
> Thanks for comments, Daniel and Michael. I will try to describe my
> environment.
> 
> For statically building of application I use gcc 4.1.1, glibc 2.3.2 and all
> glibc components configured and built together. NPTL threads are not used.
> Target platform is VMware ESX Server with 2.4.21 kernel, accordingly dynamic
> linking doesn't match, as glibc's conflict is possible. How I can realize,
> that problem doesn't have decision, but I should know possible reasons for
> receiving SIG32 and SIG33 signals.

Kernel 2.<b>4</b>.21?   That kernel may possibly be too old.
I don't know for sure.

The reason for receiving the sig32 and sig33 is because the libpthreads
library uses them for inter-thread communication and synchronization.

If everything is working properly, GDB intercepts them and understands
some information from them, giving gdb an insight into what the thread
library is doing.  You (the user) should never see them or be aware of
them.

There is some underlying problem that is preventing this system
(kernel -- glibc -- libpthreads -- gdb) from working properly.