core while attached

core while attached

[Bennett F. Dill]

I hope that someone out there has the patience to look at my problem :-)

Hi, I'm working with gdb and I have the -g option on my application.
Typically I get very descriptive error messages.  This latest problem
however is really interesting though...  It seems to be a problem with
libc.so.1...

[history]
Anyway, the application runs in the background and is a server to about
25 clients concurrently.  It takes a few hours for the app to sigsegv.
Now, it never dumped a core for some reason, so i just attach gdb to the
process and continue and wait for the error to rear its head.  Now,
since the app serves ~25 people, I don't really have time to look around
with the debugger...  (I'm new to gdb, noo suprise I'm sure)
[/history]

So my question is, is it possible to have the gdb that is attached to a
running process that has hit a signal sigsegv, to dump a core?  This
way I can leisurly mull over the data.

More questions I have (totally n00bishness I'm sure), is how can I
find out what line of code in my app, called the 'deadly' function in
libc.so.1 (fprintf) and is there any way to view the call to fprintf
based on the memory address?

Below is what I get from a where or a BT etc...  If I try to use a
'return' I go no where real fast.

TIA for your assistance :-)
Ben



Program received signal SIGSEGV, Segmentation fault.
0xff30ceb8 in _getorientation () from /usr/lib/libc.so.1
(gdb) where
#0  0xff30ceb8 in _getorientation () from /usr/lib/libc.so.1
#1  0xff30d0dc in _set_orientation_byte () from /usr/lib/libc.so.1
#2  0xff300710 in fprintf () from /usr/lib/libc.so.1
#3  0x3aa80 in ?? ()
#4  0x30420 in ?? ()
#5  0x2f6f0 in ?? ()
#6  0x2c7c8 in ?? ()
#7  0x2c258 in ?? ()
(gdb) 

Re: core while attached

[Daniel Jacobowitz]

On Mon, Jan 27, 2003 at 01:32:52PM -0500, Bennett F. Dill wrote:
> I hope that someone out there has the patience to look at my problem :-)
> 
> Hi, I'm working with gdb and I have the -g option on my application.
> Typically I get very descriptive error messages.  This latest problem
> however is really interesting though...  It seems to be a problem with
> libc.so.1...
> 
> [history]
> Anyway, the application runs in the background and is a server to about
> 25 clients concurrently.  It takes a few hours for the app to sigsegv.
> Now, it never dumped a core for some reason, so i just attach gdb to the
> process and continue and wait for the error to rear its head.  Now,
> since the app serves ~25 people, I don't really have time to look around
> with the debugger...  (I'm new to gdb, noo suprise I'm sure)
> [/history]
> 
> So my question is, is it possible to have the gdb that is attached to a
> running process that has hit a signal sigsegv, to dump a core?  This
> way I can leisurly mull over the data.

You're in luck.  Grab a recent version of GDB and use the
"generate-core-file" command.

> More questions I have (totally n00bishness I'm sure), is how can I
> find out what line of code in my app, called the 'deadly' function in
> libc.so.1 (fprintf) and is there any way to view the call to fprintf
> based on the memory address?
> 
> Below is what I get from a where or a BT etc...  If I try to use a
> 'return' I go no where real fast.
> 
> TIA for your assistance :-)
> Ben
> 
> 
> 
> Program received signal SIGSEGV, Segmentation fault.
> 0xff30ceb8 in _getorientation () from /usr/lib/libc.so.1
> (gdb) where
> #0  0xff30ceb8 in _getorientation () from /usr/lib/libc.so.1
> #1  0xff30d0dc in _set_orientation_byte () from /usr/lib/libc.so.1
> #2  0xff300710 in fprintf () from /usr/lib/libc.so.1
> #3  0x3aa80 in ?? ()
> #4  0x30420 in ?? ()
> #5  0x2f6f0 in ?? ()
> #6  0x2c7c8 in ?? ()
> #7  0x2c258 in ?? ()
> (gdb) 

That really looks like you're missing debug info; I don't know if those
are legitimate code addresses on your platform or not.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: core while attached

[Bennett F. Dill]

On Mon, Jan 27, 2003 at 02:19:30PM -0500, Daniel Jacobowitz wrote:
> On Mon, Jan 27, 2003 at 01:32:52PM -0500, Bennett F. Dill wrote:
> > I hope that someone out there has the patience to look at my problem :-)
> > 
> > Hi, I'm working with gdb and I have the -g option on my application.
> > Typically I get very descriptive error messages.  This latest problem
> > however is really interesting though...  It seems to be a problem with
> > libc.so.1...
> > 
> > [history]
> > Anyway, the application runs in the background and is a server to about
> > 25 clients concurrently.  It takes a few hours for the app to sigsegv.
> > Now, it never dumped a core for some reason, so i just attach gdb to the
> > process and continue and wait for the error to rear its head.  Now,
> > since the app serves ~25 people, I don't really have time to look around
> > with the debugger...  (I'm new to gdb, noo suprise I'm sure)
> > [/history]
> > 
> > So my question is, is it possible to have the gdb that is attached to a
> > running process that has hit a signal sigsegv, to dump a core?  This
> > way I can leisurly mull over the data.
> 
> You're in luck.  Grab a recent version of GDB and use the
> "generate-core-file" command.
> 
> > More questions I have (totally n00bishness I'm sure), is how can I
> > find out what line of code in my app, called the 'deadly' function in
> > libc.so.1 (fprintf) and is there any way to view the call to fprintf
> > based on the memory address?
> > 
> > Below is what I get from a where or a BT etc...  If I try to use a
> > 'return' I go no where real fast.
> > 
> > TIA for your assistance :-)
> > Ben
> > 
> > 
> > 
> > Program received signal SIGSEGV, Segmentation fault.
> > 0xff30ceb8 in _getorientation () from /usr/lib/libc.so.1
> > (gdb) where
> > #0  0xff30ceb8 in _getorientation () from /usr/lib/libc.so.1
> > #1  0xff30d0dc in _set_orientation_byte () from /usr/lib/libc.so.1
> > #2  0xff300710 in fprintf () from /usr/lib/libc.so.1
> > #3  0x3aa80 in ?? ()
> > #4  0x30420 in ?? ()
> > #5  0x2f6f0 in ?? ()
> > #6  0x2c7c8 in ?? ()
> > #7  0x2c258 in ?? ()
> > (gdb) 
> 
> That really looks like you're missing debug info; I don't know if those
> are legitimate code addresses on your platform or not.
> 
> -- 
> Daniel Jacobowitz
> MontaVista Software                         Debian GNU/Linux Developer
Well...  libc.so.1 (i think) isn't compiled with debug info in it.  So
that's why that's missing (i think)...  Usually, I see the code, the
values, and the code line number, and what file the code is in...

I guess i could try to get some sa to recompile libc.so.1 with debugging
info in it, but im not going to hold my breath.

Re: core while attached

[Bennett F. Dill]

On Mon, Jan 27, 2003 at 02:19:30PM -0500, Daniel Jacobowitz wrote:
> On Mon, Jan 27, 2003 at 01:32:52PM -0500, Bennett F. Dill wrote:
> > I hope that someone out there has the patience to look at my problem :-)
> > 
> > Hi, I'm working with gdb and I have the -g option on my application.
> > Typically I get very descriptive error messages.  This latest problem
> > however is really interesting though...  It seems to be a problem with
> > libc.so.1...
> > 
> > [history]
> > Anyway, the application runs in the background and is a server to about
> > 25 clients concurrently.  It takes a few hours for the app to sigsegv.
> > Now, it never dumped a core for some reason, so i just attach gdb to the
> > process and continue and wait for the error to rear its head.  Now,
> > since the app serves ~25 people, I don't really have time to look around
> > with the debugger...  (I'm new to gdb, noo suprise I'm sure)
> > [/history]
> > 
> > So my question is, is it possible to have the gdb that is attached to a
> > running process that has hit a signal sigsegv, to dump a core?  This
> > way I can leisurly mull over the data.
> 
> You're in luck.  Grab a recent version of GDB and use the
> "generate-core-file" command.
> 
> > More questions I have (totally n00bishness I'm sure), is how can I
> > find out what line of code in my app, called the 'deadly' function in
> > libc.so.1 (fprintf) and is there any way to view the call to fprintf
> > based on the memory address?
> > 
> > Below is what I get from a where or a BT etc...  If I try to use a
> > 'return' I go no where real fast.
> > 
> > TIA for your assistance :-)
> > Ben
> > 
> > 
> > 
> > Program received signal SIGSEGV, Segmentation fault.
> > 0xff30ceb8 in _getorientation () from /usr/lib/libc.so.1
> > (gdb) where
> > #0  0xff30ceb8 in _getorientation () from /usr/lib/libc.so.1
> > #1  0xff30d0dc in _set_orientation_byte () from /usr/lib/libc.so.1
> > #2  0xff300710 in fprintf () from /usr/lib/libc.so.1
> > #3  0x3aa80 in ?? ()
> > #4  0x30420 in ?? ()
> > #5  0x2f6f0 in ?? ()
> > #6  0x2c7c8 in ?? ()
> > #7  0x2c258 in ?? ()
> > (gdb) 
> 
> That really looks like you're missing debug info; I don't know if those
> are legitimate code addresses on your platform or not.
> 
> -- 
> Daniel Jacobowitz
> MontaVista Software                         Debian GNU/Linux Developer
I'm ready to ask for a new version of gdb (5.3 is the latest i see), but
I don't see any documentation on generate-core-file anywhere...  Before
I ask for a new version, can someone verify that generate-core-file is
the correct syntax/command?  All I see is how to switch core files
etc...

TIA,
Ben

Re: core while attached

[Daniel Jacobowitz]

On Mon, Jan 27, 2003 at 03:58:15PM -0500, Bennett F. Dill wrote:
> On Mon, Jan 27, 2003 at 02:19:30PM -0500, Daniel Jacobowitz wrote:
> > On Mon, Jan 27, 2003 at 01:32:52PM -0500, Bennett F. Dill wrote:
> > > I hope that someone out there has the patience to look at my problem :-)
> > > 
> > > Hi, I'm working with gdb and I have the -g option on my application.
> > > Typically I get very descriptive error messages.  This latest problem
> > > however is really interesting though...  It seems to be a problem with
> > > libc.so.1...
> > > 
> > > [history]
> > > Anyway, the application runs in the background and is a server to about
> > > 25 clients concurrently.  It takes a few hours for the app to sigsegv.
> > > Now, it never dumped a core for some reason, so i just attach gdb to the
> > > process and continue and wait for the error to rear its head.  Now,
> > > since the app serves ~25 people, I don't really have time to look around
> > > with the debugger...  (I'm new to gdb, noo suprise I'm sure)
> > > [/history]
> > > 
> > > So my question is, is it possible to have the gdb that is attached to a
> > > running process that has hit a signal sigsegv, to dump a core?  This
> > > way I can leisurly mull over the data.
> > 
> > You're in luck.  Grab a recent version of GDB and use the
> > "generate-core-file" command.
> > 
> > > More questions I have (totally n00bishness I'm sure), is how can I
> > > find out what line of code in my app, called the 'deadly' function in
> > > libc.so.1 (fprintf) and is there any way to view the call to fprintf
> > > based on the memory address?
> > > 
> > > Below is what I get from a where or a BT etc...  If I try to use a
> > > 'return' I go no where real fast.
> > > 
> > > TIA for your assistance :-)
> > > Ben
> > > 
> > > 
> > > 
> > > Program received signal SIGSEGV, Segmentation fault.
> > > 0xff30ceb8 in _getorientation () from /usr/lib/libc.so.1
> > > (gdb) where
> > > #0  0xff30ceb8 in _getorientation () from /usr/lib/libc.so.1
> > > #1  0xff30d0dc in _set_orientation_byte () from /usr/lib/libc.so.1
> > > #2  0xff300710 in fprintf () from /usr/lib/libc.so.1
> > > #3  0x3aa80 in ?? ()
> > > #4  0x30420 in ?? ()
> > > #5  0x2f6f0 in ?? ()
> > > #6  0x2c7c8 in ?? ()
> > > #7  0x2c258 in ?? ()
> > > (gdb) 
> > 
> > That really looks like you're missing debug info; I don't know if those
> > are legitimate code addresses on your platform or not.
> > 
> > -- 
> > Daniel Jacobowitz
> > MontaVista Software                         Debian GNU/Linux Developer
> I'm ready to ask for a new version of gdb (5.3 is the latest i see), but
> I don't see any documentation on generate-core-file anywhere...  Before
> I ask for a new version, can someone verify that generate-core-file is
> the correct syntax/command?  All I see is how to switch core files
> etc...

For whatever reason, it never made it into the manual.  It's there;
check the command-prompt help.

(gdb) help generate-core-file 
Save a core file with the current state of the debugged process.
Argument is optional filename.  Default filename is 'core.<process_id>'.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: core while attached

[Bennett F. Dill]

On Mon, Jan 27, 2003 at 04:02:54PM -0500, Daniel Jacobowitz wrote:
> On Mon, Jan 27, 2003 at 03:58:15PM -0500, Bennett F. Dill wrote:
> > I'm ready to ask for a new version of gdb (5.3 is the latest i see), but
> > I don't see any documentation on generate-core-file anywhere...  Before
> > I ask for a new version, can someone verify that generate-core-file is
> > the correct syntax/command?  All I see is how to switch core files
> > etc...
> 
> For whatever reason, it never made it into the manual.  It's there;
> check the command-prompt help.
> 
> (gdb) help generate-core-file 
> Save a core file with the current state of the debugged process.
> Argument is optional filename.  Default filename is 'core.<process_id>'.
> 
> -- 
> Daniel Jacobowitz
> MontaVista Software                         Debian GNU/Linux Developer
That sounds good...  We currently have 4.3 and its help didn't like
generate-core-file.  I asked them to install 5.3 since you said the
command is there.

Thanks!
Ben

Re: core while attached

[Joel Brobecker]

> Anyway, the application runs in the background and is a server to about
> 25 clients concurrently.  It takes a few hours for the app to sigsegv.
> Now, it never dumped a core for some reason, so i just attach gdb to the
> process and continue and wait for the error to rear its head.

Maybe your server is larger than a certain size and that past this size
the system elects not to dump a core file. I don't know on which system
you are running, but you can usually raise the limit by using utilities
like "ulimit", for instance "ulimit -c 100000" or somesuch.

-- 
Joel

Re: core while attached

[Bennett F. Dill]

On Tue, Jan 28, 2003 at 07:51:47AM +0400, Joel Brobecker wrote:
> > Anyway, the application runs in the background and is a server to about
> > 25 clients concurrently.  It takes a few hours for the app to sigsegv.
> > Now, it never dumped a core for some reason, so i just attach gdb to the
> > process and continue and wait for the error to rear its head.
> 
> Maybe your server is larger than a certain size and that past this size
> the system elects not to dump a core file. I don't know on which system
> you are running, but you can usually raise the limit by using utilities
> like "ulimit", for instance "ulimit -c 100000" or somesuch.
> 
> -- 
> Joel
That sounds good.  In fact that sounds great...  I believe you can use
ulimit -c unlimited...  My only question (if anyone knows off hand), is
how to apply the ulimit setting to an application that is started via a
sticky bit...  Who's ulimit is actually used?  The user or the sticky
bit?  And if its the sticky bit, how do i check / modify it?

Thanks,
Ben