Re: collecting data from a coring process

[Dmitry Samersoff <dms@samersoff.net>]

[-- Attachment #1.1: Type: text/plain, Size: 3736 bytes --]

Paul,

>> 1) Why not dump the information that you are looking for into a
>> file in the process signal handler ?
>
> Would love to, but I have no idea what state the process is in once
> the SEGV has been triggered.

If you use altstack and avoid malloc you can dump bunch of information
from the signal handler more or less safely.

e.g.

http://hg.openjdk.java.net/jdk9/hs/hotspot/file/tip/src/share/vm/utilities/vmError.cpp

>>> My first thought was to add a script in
>>> /proc/sys/kernel/core_pattern to catch the process as it is
>>> coring. Then I get gdb to attach to the PID of the process that
>>> is about to core. Unfortunately, when I tried that, gdb gives me
>>> this error

One of possible solution is:

1. Change /proc/sys/kernel/core_pattern to have all coredumps from your
app in a separate directory, something like /var/dumps/%e/core.%p

2. Have a cron job that looks over this directory and run
gdb <exe image name> <core_name> < gdb_script > core.%p.out
on demand.

-Dmitry

On 2016-08-26 15:33, Paul Marquess wrote:
> From: vijay nag [mailto:vijunag@gmail.com]
>> Sent: 26 August 2016 13:02 To: Paul Marquess
>> <Paul.Marquess@owmobility.com> Cc: gdb@sourceware.org Subject: Re:
>> collecting data from a coring process
>> 
>> On Fri, Aug 26, 2016 at 2:36 PM, Paul Marquess
>> <Paul.Marquess@owmobility.com> wrote:
>>> I have an existing Linux application that uses gdb to collect
>>> data from a process if it cores. Currently I've been doing that
>>> with gdb after the core is written to disk. No problem there.
>>> 
>>> The requirements have now changed and it won't be possible to
>>> allow the core file to be written to disk. That means I need a
>>> way to (somehow) get gdb to collect the data while the process is
>>> still in memory.
>>> 
>>> My first thought was to add a script in
>>> /proc/sys/kernel/core_pattern to catch the process as it is
>>> coring. Then I get gdb to attach to the PID of the process that
>>> is about to core. Unfortunately, when I tried that, gdb gives me
>>> this error
>>> 
>>> Unable to attach: program terminated with signal SIGSEGV,
>>> Segmentation fault. No stack.
>>> 
>>> That seems to imply that by the time
>>> /proc/sys/kernel/core_pattern kicks in it is too late to use the
>>> PID with gdb.
>>> 
>>> Anyone know of a way to do this? Preferably one that doesn't
>>> involve changing the process itself.
>>> 
>>> cheers Paul
>> You can do one of the following
>> 
>> 1) Why not dump the information that you are looking for into a
>> file in the process signal handler ?
> 
> Would love to, but I have no idea what state the process is in once
> the SEGV has been triggered. Just taking pointes as an example, once
> in the signal handler I'm in a situation where I can't trust that any
> pointer contains a valid value. A quick search suggests I can guard
> against that by using a signal handler. But I'm already in one! If
> there is some prior art that shows how to safely dump data from a
> process once a SEGV has been triggered it would make my day.
> 
> For now, using gdb means I'm isolated from the risk of the data
> collection crashing the process again inside the signal handler.
> 
>> 2) RTFM core file piping on linux (Probably you've done this
>> already
> 
> Looked, but did not find anything.
> 
>> ?) - The idea may seem dangerous, but you can try inserting sleep
>> in the script for sometime till you gather whatever information
>> that is required.
> 
> Not sure what you mean by this. Can you elaborate please?
> 
> Paul
> 


-- 
Dmitry Samersoff
Saint Petersburg, Russia, http://devnull.samersoff.net
* There will come soft rains  ...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]