Re: how could gdb handle truncated core files?

["Paul Pluzhnikov" <ppluzhnikov@google.com>]

On Wed, Aug 27, 2008 at 8:21 AM, Jean-Marc Saffroy <saffroy@gmail.com> wrote:

> For now, gdb does not seem to be able to do anything useful with a truncated
> core file on Linux (ie. what you get when your process dies and the core
> size limit is not 0 but less than the size of the process).
>
> In a number of cases, I think it would be nice to be able to at least get a
> stack trace, and examine local variables. This could require a limited
> amount of data to be dumped by the kernel.
...
> In search of a solution, I patched my Linux kernel so that dumping a core
> would start with the segments that hold a stack (assuming user stack
> pointers are valid): thus these segments have a chance of being dumped
> before the core limit is reached.

You may also want to look at Google user-space coredumper:
  http://code.google.com/p/google-coredumper/

It is often easier to play with than to boot custom kernels,
and it already has support for prioritisation of what is dumped,
as well as compression of the core (core files are often *extremely*
compressible).

> This approach gives interesting results with a (very simple) single threaded
> process. However, my attempts with a multithreaded process failed, like
> this:
>
> $ gdb <binary> <core>
> GNU gdb 6.8
> <snip>
> This GDB was configured as "x86_64-unknown-linux-gnu"...
> Cannot access memory at address 0x2aaaaabc29c8
> (gdb) bt
> #0  0x00002aaaaabc9345 in ?? ()
> #1  0x00000000400179f0 in ?? ()
> #2  0x0000000000000000 in ?? ()
>
> That is:
>  - gdb does not load symbols from binaries

The problem here most likely is that _r_debug.r_map was not found
in the (truncated) core. Without it, GDB can't know which libraries
were loaded, hence can't load unwind info for libpthread, hence
can't produce correct stack trace.

> So, I have the following questions to the community:
>  - what can I do (eg. in my kernel patch) to have gdb load symbols from
> binaries?

You might get better mileage if you dump at least the beginning of
the initial data segment.

>  - do you have any comment on my approach? (eg. I *think* I've seen AIX
> produce small dumps, but I have no idea how they do it, if it's a special
> file format, etc.)

I don't believe AIX has "small" dumps.

AFAIK, they have "regular" dumps (similar to Linux) and "full" dumps,
where full dump includes all the shared libraries, and thus allows
one to examine the core on a developer machine (which may not have
the same version of shared libs as the one used at runtime).

Cheers,
-- 
Paul Pluzhnikov