core files are host dependent

core files are host dependent

[Sergey Kamshilin]

Hi all,
 
I'm newbie, so the question may be is too simple...
 
We always have the issues in debugging cores where gdb traces are only
valid on the original host.

For example gdb traces on one host (where the dump occur) is

#0 0x003267a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.

And on the host where trace goes:

#0 0x003267a2 in ?? ()

Both hosts have identical set of kernel, libraries and other packages
installed. Why this could be and how to fix it? Thank you in advance!

 
/SergeyK

 

Re: core files are host dependent

[Daniel Jacobowitz]

On Wed, Jul 08, 2009 at 10:44:25AM -0700, Sergey Kamshilin wrote:
> Both hosts have identical set of kernel, libraries and other packages
> installed. Why this could be and how to fix it? Thank you in advance!

Your statement is very unlikely to be accurate :-)

Try running strace -e open on GDB to be absolutely sure what libraries
it's opening.  My first guess would be debug library packages
installed on only one of the hosts.

-- 
Daniel Jacobowitz
CodeSourcery

RE: core files are host dependent

[Sergey Kamshilin]

Daniel,

Thank you for the note - it seems that traces are the same... After
closer look I found that /lib/ld-2.3.4.so are different on both
machines:

Both of them installed from the same package:

[root@aragorn1 ~]# rpm -qf /lib/ld-2.3.4.so
glibc-2.3.4-2.25
[root@legolas1 ~]# rpm -qf /lib/ld-2.3.4.so
glibc-2.3.4-2.25

I have the same kernel version installed:

[root@legolas1 ~]# uname -a
Linux legolas1.lab.convedia.com 2.6.9-11.EL #1 Fri May 20 18:17:57 EDT
2005 i686 i686 i386 GNU/Linux

[root@aragorn1 ~]# uname -a
Linux aragorn1.lab.convedia.com 2.6.9-11.EL #1 Fri May 20 18:17:57 EDT
2005 i686 i686 i386 GNU/Linux

But! 
[root@aragorn1 ~]# rsh legolas1 cat /lib/ld-2.3.4.so | diff -
/lib/ld-2.3.4.so
Binary files - and /lib/ld-2.3.4.so differ

Even if I copy remote ld-2.3.4.so locally and compare, the difference is
scaring - almost in every line...:

[root@aragorn1 ~]# diff <(hexdump -C ld-2.3.4.so) <(hexdump -C
/lib/ld-2.3.4.so) 
2c2
< 00000010  03 00 03 00 01 00 00 00  c0 67 32 00 34 00 00 00
|.........g2.4...|
---
> 00000010  03 00 03 00 01 00 00 00  c0 27 7d 00 34 00 00 00
|.........'}.4...|
4,11c4,11
< 00000030  18 00 15 00 01 00 00 00  00 00 00 00 00 60 32 00
|.............`2.|
< 00000040  00 60 32 00 b4 4d 01 00  b4 4d 01 00 05 00 00 00
|.`2..M...M......|
< 00000050  00 10 00 00 01 00 00 00  a0 5c 01 00 a0 bc 33 00
|.........\....3.|
< 00000060  a0 bc 33 00 9c 07 00 00  54 08 00 00 06 00 00 00
|..3.....T.......|
< 00000070  00 10 00 00 02 00 00 00  14 5f 01 00 14 bf 33 00
|........._....3.|
< 00000080  14 bf 33 00 c0 00 00 00  c0 00 00 00 06 00 00 00
|..3.............|
< 00000090  04 00 00 00 50 e5 74 64  fc 4b 01 00 fc ab 33 00
|....P.td.K....3.|
< 000000a0  fc ab 33 00 5c 00 00 00  5c 00 00 00 04 00 00 00
|..3.\...\.......|
---
> 00000030  18 00 15 00 01 00 00 00  00 00 00 00 00 20 7d 00
|............. }.|
> 00000040  00 20 7d 00 b4 4d 01 00  b4 4d 01 00 05 00 00 00  |.
}..M...M......|
> 00000050  00 10 00 00 01 00 00 00  a0 5c 01 00 a0 7c 7e 00
|.........\...|~.|
> 00000060  a0 7c 7e 00 9c 07 00 00  54 08 00 00 06 00 00 00
|.|~.....T.......|
> 00000070  00 10 00 00 02 00 00 00  14 5f 01 00 14 7f 7e 00
|........._....~.|
> 00000080  14 7f 7e 00 c0 00 00 00  c0 00 00 00 06 00 00 00
|..~.............|
> 00000090  04 00 00 00 50 e5 74 64  fc 4b 01 00 fc 6b 7e 00
|....P.td.K...k~.|
> 000000a0  fc 6b 7e 00 5c 00 00 00  5c 00 00 00 04 00 00 00
|.k~.\...\.......|
............. Etc..

Both are IBM servers, Intel CPUs. One is 2.6GHz another 2.8GHz.
Everything else is the same...

How this could be??? Sorry if it became non gdb community question....

/SergeyK




-----Original Message-----
From: Daniel Jacobowitz [mailto:drow@false.org] 
Sent: Wednesday, July 08, 2009 11:06 AM
To: Sergey Kamshilin
Cc: gdb@sourceware.org
Subject: Re: core files are host dependent

On Wed, Jul 08, 2009 at 10:44:25AM -0700, Sergey Kamshilin wrote:
> Both hosts have identical set of kernel, libraries and other packages 
> installed. Why this could be and how to fix it? Thank you in advance!

Your statement is very unlikely to be accurate :-)

Try running strace -e open on GDB to be absolutely sure what libraries
it's opening.  My first guess would be debug library packages installed
on only one of the hosts.

--
Daniel Jacobowitz
CodeSourcery

Re: core files are host dependent

[Daniel Jacobowitz]

On Wed, Jul 08, 2009 at 01:37:17PM -0700, Sergey Kamshilin wrote:
> Daniel,
> 
> Thank you for the note - it seems that traces are the same... After
> closer look I found that /lib/ld-2.3.4.so are different on both
> machines:

Ah-hah.  This usually means they have been prelinked to random
addresses.

I thought GDB knew how to correct for different prelink addresses,
but I'm not sure about that.  It helps to use a current GDB.

-- 
Daniel Jacobowitz
CodeSourcery