_r_debug_state getting screwed on x86_64 ?

[Mathieu Lacage <mathieu.lacage@gmail.com>]

hi,

I have an adhoc _r_debug_state function in my elf loader:

static void _r_debug_state (void)
{
  // GDB
  // the debugger will put a breakpoint here.
}

on x86_64, this code compiles to:
0000000000007564 <_r_debug_state>:
    7564:       55                      push   %rbp
    7565:       48 89 e5                mov    %rsp,%rbp
    7568:       c9                      leaveq
    7569:       c3                      retq

I invoke this function to notify gdb that I do have a valid linkmap
ready to be parsed and I get the following events:

34	  g_vdl.breakpoint ();
(gdb) disas $rip $rip+20
Dump of assembler code from 0x7ffff7df75dc to 0x7ffff7df75f0:
0x00007ffff7df75dc <gdb_notify+18>:	mov    0x206b5d(%rip),%rax
# 0x7ffff7ffe140
0x00007ffff7df75e3 <gdb_notify+25>:	mov    0x10(%rax),%rax
0x00007ffff7df75e7 <gdb_notify+29>:	callq  *%rax
0x00007ffff7df75e9 <gdb_notify+31>:	leaveq
0x00007ffff7df75ea <gdb_notify+32>:	retq
0x00007ffff7df75eb:	nop
0x00007ffff7df75ec <vdl_dlopen_private+0>:	push   %rbp
0x00007ffff7df75ed <vdl_dlopen_private+1>:	mov    %rsp,%rbp
End of assembler dump.
(gdb) ni
0x00007ffff7df75e3	34	  g_vdl.breakpoint ();
(gdb)
0x00007ffff7df75e7	34	  g_vdl.breakpoint ();
(gdb) x/1gx $rsp
0x7fffffffe018:	0x00007fffffffe1a8

----------- Ok, before calling, the stack contains this value for a
local variable

(gdb) disas _r_debug_state
Dump of assembler code for function _r_debug_state:
0x00007ffff7df7564 <_r_debug_state+0>:	push   %rbp
0x00007ffff7df7565 <_r_debug_state+1>:	mov    %rsp,%rbp
0x00007ffff7df7568 <_r_debug_state+4>:	leaveq
0x00007ffff7df7569 <_r_debug_state+5>:	retq
End of assembler dump.
(gdb) si
warning: Temporarily disabling breakpoints for unloaded shared library "../ldso"
Stopped due to shared library event

-------------- Ok, now, I did hit the breakpoint at the start of
_r_debug_state and gdb has indeed read the linkmap ---------------

(gdb)
0x00007ffff7df7565 in _r_debug_state () at gdb.c:9
9	{
(gdb) disas _r_debug_state
Dump of assembler code for function _r_debug_state:
0x00007ffff7df7564 <_r_debug_state+0>:	int3
0x00007ffff7df7565 <_r_debug_state+1>:	mov    %rsp,%rbp
0x00007ffff7df7568 <_r_debug_state+4>:	leaveq
0x00007ffff7df7569 <_r_debug_state+5>:	retq
End of assembler dump.
(gdb)

------------ and, yes, we can see the gdb int3 instruction in the
function. -------------

(gdb) p $rsp
$3 = (void *) 0x7fffffffe010
(gdb) x/1gx $rsp
0x7fffffffe010:	0x00007ffff7df75e9

------------- yep, the stack has grown downward by 8 bytes: we can see
the return address pushed there

(gdb) ni
0x00007ffff7df7565 in _r_debug_state () at gdb.c:9
9	{

--------------- Ok, I have now executed theoretically the initial push %rbp

(gdb) x/1gx $rsp
0x7fffffffe010:	0x00007ffff7df75e9

-------------- but my stack pointer did not change ?

(gdb) ni
12	}
gdb) p $rbp
$5 = (void *) 0x7fffffffe010

------------ %rbp before executing the leaveq

(gdb) ni
0x00007ffff7df7569 in _r_debug_state () at gdb.c:12
12	}
(gdb) p $rbp
$6 = (void *) 0x7ffff7df75e9

------------ %rbp has been changed, but, well, it's been set to the
return _address_ pushed by the initial call, not to the value which
should have been saved by push %rbp

(gdb) si
0x00007fffffffe1a8 in ?? ()
(gdb)

------------ and now, the code jumps into what it thinks is the return
address but it's garbage :/

The problem appears to be that the initial push %rbp which is the
first instruction of the function is not being really executed so,
when we leave the body of the function, leaveq pops the return value
instead of poping the frame pointer. I never observed this problem on
ia32 systems so, I wonder what could be wrong here: could it be really
that gdb did clobber the initial push %rbp in _r_debug_state with an
int3 and never executed the original push ?

Mathieu
-- 
Mathieu Lacage <mathieu.lacage@gmail.com>