how GDB use ptrace to return from a function

how GDB use ptrace to return from a function

[Yubin Ruan]

Hi GDB developer ;-)

I am writing a toy debugger and currently looking into the GDB source
because I want to know: after setting the tracee's registers and
trying to let it execute a function with ptrace(PTRACE_CONT, ...), how
can the tracee return to the tracer?

Currently I manipulate the tracee's stack and place a NULL return
address there (I am on X86), so that after ptrace(PTRACE_CONT, ...),
the tracee will execute a function and return, at which point a SIGSEV
is generated (because the return address is NULL), so tracee will be
caught by the tracer again.

I don't know whether GDB is using this kind of technique. If anyone
know that, can you enlighten me, and probably point me to the source?

Yubin

Re: how GDB use ptrace to return from a function

[Joel Brobecker]

> I don't know whether GDB is using this kind of technique. If anyone
> know that, can you enlighten me, and probably point me to the source?

GDB sets the call up so that the return address is at specific
location (usually the program's entry point, but that's arch-
dependent), and then places a breakpoint at that address. It
then knows, when receiving the corresponding breakpoint event,
that a breakpoint at that address corresponds to the end of
the function that we called.

-- 
Joel

Re: how GDB use ptrace to return from a function

[Yubin Ruan]

Thanks Joel,

2017-11-18 0:10 GMT+08:00 Joel Brobecker <brobecker@adacore.com>:
>> I don't know whether GDB is using this kind of technique. If anyone
>> know that, can you enlighten me, and probably point me to the source?
>
> GDB sets the call up so that the return address is at specific
> location (usually the program's entry point, but that's arch-
> dependent), and then places a breakpoint at that address. It
> then knows, when receiving the corresponding breakpoint event,
> that a breakpoint at that address corresponds to the end of
> the function that we called.

what break point events are common for X86?

Yubin

Re: how GDB use ptrace to return from a function

[Joel Brobecker]

> what break point events are common for X86?

IIRC, the breakpoint instruction on x86 is the int3 instruction.
It should generate a SIGTRAP upon execution, just like any user-
inserted breakpoints.

-- 
Joel

Re: how GDB use ptrace to return from a function

[Yubin Ruan]

2017-11-18 12:03 GMT+08:00 Joel Brobecker <brobecker@adacore.com>:
>> what break point events are common for X86?
>
> IIRC, the breakpoint instruction on x86 is the int3 instruction.
> It should generate a SIGTRAP upon execution, just like any user-
> inserted breakpoints.

Thanks Joel ;-)

Yubin

Re: how GDB use ptrace to return from a function

[Jan Kratochvil]

On Sat, 18 Nov 2017 05:03:11 +0100, Joel Brobecker wrote:
> > what break point events are common for X86?
> 
> IIRC, the breakpoint instruction on x86 is the int3 instruction.
> It should generate a SIGTRAP upon execution, just like any user-
> inserted breakpoints.

A toy debugger should have available also a hardware breakpoint (hbreak) using
the hardware registers commonly used only for hardware watchpoints.

Then the return address can be arbitrary as the debugger does not have to
write anything into that address.


Jan