Problems with hook-stop

Problems with hook-stop

[Carl Shapiro]

Hi,

I am trying to write a user-defined command hook that automatically
passes a SIGTRAP to my program.  Here is the definition I am working
with:

define hook-stop
  if (((* (char *) ($pc - 1)) & 0xff) == 0xcc)
    signal SIGTRAP
  end
end

My target architecture is x86.  This command fires the first time I
hit an "int3" instruction.  The next time I hit an int3, hook-stop is
not executed and I must manually pass a SIGTRAP by evaluating "signal
SIGTRAP".  It seems that every other time gdb is stopped with a
SIGTRAP the stop hook is not executed.  Here's a program that can be
used to demonstrate this:

#include <signal.h>
#include <stdio.h>

void handler(int signum) {
  fprintf(stderr, "handler(signum=%d)\n", signum);
}

int main(int argc, char *argv[]) {
  struct sigaction sa;
  int i;
  sa.sa_handler = handler;
  for (i = 0;; ++i) {
    fprintf(stderr, "i=%d\n", i);
    asm("int3");
  }
  return 0;
}

If I remove the "signal SIGTRAP" statement from hook-stop, it fires
every time around the loop.  Does anyone know where am I going wrong
wrong?

Thanks,

Carl

Re: Problems with hook-stop

[Mark Kettenis]

> Date: Wed, 26 Sep 2007 01:33:11 -0700
> From: "Carl Shapiro" <carl.shapiro@gmail.com>
> 
> Hi,
> 
> I am trying to write a user-defined command hook that automatically
> passes a SIGTRAP to my program.  Here is the definition I am working
> with:
> 
> define hook-stop
>   if (((* (char *) ($pc - 1)) & 0xff) == 0xcc)
>     signal SIGTRAP
>   end
> end
> 
> My target architecture is x86.  This command fires the first time I
> hit an "int3" instruction.  The next time I hit an int3, hook-stop is
> not executed and I must manually pass a SIGTRAP by evaluating "signal
> SIGTRAP".  It seems that every other time gdb is stopped with a
> SIGTRAP the stop hook is not executed.  Here's a program that can be
> used to demonstrate this:
> 
> #include <signal.h>
> #include <stdio.h>
> 
> void handler(int signum) {
>   fprintf(stderr, "handler(signum=%d)\n", signum);
> }
> 
> int main(int argc, char *argv[]) {
>   struct sigaction sa;
>   int i;
>   sa.sa_handler = handler;
>   for (i = 0;; ++i) {
>     fprintf(stderr, "i=%d\n", i);
>     asm("int3");
>   }
>   return 0;
> }
> 
> If I remove the "signal SIGTRAP" statement from hook-stop, it fires
> every time around the loop.  Does anyone know where am I going wrong
> wrong?

Since SIGTRAP has a very special meaning for debugging, it's probably
unwise to play games like this.

Re: Problems with hook-stop

[Carl Shapiro]

On 9/27/07, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> Since SIGTRAP has a very special meaning for debugging, it's probably
> unwise to play games like this.

The problem I am having is not specific to SIGTRAP.  You can replace
the interrupt instruction with a kill(2), providing any signal you'd
like as an argument.  The stop hook will fails execute every other
time.

Here's the hook-stop definition to use:

define hook-stop
  signal SIGUSR1
end

Here's the program to run under GDB:

#include <signal.h>
#include <stdio.h>
#include <unistd.h>

void handler(int signum) {
 fprintf(stderr, "handler(signum=%d)\n", signum);
}

int main(int argc, char *argv[]) {
 struct sigaction sa;
 int i;
 sa.sa_handler = handler;
 sigaction(SIGUSR1, &sa, NULL);
 for (i = 0;; ++i) {
   fprintf(stderr, "i=%d\n", i);
   kill(getpid(), SIGUSR1);
 }
 return 0;
}

Re: Problems with hook-stop

[Daniel Jacobowitz]

On Thu, Sep 27, 2007 at 05:25:33PM -0700, Carl Shapiro wrote:
> On 9/27/07, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> > Since SIGTRAP has a very special meaning for debugging, it's probably
> > unwise to play games like this.
> 
> The problem I am having is not specific to SIGTRAP.  You can replace
> the interrupt instruction with a kill(2), providing any signal you'd
> like as an argument.  The stop hook will fails execute every other
> time.

Thank you for the test case.  This was very useful.

The problem is that we are still running the stop hook for the
first stop when the program stops the second time.  Basically,
the stop "command" has been run again from inside the first
hook-stop.  There's a safety check for recursive hooks to prevent
GDB blowing out its stack.  If that weren't there you'd have an
even more unpleasant bug to report.

To fix this we would need to make a decision about what happens when
a hook (or a breakpoint commands list, it's basically the same thing)
runs the target.  We don't want to necessarily abort the hook.
Consider:

define hook-stop
  call dump_state ()
  echo done
end

Assuming there's no breakpoint or crash in dump_state, then we should
run the function to completion and continue executing the stop hook.

Maybe we should handle hooks and commands lists specially when they
end in continue or signal.  Basically a tail recursion optimization.

I'm not going to try to do that right now, but I hope this explanation
is helpful :-)

-- 
Daniel Jacobowitz
CodeSourcery

Re: Problems with hook-stop

[Carl Shapiro]

On 9/29/07, Daniel Jacobowitz <drow@false.org> wrote:
> Maybe we should handle hooks and commands lists specially when they
> end in continue or signal.  Basically a tail recursion optimization.

If there is no infrastructure in-place to perform this analysis, it
would seem easier to add a new verb to the GDB command language that
signals and exits the currently executing command.

Re: Problems with hook-stop

[Daniel Jacobowitz]

On Tue, Oct 02, 2007 at 12:57:33AM -0700, Carl Shapiro wrote:
> On 9/29/07, Daniel Jacobowitz <drow@false.org> wrote:
> > Maybe we should handle hooks and commands lists specially when they
> > end in continue or signal.  Basically a tail recursion optimization.
> 
> If there is no infrastructure in-place to perform this analysis, it
> would seem easier to add a new verb to the GDB command language that
> signals and exits the currently executing command.

Oh, the analysis is trivial.  It's doing something with the result of
the analysis that will be harder :-)

-- 
Daniel Jacobowitz
CodeSourcery

Re: Problems with hook-stop

[Christian Thalinger]

On Wed, 2007-09-26 at 01:33 -0700, Carl Shapiro wrote:
> Hi,
> 
> I am trying to write a user-defined command hook that automatically
> passes a SIGTRAP to my program.  Here is the definition I am working
> with:
> 
> define hook-stop
>   if (((* (char *) ($pc - 1)) & 0xff) == 0xcc)
>     signal SIGTRAP
>   end
> end

Hi!

I'm also searching for a solution for that problem.  The reason why we
want to use the INT3 instruction on i386 and x86_64 is because we must
be able to replace a machine instruction atomically in our JIT compiler
(where "atomically" reads as "the patching instruction must not be
shorter than the patched one", for obvious reasons).  And INT3 is the
only instruction that is suitable.

Right now we're using UD2, but that's only a temporary solution.

- twisti