gdb: detection and/or fork+gethostbyname crash workaround?

gdb: detection and/or fork+gethostbyname crash workaround?

[Glenn F. Maynard]

Is there any way to tell if a (C) program is being debugged by gdb?

I've had the "dynamic linking in a forked process being debugged"
bite me twice now.  The first time around, I moved the process
to another binary (a resolver binary; exec()ing it fixed this).  I'm
trying to work around this bug in another program, now, where making
a separate binary for resolving is undesirable; I want to force it
to not fork resolves when being debugged, and do so automatically.

Portability isn't much of a concern: the workaround can be autoconf'd
to only happen on the architectures it works (and/or is necessary) for.

Alternatively, a workaround for this particular bug would be equally
sufficient.  Of course, having it fixed would be nice ... it's been around
for at least a year and prevents all "fork for DNS" code from working under
gdb unless it happens to exec to do so, and it's an odd enough bug
that it's probably cost a great deal of time for those hit by it. (It's
been reported multiple times in both the "forked gethostbyname crashes"
and "forked dynamic linking crashes" guises.)

-- 
Glenn Maynard

Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Kevin Buettner]

On Jul 23,  9:01pm, Glenn F. Maynard wrote:

> Is there any way to tell if a (C) program is being debugged by gdb?

You didn't say what platform you want this for.

If it's one which uses ptrace(), the kernel usually prohibits two
processes from invoking ptrace() on the same inferior.  So one
strategy might be to have the program in question cause ptrace to be
invoked on itself.  I don't think the process will be able to do this
itself; I think it's likely that it would have to fork and let the
child attempt this.  The return status from wait() or waitpid() could
indicate whether the attempt to invoke ptrace() was successful or not.

If it's a system which uses /proc as the debug interface, I think
/proc/PID/ctl may only be opened with write access by only one process.
So you could test to see if this file can be opened with write access.
If it can, it means that it's not being debugged; if it can't, it's
being debugged.  (If you're successful in opening it, make sure you
close it...)

More generally, the debug interface provided by the OS (of which
ptrace() and /proc are but two examples) usually has a mechanism
which prevents a process from being debugged twice.  You just have
to figure out what that mechanism is...

Of course, you should keep in mind that a race condition is possible.
Just because you've determined at one point in your program that you
aren't being debugged doesn't mean that this condition will hold
later on.

> I've had the "dynamic linking in a forked process being debugged"
> bite me twice now.  The first time around, I moved the process
> to another binary (a resolver binary; exec()ing it fixed this).  I'm
> trying to work around this bug in another program, now, where making
> a separate binary for resolving is undesirable; I want to force it
> to not fork resolves when being debugged, and do so automatically.

Could you describe this problem in a bit more detail?  It sounds to
me like something ought to be fixed on the GDB side of things, but
I don't really understand what the problem is...

> Portability isn't much of a concern: the workaround can be autoconf'd
> to only happen on the architectures it works (and/or is necessary) for.
> 
> Alternatively, a workaround for this particular bug would be equally
> sufficient.  Of course, having it fixed would be nice ... it's been around
> for at least a year and prevents all "fork for DNS" code from working under
> gdb unless it happens to exec to do so, and it's an odd enough bug
> that it's probably cost a great deal of time for those hit by it. (It's
> been reported multiple times in both the "forked gethostbyname crashes"
> and "forked dynamic linking crashes" guises.)

I searched gdb@sources.redhat.com w/ both of these search phrases and
only came up with the message to which I'm responding.  Can you provide
some URLs for the archived messages?

Kevin

Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Glenn F. Maynard]

On Tue, Jul 24, 2001 at 08:52:19AM -0700, Kevin Buettner wrote:
> If it's one which uses ptrace(), the kernel usually prohibits two
> processes from invoking ptrace() on the same inferior.  So one
> strategy might be to have the program in question cause ptrace to be
> invoked on itself.  I don't think the process will be able to do this
> itself; I think it's likely that it would have to fork and let the
> child attempt this.  The return status from wait() or waitpid() could
> indicate whether the attempt to invoke ptrace() was successful or not.

That's what I tried (x86 Linux); a process can't ptrace itself.  I could
fork a process to test this, I suppose; I'll try that.  (Wouldn't it
be the return status from ptrace(), though?  I assume it'd return something
like EPERM if a process is already being debugged.)

> Of course, you should keep in mind that a race condition is possible.
> Just because you've determined at one point in your program that you
> aren't being debugged doesn't mean that this condition will hold
> later on.

That's OK.  The main problem is that every time I debug a process
(in this case, lftp), I have to manually disable DNS forking; I want
to prevent that.

> I searched gdb@sources.redhat.com w/ both of these search phrases and
> only came up with the message to which I'm responding.  Can you provide
> some URLs for the archived messages?

http://sources.redhat.com/ml/bug-glibc/2000-04/msg00018.html
http://sources.redhat.com/ml/bug-gdb/2001-03/msg00049.html

(the first is more useful; it was apparently CC'd to both
bug-glibc and bug-gdb, but that's what google turned up.)

Thanks.

-- 
Glenn Maynard

Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Kevin Buettner]

On Jul 24,  2:22pm, Glenn F. Maynard wrote:

> On Tue, Jul 24, 2001 at 08:52:19AM -0700, Kevin Buettner wrote:
> > If it's one which uses ptrace(), the kernel usually prohibits two
> > processes from invoking ptrace() on the same inferior.  So one
> > strategy might be to have the program in question cause ptrace to be
> > invoked on itself.  I don't think the process will be able to do this
> > itself; I think it's likely that it would have to fork and let the
> > child attempt this.  The return status from wait() or waitpid() could
> > indicate whether the attempt to invoke ptrace() was successful or not.
> 
> That's what I tried (x86 Linux); a process can't ptrace itself.  I could
> fork a process to test this, I suppose; I'll try that.  (Wouldn't it
> be the return status from ptrace(), though?  I assume it'd return something
> like EPERM if a process is already being debugged.)

Right; you'd pass back the results of the attempted ptrace() (and it
looks to me like EPERM is the right thing to test for) through the
child's exit status.  That's why I said you'd use wait() or waitpid()
to determine it.

> > I searched gdb@sources.redhat.com w/ both of these search phrases and
> > only came up with the message to which I'm responding.  Can you provide
> > some URLs for the archived messages?
> 
> http://sources.redhat.com/ml/bug-glibc/2000-04/msg00018.html

Thanks.  I think I can explain what is happening now...

GDB is placing a breakpoint in the dummy function, _dl_debug_state(),
which is the function that the shared library machinery calls to
communicate the fact that that the set of shared libraries has changed
(i.e, the application has either loaded or unloaded one of them). 
When a child is forked, this breakpoint still exists in the child
process with no one to catch it.  Ideally, gdb would remove all
breakpoints prior to forking the child and then reinstall these
breakpoints in the parent again after the child has started.  Some
operating systems (such as those which use /proc for the debug
interface to the OS) give the debugger facilities to do this, but to
the best of my knowledge, the ptrace() interface provided by Linux
does not provide a mechanism by which this may be accomplished. [1]
(Basically, GDB needs a way to stop the parent's execution in the
return path from fork().  It would also be nice if gdb allowed the
user to choose to follow the child or follow the parent, but that's
another problem...)

Anyway, for your problem, there are at least two other approaches
that you may wish to explore:

    1) Set up a signal handler in the child for SIGTRAP.  Have the
       signal handler replace the trap instruction with a nop prior
       to returning.

    2) In the parent, call dlopen() to explicitly load the shared
       libraries needed by gethostbyname().  That way, when
       gethostbyname() is called, there won't be any unloaded
       functions.

    3) See footnote...

[1] Actually, maybe there is.  It may be possible to place a
breakpoint libc's fork() entry point.  When this breakpoint is hit,
we call ptrace(PTRACE_SYSCALL,...) twice.  From my reading of the
documentation, the first PTRACE_SYSCALL call will stop at the fork
syscall's entry point, the second will stop when the syscall is about
to return.  We could remove all breakpoints on the first stop (fork
entry) and restore them on the second stop (fork exit).

Kevin

Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Daniel Jacobowitz]

On Tue, Jul 24, 2001 at 02:22:48PM -0400, Glenn F. Maynard wrote:
> On Tue, Jul 24, 2001 at 08:52:19AM -0700, Kevin Buettner wrote:
> > If it's one which uses ptrace(), the kernel usually prohibits two
> > processes from invoking ptrace() on the same inferior.  So one
> > strategy might be to have the program in question cause ptrace to be
> > invoked on itself.  I don't think the process will be able to do this
> > itself; I think it's likely that it would have to fork and let the
> > child attempt this.  The return status from wait() or waitpid() could
> > indicate whether the attempt to invoke ptrace() was successful or not.
> 
> That's what I tried (x86 Linux); a process can't ptrace itself.  I could
> fork a process to test this, I suppose; I'll try that.  (Wouldn't it
> be the return status from ptrace(), though?  I assume it'd return something
> like EPERM if a process is already being debugged.)

Try ptrace( PTRACE_TRACEME, ...) ?


-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Kevin Buettner]

On Jul 24,  1:26pm, Daniel Jacobowitz wrote:

> > That's what I tried (x86 Linux); a process can't ptrace itself.  I could
> > fork a process to test this, I suppose; I'll try that.  (Wouldn't it
> > be the return status from ptrace(), though?  I assume it'd return something
> > like EPERM if a process is already being debugged.)
> 
> Try ptrace( PTRACE_TRACEME, ...) ?

Actually, what I had in mind was for a child process to call
ptrace (PTRACE_ATTACH, ...) with the pid of the parent.  (And then, if
that succeeds immediately call ptrace (PTRACE_DETACH,...).

The reason for for not using PTRACE_PTRACEME is that it is unlikely
that the parent will know what to do with various wait messages
that'll be sent to it.  (I.e, you don't want to call PTRACE_TRACEME
unless your parent is actually a debugger.)

However, there are other approaches that Glenn may wish to try.  See

    http://sources.redhat.com/ml/gdb/2001-07/msg00346.html

We should also try to fix GDB so that all breakpoints are removed
when the inferior decides to do a fork().  (The trick will be to
restore them, but the footnote in the above link outlines how it
might be done.)

Kevin

Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Mark Kettenis]

"Glenn F. Maynard" <g_gdb@zewt.org> writes:

> > I searched gdb@sources.redhat.com w/ both of these search phrases and
> > only came up with the message to which I'm responding.  Can you provide
> > some URLs for the archived messages?
> 
> http://sources.redhat.com/ml/bug-glibc/2000-04/msg00018.html
> http://sources.redhat.com/ml/bug-gdb/2001-03/msg00049.html
> 
> (the first is more useful; it was apparently CC'd to both
> bug-glibc and bug-gdb, but that's what google turned up.)

Hmm, I'd expect a Trace/breakpoint trap...

Yup, that's what I get.  What's happening is that when you fork, GDB
has its breakpoints inserted in the inferior.  Therefore the child
inherits those breakpoints from its parents.  When it trips such a
breakpoint, it gets killed since the child isn't traced by GDB.

In this particular case it's a GDB-internal solib_event breakpoint
that gets tripped, since gethostbyname() causes one or more NSS
modules to be loaded.

It's not easy to solve this (and I suspect that a lot of platforms
suffer from the same problems).  Implementing the follow-fork stuff
for Linux would probably do the trick.  Perhaps I can find some time
to implement that, but it won't be ready for the next release.

Mark

Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Glenn F. Maynard]

To the other reply: well, I can already manually disable DNS
forking; I just don't want to have to do that.

On Tue, Jul 24, 2001 at 01:26:07PM -0700, Kevin Buettner wrote:
> > On Tue, Jul 24, 2001 at 08:52:19AM -0700, Kevin Buettner wrote:
> > > If it's one which uses ptrace(), the kernel usually prohibits two
> > > processes from invoking ptrace() on the same inferior.  So one
> > > strategy might be to have the program in question cause ptrace to be
> > > invoked on itself.  I don't think the process will be able to do this
> > > itself; I think it's likely that it would have to fork and let the
> > > child attempt this.  The return status from wait() or waitpid() could
> > > indicate whether the attempt to invoke ptrace() was successful or not.
> > 
> > That's what I tried (x86 Linux); a process can't ptrace itself.  I could
> > fork a process to test this, I suppose; I'll try that.  (Wouldn't it
> > be the return status from ptrace(), though?  I assume it'd return something
> > like EPERM if a process is already being debugged.)
> 
> Right; you'd pass back the results of the attempted ptrace() (and it
> looks to me like EPERM is the right thing to test for) through the
> child's exit status.  That's why I said you'd use wait() or waitpid()
> to determine it.

Got it.  In case anyone else wants it (or can point out an error, of course):

#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <assert.h>

#include <sys/ptrace.h>
#include <signal.h>

#include <sys/types.h>
#include <sys/wait.h>

bool in_debugger()
{
	int cpid = fork();
	assert(cpid != -1);

	if(cpid == 0) {
		signal(SIGCHLD, SIG_IGN);
		if(ptrace(PTRACE_ATTACH, getppid(), NULL, NULL) == -1) {
			/* Attach failed; it's probably already being debugged. */
			if(errno != EPERM)
				fprintf(stderr, "c ptrace: unexpected error: %s\n", strerror(errno));
			exit(1);
		}
		
		/* We just SIGSTOPped it; we need to CONT it now. */
		int ret;
		waitpid(getppid(), &ret, 0);

		if(ptrace(PTRACE_DETACH, getppid(), NULL, NULL) == -1)
			fprintf(stderr, "detach failed: %s\n", strerror(errno));

		kill(getppid(), SIGCONT);

		exit(0);
	}
	int ret;
	waitpid(cpid, &ret, 0);
	assert(!WIFSIGNALED(ret));

	return WEXITSTATUS(ret) != 0;

}

main()
{
	printf("%i\n", in_debugger());
}


>     1) Set up a signal handler in the child for SIGTRAP.  Have the
>        signal handler replace the trap instruction with a nop prior
>        to returning.

I tried that, with no luck.  I might have botched something silly, of course.

>     2) In the parent, call dlopen() to explicitly load the shared
>        libraries needed by gethostbyname().  That way, when
>        gethostbyname() is called, there won't be any unloaded
>        functions.

That'd mean searching out NSS libraries, and loading *all* of them ... and
there's no real guarantee that they're even in /lib.

-- 
Glenn Maynard

Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Daniel Jacobowitz]

On Tue, Jul 24, 2001 at 01:26:07PM -0700, Kevin Buettner wrote:
> [1] Actually, maybe there is.  It may be possible to place a
> breakpoint libc's fork() entry point.  When this breakpoint is hit,
> we call ptrace(PTRACE_SYSCALL,...) twice.  From my reading of the
> documentation, the first PTRACE_SYSCALL call will stop at the fork
> syscall's entry point, the second will stop when the syscall is about
> to return.  We could remove all breakpoints on the first stop (fork
> entry) and restore them on the second stop (fork exit).

Yeah, that's how PTRACE_SYSCALL works - see strace for the gory
details.  It can even be used to stop in both the parent and the child.

I've wanted to be able to even access PTRACE_SYSCALL from gdb; it can
be convenient for getting to a particular point in execution.  It would
require defining a new type of breakpoint, though.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Kevin Buettner]

On Jul 24,  4:57pm, Glenn F. Maynard wrote:

> >     1) Set up a signal handler in the child for SIGTRAP.  Have the
> >        signal handler replace the trap instruction with a nop prior
> >        to returning.
> 
> I tried that, with no luck.  I might have botched something silly, of course.

Well, I didn't really like this suggestion anyway, but often times
when writing self modifying code (I used to work on JIT compilers),
you need to make sure that the instruction cache and the data cache
are synchronized.  (This often means that you need to flush the
D-cache and invalidate the I-cache.)

Kevin

Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Glenn F. Maynard]

Didn't read this well enough the first time around:

> > How about something like this:
> > 
> >   /* Global variable.  Do resolve by forking a child process.
> >      This is usually one, but you can turn it off to make debugging
> >      easier. */
> >   int g_resolve_fork = 1;
> > 
> > Then in your .gdbinit file, say:
> > 
> >   print g_resolve_fork = 0

I prefer a self-contained workaround.  If someone loads a program into
a debugger to fix a problem, they shouldn't have to sift through documentation
to find something like this out--and they probably will end up mailing
the program's maintainer instead, as the result of not doing this looks like
a bug.  `It's similar to a recent locale problem discussed on mutt-dev; one of
OpenBSD's "YES" regexes was "^[Yn]"; someone suggested adding a --without-
autoconf toggle to disable using this string--but I'd certainly never have
thought to look for a manual fix.

Also, that'd be printing "no symbol" errors for other programs (and "no
symtab" errors for empty gdbs).

-- 
Glenn Maynard

Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Kevin Buettner]

On Jul 24,  1:00pm, Michael Elizabeth Chastain wrote:

> > I'm trying to work around this bug in another program, now, where making
> > a separate binary for resolving is undesirable; I want to force it to
> > not fork resolves when being debugged, and do so automatically.
> 
> How about something like this:
> 
>   /* Global variable.  Do resolve by forking a child process.
>      This is usually one, but you can turn it off to make debugging
>      easier. */
>   int g_resolve_fork = 1;
> 
> Then in your .gdbinit file, say:
> 
>   print g_resolve_fork = 0

I like Michael's suggestion (above) better than any of my suggested
workarounds.

Kevin

Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Michael Elizabeth Chastain]

> I'm trying to work around this bug in another program, now, where making
> a separate binary for resolving is undesirable; I want to force it to
> not fork resolves when being debugged, and do so automatically.

How about something like this:

  /* Global variable.  Do resolve by forking a child process.
     This is usually one, but you can turn it off to make debugging
     easier. */
  int g_resolve_fork = 1;

Then in your .gdbinit file, say:

  print g_resolve_fork = 0

Alternatively, you can wire up g_resolve_fork to a command line
argument or to an environment variable.  You can set an environment
variable in gdb with "set env FOO BAR", and then stash that in a
.gdbinit file.

It doesn't solve the core problem but it might be a useful
workaround.

Michael Elizabeth Chastain
<chastain@redhat.com>
"love without fear"