GDB (not) handling SIGINT...?

GDB (not) handling SIGINT...?

[Paul Smith]

Hi all; I'm using GDB 8.1 on a modern 64bit GNU/Linux system (Ubuntu
18.04LTS).

Recently I added a call to sigtimedwait() for SIGINT into my
(multithreaded) program and now I'm having an issue with GDB.

What I want is that if I attach to my program then continue, then use
^C at the GDB terminal, I should get a (gdb) prompt back but I do NOT
want the SIGINT delivered to my program to wake up my sigtimedwait()
call (because it will cause my program to do various things that I
don't want it to do).

I see that SIGINT is set to nopass:

  (gdb) info signals SIGINT
  Signal        Stop      Print   Pass to program Description
  SIGINT        Yes       Yes     No              Interrupt

but yet when I use ^C at the GDB prompt my sigtimedwait() call inside
my program does return with "2" (SIGINT), which I don't want.

I guess I don't understand what the docs mean when they say that the
signal is not passed to the process under debug.  Note that in my case
I'm attaching to the program from a completely different terminal so
there's no issue with process groups etc. and as far as I can
understand it, the signal should only be delivered to GDB not my
process, so unless there's some weird magic at work here it must be GDB
forwarding that signal to my process.

Anyone have any thoughts about this?

Cheers!

Re: GDB (not) handling SIGINT...?

[Simon Marchi]

On 2018-11-13 17:37, Paul Smith wrote:
> Hi all; I'm using GDB 8.1 on a modern 64bit GNU/Linux system (Ubuntu
> 18.04LTS).
> 
> Recently I added a call to sigtimedwait() for SIGINT into my
> (multithreaded) program and now I'm having an issue with GDB.
> 
> What I want is that if I attach to my program then continue, then use
> ^C at the GDB terminal, I should get a (gdb) prompt back but I do NOT
> want the SIGINT delivered to my program to wake up my sigtimedwait()
> call (because it will cause my program to do various things that I
> don't want it to do).
> 
> I see that SIGINT is set to nopass:
> 
>   (gdb) info signals SIGINT
>   Signal        Stop      Print   Pass to program Description
>   SIGINT        Yes       Yes     No              Interrupt
> 
> but yet when I use ^C at the GDB prompt my sigtimedwait() call inside
> my program does return with "2" (SIGINT), which I don't want.
> 
> I guess I don't understand what the docs mean when they say that the
> signal is not passed to the process under debug.  Note that in my case
> I'm attaching to the program from a completely different terminal so
> there's no issue with process groups etc. and as far as I can
> understand it, the signal should only be delivered to GDB not my
> process, so unless there's some weird magic at work here it must be GDB
> forwarding that signal to my process.
> 
> Anyone have any thoughts about this?
> 
> Cheers!

I was able to reproduce it with, it really sounds like a bug.

If others want to try, here's my test program:

#include <stdio.h>
#include <signal.h>
#include <stdlib.h>

int main()
{
	sigset_t set;
	sigemptyset(&set);
	sigaddset(&set, SIGINT);
	sigprocmask(SIG_BLOCK, &set, NULL);
	for (int i = 0; i < 10; i++) {
		int n = sigwaitinfo(&set, NULL);
		printf("signal %d\n", n);
	}
}


Run in a terminal, attach with GDB in another terminal, continue, then 
ctrl-C in GDB's terminal.  The first call to sigwaitinfo returns -1, I 
think it's expected as the syscall gets interrupted.  But the subsequent 
calls return 2, showing that indeed the process has received a SIGINT.

Paul, could you please file a bug?  You can re-use this test program if 
you want.

Simon

Re: GDB (not) handling SIGINT...?

[Pedro Alves]

On 11/14/2018 10:32 PM, Simon Marchi wrote:
> On 2018-11-13 17:37, Paul Smith wrote:
>> Hi all; I'm using GDB 8.1 on a modern 64bit GNU/Linux system (Ubuntu
>> 18.04LTS).
>>
>> Recently I added a call to sigtimedwait() for SIGINT into my
>> (multithreaded) program and now I'm having an issue with GDB.
>>
>> What I want is that if I attach to my program then continue, then use
>> ^C at the GDB terminal, I should get a (gdb) prompt back but I do NOT
>> want the SIGINT delivered to my program to wake up my sigtimedwait()
>> call (because it will cause my program to do various things that I
>> don't want it to do).
>>
>> I see that SIGINT is set to nopass:
>>
>>   (gdb) info signals SIGINT
>>   Signal        Stop      Print   Pass to program Description
>>   SIGINT        Yes       Yes     No              Interrupt
>>
>> but yet when I use ^C at the GDB prompt my sigtimedwait() call inside
>> my program does return with "2" (SIGINT), which I don't want.
>>
>> I guess I don't understand what the docs mean when they say that the
>> signal is not passed to the process under debug.  Note that in my case
>> I'm attaching to the program from a completely different terminal so
>> there's no issue with process groups etc. and as far as I can
>> understand it, the signal should only be delivered to GDB not my
>> process, so unless there's some weird magic at work here it must be GDB
>> forwarding that signal to my process.
>>
>> Anyone have any thoughts about this?
>>
>> Cheers!
> 
> I was able to reproduce it with, it really sounds like a bug.
> 
> If others want to try, here's my test program:
> 
> #include <stdio.h>
> #include <signal.h>
> #include <stdlib.h>
> 
> int main()
> {
>     sigset_t set;
>     sigemptyset(&set);
>     sigaddset(&set, SIGINT);
>     sigprocmask(SIG_BLOCK, &set, NULL);
>     for (int i = 0; i < 10; i++) {
>         int n = sigwaitinfo(&set, NULL);
>         printf("signal %d\n", n);
>     }
> }
> 
> 
> Run in a terminal, attach with GDB in another terminal, continue, then ctrl-C in GDB's terminal.  The first call to sigwaitinfo returns -1, I think it's expected as the syscall gets interrupted.  But the subsequent calls return 2, showing that indeed the process has received a SIGINT.
> 
> Paul, could you please file a bug?  You can re-use this test program if you want.

There are a few things going on here, none of it is really new, though:

#1 - If your program blocks SIGINT and then uses sigwait, then GDB won't
     ever intercept the SIGINT, because ptrace doesn't ever see the signal.
     This is an ancient issue.  See:
       https://sourceware.org/bugzilla/show_bug.cgi?id=9425

#2 - If you attach to a process instead of running it from GDB, then ctrl-c
     reaches gdb first, and then GDB sends/forwards a SIGINT to the child process.
     Normally this then causes ptrace to intercept the SIGINT, but, see #1 above.
     This case ("attach"), could be handled by GDB instead stopping the
     process with SIGTOP or even better, PTRACE_INTERRUPT.  Note, TBC, would
     only work with attach/a separate terminal.

#3 - If you run the process instead of attaching, in the same terminal
     as GDB, then ctrl-c reaches the inferior process first (because
     the inferior is put in the foreground), gdb would have no chance to
     do PTRACE_INTERRUPT at all.  That would be fixable (combined with 
     PTRACE_INTERRUPT) by my WIP branch here:
       https://github.com/palves/gdb/commits/palves/tty-always-separate-session
     which effectively makes the "run" case the same as the "attach" case,
     by always putting the inferior in a separate terminal session.

Thanks,
Pedro Alves

Re: GDB (not) handling SIGINT...?

[Paul Smith]

On Thu, 2018-11-15 at 13:55 +0000, Pedro Alves wrote:
> #2 - If you attach to a process instead of running it from GDB, then ctrl-c
>      reaches gdb first, and then GDB sends/forwards a SIGINT to the child process.

To me this seems to be the problem.  I've explicitly said I don't want
the signal passed to the process under debug (not a child process in
this situation right?)--in fact that's the default setting for SIGINT
--but yet GDB is still passing the signal along.

Isn't that incorrect behavior?

Is it there just for parity between the behavior of attaching to an
existing process versus running the process under GDB?  I'd prefer it
work one way and not the other, rather than not work either way :).

In my environment I'm _always_ attaching because our processes need to
be started through a separate service, or they don't have proper
security tokens to join the system.  So it would work OK for me if just
the attached version behaved as expected.  I get that others may have
different needs.

Re: GDB (not) handling SIGINT...?

[Pedro Alves]

On 11/15/2018 03:51 PM, Paul Smith wrote:
> On Thu, 2018-11-15 at 13:55 +0000, Pedro Alves wrote:
>> #2 - If you attach to a process instead of running it from GDB, then ctrl-c
>>      reaches gdb first, and then GDB sends/forwards a SIGINT to the child process.
> 
> To me this seems to be the problem.  I've explicitly said I don't want
> the signal passed to the process under debug (not a child process in
> this situation right?)--in fact that's the default setting for SIGINT
> --but yet GDB is still passing the signal along.
> 
> Isn't that incorrect behavior?

Not normally, because GDB is not really "passing" the signal.  It's
kill-ing the process with SIGINT, just like if you send a SIGINT
from a separate terminal with "kill -INT $PID".  Normally, ptrace (and
thus gdb) intercepts the just-sent SIGINT _before_ the inferior ever
sees it, and prints the "Program received signal SIGINT" message.
At which point, the handle SIGINT pass/nopass settings come into effect,
if the inferior is resumed again.

The problem is that sigwait makes it so that ptrace does _not_
intercept that SIGINT.  It's sigwait that is special.  So that
SIGINT that GDB sends thinking that ptrace will intercept it,
is not intercepted, but does straight to the inferior.

If your program was instead stopped in sleep(), then with
"handle SIGINT nopass", the SIGINT would _not_ ever be passed to
the inferior.

Of course this means that to handle this, GDB's mechanism to
interrupt the inferior needs to be different.  Evidently kill'ing the
inferior with SIGINT thinking that ptrace will intercept the SIGINT
doesn't work in all cases.

> 
> Is it there just for parity between the behavior of attaching to an
> existing process versus running the process under GDB?  

Yeah, I think so.

> I'd prefer it
> work one way and not the other, rather than not work either way :).

Yeah.  :-)

> 
> In my environment I'm _always_ attaching because our processes need to
> be started through a separate service, or they don't have proper
> security tokens to join the system.  So it would work OK for me if just
> the attached version behaved as expected.  I get that others may have
> different needs.

With current GDB, you can work around this by attaching in non-stop mode
("set non-stop on"), and then using "continue&" to continue in the
background, and the "interrupt" command to interrupt.  In that mode,
that command stops processes with SIGSTOP, not SIGINT.  It's a lot
more convenient with an IDE driving GDB than on the CLI, though.

If you're OK with hacking GDB, s/SIGINT/SIGSTOP/g in child_pass_ctrlc
should work reasonably OK, even though not perfect.

Thanks,
Pedro Alves

Re: GDB (not) handling SIGINT...?

[Paul Smith]

On Thu, 2018-11-15 at 16:24 +0000, Pedro Alves wrote:
> With current GDB, you can work around this by attaching in non-stop
> mode ("set non-stop on"), and then using "continue&" to continue in
> the background, and the "interrupt" command to interrupt.

Tricksy!

> If you're OK with hacking GDB, s/SIGINT/SIGSTOP/g in child_pass_ctrlc
> should work reasonably OK, even though not perfect.

Oh, I understand now.  The SIGINT is passed to the child to get it to
stop.  Somehow I assumed that there was a magical ptrace way to do
this, rather than having to send normal signals.

I guess there is now: using PTRACE_SEIZE instead of PTRACE_ATTACH,
which will allow you to later send PTRACE_INTERRUPT: presumably if this
were used would solve the problem for attached processes anyway...?

Re: GDB (not) handling SIGINT...?

[Pedro Alves]

On 11/15/2018 05:12 PM, Paul Smith wrote:

> Oh, I understand now.  The SIGINT is passed to the child to get it to
> stop.  Somehow I assumed that there was a magical ptrace way to do
> this, rather than having to send normal signals.
> 

Exactly.

> I guess there is now: using PTRACE_SEIZE instead of PTRACE_ATTACH,
> which will allow you to later send PTRACE_INTERRUPT: presumably if this
> were used would solve the problem for attached processes anyway...?
Yup, assuming we're OK with "ctrl-c" in the GDB console not
resulting in SIGINT.

Thanks,
Pedro Alves

Re: GDB (not) handling SIGINT...?

[Paul Smith]

On Thu, 2018-11-15 at 18:04 +0000, Pedro Alves wrote:> 
> > I guess there is now: using PTRACE_SEIZE instead of PTRACE_ATTACH,
> > which will allow you to later send PTRACE_INTERRUPT: presumably if this
> > ere used would solve the problem for attached processes anyway...?
> Yup, assuming we're OK with "ctrl-c" in the GDB console not
> resulting in SIGINT.

It seems to me that, as long as nopass is set on SIGINT, that would be
the expected/correct behavior... is there something unexpected about
the way it would work?

I'll look at using SIGSTOP instead of SIGINT, for my situation.

Re: GDB (not) handling SIGINT...?

[Paul Smith]

On Thu, 2018-11-15 at 16:24 +0000, Pedro Alves wrote:
> If you're OK with hacking GDB, s/SIGINT/SIGSTOP/g in child_pass_ctrlc
> should work reasonably OK, even though not perfect.

I got around to trying this and it did work... except for one thing:
when I wanted to continue I had to continue once for each thread in my
program :-/.  Actually it stopped thread 1 twice, once at first then
again...

This is GDB 8.2 on x86_64 GNU/Linux (Linux 4.15, GNU libc 2.27)

Somewhat confusing.


$ gdb -p 7351
  ...
Attaching to process 7351
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f17b0b0826c in sigtimedwait () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) c
Continuing.
^C
Thread 1 "foo" received signal SIGSTOP, Stopped (signal).
0x00007f17b0b0826c in sigtimedwait () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) c
Continuing.

Thread 11 "foo" received signal SIGSTOP, Stopped (signal).
[Switching to Thread 0x7f17a91fe700 (LWP 7369)]
0x00007f17b0ec69f3 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb)
Continuing.

Thread 6 "foo" received signal SIGSTOP, Stopped (signal).
[Switching to Thread 0x7f17acfff700 (LWP 7361)]
0x00007f17b0ec6f85 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb)
Continuing.

Thread 3 "foo" received signal SIGSTOP, Stopped (signal).
[Switching to Thread 0x7f17af7ff700 (LWP 7356)]
0x00007f17b0be9bb7 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
(gdb)
Continuing.

Thread 9 "foo" received signal SIGSTOP, Stopped (signal).
[Switching to Thread 0x7f17aa5ff700 (LWP 7367)]
0x00007f17b0ec69f3 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb)
Continuing.

Thread 4 "foo" received signal SIGSTOP, Stopped (signal).
[Switching to Thread 0x7f17ae7ff700 (LWP 7357)]
0x00007f17b0ec69f3 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb)
Continuing.

Thread 7 "foo" received signal SIGSTOP, Stopped (signal).
[Switching to Thread 0x7f17ac7fe700 (LWP 7362)]
0x00007f17b0ec6f85 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb)
Continuing.

Thread 1 "foo" received signal SIGSTOP, Stopped (signal).
[Switching to Thread 0x7f17b3ccb8c0 (LWP 7351)]
0x00007f17b0b0826c in sigtimedwait () from /lib/x86_64-linux-gnu/libc.so.6
(gdb)
Continuing.

Thread 12 "foo" received signal SIGSTOP, Stopped (signal).
[Switching to Thread 0x7f17a3fff700 (LWP 7370)]
0x00007f17b0ec6f85 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb)
Continuing.

Thread 2 "foo" received signal SIGSTOP, Stopped (signal).
[Switching to Thread 0x7f17b03ed700 (LWP 7355)]
0x00007f17b0ec6f85 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb)
Continuing.

Thread 10 "foo" received signal SIGSTOP, Stopped (signal).
[Switching to Thread 0x7f17a99ff700 (LWP 7368)]
0x00007f17b0eca82e in recv () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb)
Continuing.

Thread 8 "foo" received signal SIGSTOP, Stopped (signal).
[Switching to Thread 0x7f17ab3ff700 (LWP 7363)]
0x00007f17b0ec69f3 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb)
Continuing.

Thread 5 "foo" received signal SIGSTOP, Stopped (signal).
[Switching to Thread 0x7f17adbff700 (LWP 7359)]
0x00007f17b0eca82e in recv () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb)
Continuing.

  ... finally here the program is really running again ...

Re: GDB (not) handling SIGINT...?

[Paul Smith]

On Mon, 2018-11-19 at 14:07 -0500, Paul Smith wrote:
> I got around to trying this and it did work... except for one thing:
> when I wanted to continue I had to continue once for each thread in
> my program :-/.  Actually it stopped thread 1 twice, once at first
> then again...

I also discovered that detaching doesn't work properly; I guess we have
to send along a SIGCONT when we detach.  Yep, if I manually kill -CONT
the process after I detach it wakes back up.

So yeah, possibly a reasonable alternative (although I'm not sure about
the delivery of SIGSTOP to each thread...) but it does need a bit more
love than just swapping SIGSTOP for SIGINT in child_pass_ctrlc() :)