[lttng-dev] SIG33 message

[lttng-dev] SIG33 message

[Mathieu Desnoyers]

----- On Apr 4, 2019, at 3:26 PM, Mosleh Uddin <mosleh1235 at gmail.com> wrote: 

> Hello,
> I am building an application which I am compiling with the 'lttng-ust' flag.
> When I try to debug the application using gdb, I get a 'Suspended: Signal:
> SIG33:Real-time event 33' message. This only happens when I compile with the
> 'lttng-ust' flag, if I remove it I no longer have this issue. Any insight?

Hi, 

Based on the signal(7) man page, it appears that real-time signal 33 is used by the NPTL 
threading library from glibc. 

Indeed, lttng-ust creates 2 threads within the processes it's linked into, so 
your application needs to appropriately interact with glibc NPTL in order to link 
lttng-ust into it. 

Can you provide more details on the nature of your application ? What version 
of gdb do you use ? On which architecture ? 

Thanks, 

Mathieu 

-- 
Mathieu Desnoyers 
EfficiOS Inc. 
http://www.efficios.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20190416/54631360/attachment.html>

[lttng-dev] SIG33 message

[Sebastien Boisvert]

Hi Mosleh,

(I just saw Mathieu's answer arrive in my inbox while I was writing this.)

First, I don't know the exact solution.


In Linux, signals are sent with the kill() function.

In the LTTng-UST source code, there are 3 signals that relate to 'Suspended: Signal: SIG33:Real-time event 33' (RT):

libringbuffer/ring_buffer_frontend.c:82:#define LTTNG_UST_RB_SIG_FLUSH		SIGRTMIN
libringbuffer/ring_buffer_frontend.c:83:#define LTTNG_UST_RB_SIG_READ		SIGRTMIN + 1
libringbuffer/ring_buffer_frontend.c:84:#define LTTNG_UST_RB_SIG_TEARDOWN	SIGRTMIN + 2

=> http://git.lttng.org/?p=lttng-ust.git;a=blob;f=libringbuffer/ring_buffer_frontend.c;h=9721df1634585f028316d13f96b184af7e99cc08;hb=HEAD#l82


On my system (Ubuntu 18.04.2 LTS with Linux 4.15.0-47-generic), SIGRTMIN = 34.

Check with 'kill -l SIGRTMIN' on your system to see if it it the same value.



In the LTTng-UST source code, there 1 this call to kill():

libringbuffer/ring_buffer_frontend.c:767:	kill(getpid(), LTTNG_UST_RB_SIG_TEARDOWN);

=> http://git.lttng.org/?p=lttng-ust.git;a=blob;f=libringbuffer/ring_buffer_frontend.c;h=9721df1634585f028316d13f96b184af7e99cc08;hb=HEAD#l767




These custom signals seem to be handled here:

libringbuffer/ring_buffer_frontend.c:660

=>  http://git.lttng.org/?p=lttng-ust.git;a=blob;f=libringbuffer/ring_buffer_frontend.c;h=9721df1634585f028316d13f96b184af7e99cc08;hb=HEAD#l660




On 2019-04-04 3:26 p.m., Mosleh Uddin wrote:
> Hello,
> 
> I am building an application which I am compiling with the 'lttng-ust' flag. When I try to debug the application using gdb, I get a 'Suspended: Signal: SIG33:Real-time event 33' message. This only happens when I compile with the 'lttng-ust' flag, if I remove it I no longer have this issue. Any insight?
> 
> Regards,
> Mosleh
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> 

[lttng-dev] SIG33 message

[Mathieu Desnoyers]

----- On Apr 16, 2019, at 1:58 PM, Sebastien Boisvert sboisvert at gydle.com wrote:

> Hi Mosleh,
> 
> (I just saw Mathieu's answer arrive in my inbox while I was writing this.)
> 
> First, I don't know the exact solution.
> 
> 
> In Linux, signals are sent with the kill() function.
> 
> In the LTTng-UST source code, there are 3 signals that relate to 'Suspended:
> Signal: SIG33:Real-time event 33' (RT):
> 
> libringbuffer/ring_buffer_frontend.c:82:#define LTTNG_UST_RB_SIG_FLUSH		SIGRTMIN
> libringbuffer/ring_buffer_frontend.c:83:#define LTTNG_UST_RB_SIG_READ		SIGRTMIN
> + 1
> libringbuffer/ring_buffer_frontend.c:84:#define LTTNG_UST_RB_SIG_TEARDOWN
> 	SIGRTMIN + 2
> 
> =>
> http://git.lttng.org/?p=lttng-ust.git;a=blob;f=libringbuffer/ring_buffer_frontend.c;h=9721df1634585f028316d13f96b184af7e99cc08;hb=HEAD#l82
> 
> 
> On my system (Ubuntu 18.04.2 LTS with Linux 4.15.0-47-generic), SIGRTMIN = 34.
> 
> Check with 'kill -l SIGRTMIN' on your system to see if it it the same value.
> 
> 
> 
> In the LTTng-UST source code, there 1 this call to kill():
> 
> libringbuffer/ring_buffer_frontend.c:767:	kill(getpid(),
> LTTNG_UST_RB_SIG_TEARDOWN);
> 
> =>
> http://git.lttng.org/?p=lttng-ust.git;a=blob;f=libringbuffer/ring_buffer_frontend.c;h=9721df1634585f028316d13f96b184af7e99cc08;hb=HEAD#l767
> 
> 
> 
> 
> These custom signals seem to be handled here:
> 
> libringbuffer/ring_buffer_frontend.c:660
> 
> =>
> http://git.lttng.org/?p=lttng-ust.git;a=blob;f=libringbuffer/ring_buffer_frontend.c;h=9721df1634585f028316d13f96b184af7e99cc08;hb=HEAD#l660

Hi Sebastien,

This part of the ring buffer should only be used by the consumer daemon through
liblttng-ust-ctl.so, never from the traced applications.

So I keep suspecting that it's NPTL's use of SIG33 which is causing an old
version of gdb to trap, ref: https://gdb.sourceware.narkive.com/SzqG56iA/program-received-signal-sig33-real-time-event-33

Thanks,

Mathieu

> 
> 
> 
> 
> On 2019-04-04 3:26 p.m., Mosleh Uddin wrote:
>> Hello,
>> 
>> I am building an application which I am compiling with the 'lttng-ust' flag.
>> When I try to debug the application using gdb, I get a 'Suspended: Signal:
>> SIG33:Real-time event 33' message. This only happens when I compile with the
>> 'lttng-ust' flag, if I remove it I no longer have this issue. Any insight?
>> 
>> Regards,
>> Mosleh
>> 
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev at lists.lttng.org
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[lttng-dev] SIG33 message

[Sebastien Boisvert]

On 2019-04-16 2:12 p.m., Mathieu Desnoyers wrote:
[snip]
> 
> Hi Sebastien,
> 
> This part of the ring buffer should only be used by the consumer daemon through
> liblttng-ust-ctl.so, never from the traced applications.
> 
> So I keep suspecting that it's NPTL's use of SIG33 which is causing an old
> version of gdb to trap, ref: https://gdb.sourceware.narkive.com/SzqG56iA/program-received-signal-sig33-real-time-event-33
> 
> Thanks,
> 
> Mathieu
> 

If it is the case that an old gdb is involved,
then it is unrelated to this kill() call site in the LTTng-UST source code:

    libringbuffer/ring_buffer_frontend.c:767:	kill(getpid(), LTTNG_UST_RB_SIG_TEARDOWN);

because LTTNG_UST_RB_SIG_TEARDOWN is SIGRTMIN + 2. According to the man page of signal(7) [1],
SIGRTMIN is 34 (Native Posix Thread Library) or 35 (LinuxThreads, whatever that is).


The equation

    33 = SIGRTMIN + 2

is not satisfied with either 34 or 35.


So, is upgrading gdg the solution ?


[1] http://man7.org/linux/man-pages/man7/signal.7.html


[snip]

[lttng-dev] SIG33 message

[Mathieu Desnoyers]

----- On Apr 16, 2019, at 2:21 PM, Sebastien Boisvert sboisvert at gydle.com wrote:

> On 2019-04-16 2:12 p.m., Mathieu Desnoyers wrote:
> [snip]
>> 
>> Hi Sebastien,
>> 
>> This part of the ring buffer should only be used by the consumer daemon through
>> liblttng-ust-ctl.so, never from the traced applications.
>> 
>> So I keep suspecting that it's NPTL's use of SIG33 which is causing an old
>> version of gdb to trap, ref:
>> https://gdb.sourceware.narkive.com/SzqG56iA/program-received-signal-sig33-real-time-event-33
>> 
>> Thanks,
>> 
>> Mathieu
>> 
> 
> If it is the case that an old gdb is involved,
> then it is unrelated to this kill() call site in the LTTng-UST source code:
> 
>    libringbuffer/ring_buffer_frontend.c:767:	kill(getpid(),
>    LTTNG_UST_RB_SIG_TEARDOWN);
> 
> because LTTNG_UST_RB_SIG_TEARDOWN is SIGRTMIN + 2. According to the man page of
> signal(7) [1],
> SIGRTMIN is 34 (Native Posix Thread Library) or 35 (LinuxThreads, whatever that
> is).
> 
> 
> The equation
> 
>    33 = SIGRTMIN + 2
> 
> is not satisfied with either 34 or 35.
> 
> 
> So, is upgrading gdg the solution ?

Based on the gdb forum link I showed, either explicitly disable handling
SIG33 in older gdb:

handle SIG33 nostop noprint pass

or upgrade gdb.

Thanks,

Mathieu

> 
> 
> [1] http://man7.org/linux/man-pages/man7/signal.7.html
> 
> 
> [snip]

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com