[lttng-dev] [RFC] User space stack trace on system calls

[lttng-dev] [RFC] User space stack trace on system calls

[Francis Giraldeau]

Hi,

To locate where are made system calls in an application, I developed two
prototypes that dumps stack traces.

The first one is using event context [1]. When an tracepoint is reached,
then the stack trace of the currently running userspace thread is done
and saved. This event context can be attached to any event, thus this
technique is general. The drawback is that the context is defined per
channel, thus the user will usually wants a separate channel for
selected events. The other drawback is related to the way lttng handles
system call events. By enabling system calls, we get automatically both
entries and exits events in a single channel, and thus the same user
space stack trace is dumped twice. It also requires changes to lttng-tools.

To avoid this issue, I also tried to create one event called
sys_entry_callsite that is called only on sys_entry [2]. Thus, the stack
trace is recorded only once. It's also possible to enable it with all
other events in one channel. It's less flexible, but easier to use. It
also requires to define a tracepoint in a module, so on latest ubuntu,
it requires the modules to be signed. The code that dumps the stack
trace is called only when the actual tracepoint is enabled. We actually
found a way to insert code inside the probe with code blocks that
returns a value. Thanks to Simon Marquis for this C trick!

Both can have their use cases, so maybe both could be used? Anyway, I
post both of them for discussion.

[1]
https://github.com/giraldeau/lttng-modules/blob/addons/lttng-context-callstack.c
[2]
https://github.com/giraldeau/lttng-modules/blob/addons/addons/lttng-syscall-entry.c

Thanks!

Francis Giraldeau

[lttng-dev] [RFC] User space stack trace on system calls

[Francis Giraldeau]

On 2013-08-30 23:38, Francis Giraldeau wrote :
> We actually
> found a way to insert code inside the probe with code blocks that
> returns a value. Thanks to Simon Marchi for this C trick!

Well, it turns out that the function save_stack_trace_user() is called
even when the tracepoint is not enabled, so this is a major issue.
Hum... Then why does it work for UST tracepoint macro?

Cheers,

Francis

[lttng-dev] [RFC] User space stack trace on system calls

[Francis Giraldeau]

Le 2013-08-30 23:59, Francis Giraldeau a ?crit :
> On 2013-08-30 23:38, Francis Giraldeau wrote :
>> We actually
>> found a way to insert code inside the probe with code blocks that
>> returns a value. Thanks to Simon Marchi for this C trick!
> 
> Well, it turns out that the function save_stack_trace_user() is called
> even when the tracepoint is not enabled, so this is a major issue.
> Hum... Then why does it work for UST tracepoint macro?

By moving the probe custom code inside the macro itself, it's called
only when the tracepoint is enabled. The difference is that the UST
tracepoint in the C code is a macro that gets expanded, while in the
kernel trace_somethig is an actual function call.

diff --git a/instrumentation/events/lttng-module/addons.h
b/instrumentation/events/lttng-module/addons.h
index 8a764b0..26623b3 100644
--- a/instrumentation/events/lttng-module/addons.h
+++ b/instrumentation/events/lttng-module/addons.h
@@ -200,15 +200,23 @@ TRACE_EVENT(sys_entry,
 )

 TRACE_EVENT(sys_entry_callsite,
-       TP_PROTO(int id, int len, unsigned long *entries),
-       TP_ARGS(id, len, entries),
+       TP_PROTO(int id),
+       TP_ARGS(id),
        TP_STRUCT__entry(
                __field(short, id)
-               __dynamic_array_hex(unsigned long, callsite, len)
+               __dynamic_array_hex(unsigned long, callsite, ({
+                       extern int stack_trace_get_size(void);
+                       int x = stack_trace_get_size();
+                       x;
+               }))
        ),
        TP_fast_assign(
                tp_assign(id, id)
-               tp_memcpy_dyn(callsite, entries)
+               tp_memcpy_dyn(callsite, ({
+                       extern unsigned long *stack_trace_get_entries(void);
+                       unsigned long *entries = stack_trace_get_entries();
+                       entries;
+               }))
        ),
        TP_printk("%d", __entry->id)
 )