[lttng-dev] workqueue tracing symbols

[lttng-dev] workqueue tracing symbols

[Akhil Veeraghanta via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 1046 bytes --]

Hello,

We are trying to debug kernel work queue related issues in our system. And as such, we've turned on workqueue_queue_work, workqueue_active_work, workqueue_execute_start, workqueue_execute_end.

The tracepoint itself stores the function pointer which we can look up from /proc/kallsyms manually.

We've been struggling to do this automatically, we've tried:


  1.
Using babeltrace2 to "convert" from lttng to lttng by just editing the events in place. This doesn't work (atleast as easily) because the event iterator is const.

  2.
Loading the kallsyms file with "Configure Symbols" in trace compass doesn't work either because it doesn't know that it can look up functions in workqueue events from that file.

Converting to a text based format like chrome trace and replacing the fields works, but then we have to use chrome::tracing which isn't as powerful.

Before we go any futher modifying trace compass or writing a smarter parser, is there a way we can trace the workqueues and do the lookup properly?

  3.


[-- Attachment #1.2: Type: text/html, Size: 2198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] workqueue tracing symbols

[Mathieu Desnoyers via lttng-dev]

On 2024-09-17 02:44, Akhil Veeraghanta via lttng-dev wrote:
> Hello,
> 
> We are trying to debug kernel work queue related issues in our system. 
> And as such, we've turned on workqueue_queue_work, 
> workqueue_active_work, workqueue_execute_start, workqueue_execute_end.
> 
> The tracepoint itself stores the function pointer which we can look up 
> from /proc/kallsyms manually.
> 
> We've been struggling to do this automatically, we've tried:
> 
>  1.
>     Using babeltrace2 to "convert" from lttng to lttng by just editing
>     the events in place. This doesn't work (atleast as easily) because
>     the event iterator is const.

Augmenting the CTF trace with extra symbolic information using
babeltrace2 would be the preferred way. We have examples on how to
do it with the debug-info babeltrace2 filter component. CCing
Philippe and Simon about this.

> 
>  2.
>     Loading the kallsyms file with "Configure Symbols" in trace compass
>     doesn't work either because it doesn't know that it can look up
>     functions in workqueue events from that file.
> 
>     Converting to a text based format like chrome trace and replacing
>     the fields works, but then we have to use chrome::tracing which
>     isn't as powerful.
> 
>     Before we go any futher modifying trace compass or writing a smarter
>     parser, is there a way we can trace the workqueues and do the lookup
>     properly?

Fetching the symbol information at tracing time would add a significant
overhead in terms of CPU cycles and wasted ring buffer space. Symbol
resolution should really be kept to post-processing unless there are
really no other options.

Thanks,

Mathieu


> 
>  3.
> 
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev