[lttng-dev] sdt.h tracepoints with unicode data and/or structs

[lttng-dev] sdt.h tracepoints with unicode data and/or structs

[Mathieu Desnoyers]

----- On Sep 22, 2016, at 8:56 AM, Milian Wolff milian.wolff at kdab.com wrote:

> On Wednesday, September 21, 2016 5:22:35 PM CEST Mathieu Desnoyers wrote:
>> ----- On Sep 20, 2016, at 7:22 PM, Philippe Proulx eeppeliteloop at gmail.com
> wrote:
>> >>> >> > Can I pass UTF16 strings? Do they need to be null-terminated?
>> >>> >> 
>> >>> >> You should convert them to UTF8.
>> >>> >> 
>> >>> >> A ctf_string() needs to be null-terminated. A ctf_sequence_text() is
>> >>> >> not required to be null-terminated. See
>> >>> >> http://lttng.org/docs/#doc-liblttng-ust-tp-fields
>> >>> > 
>> >>> > This sounds like tracing would then incur a huge runtime overhead,
>> >>> > when I
>> >>> > need to convert all my UTF-16 strings to UTF-8. How does one deal with
>> >>> > that?
>> >>> 
>> >>> Are you concerned about the runtime overhead when tracing is disabled,
>> >>> or
>> >>> enabled ? It would be good to gather some metrics on the overhead of
>> >>> this
>> >>> conversion.
>> >> 
>> >> I'm mostly concerned about the overhead when tracing is disabled.
>> >> Ideally, I would like to see these tracepoints unconditionally compiled
>> >> into Qt. But if the overhead is noticeable when they are not in use, I
>> >> may have a hard time achieving this goal. Instead, one will then
>> >> probably need to recompile Qt with the tracepoints enabled.
>> > 
>> > As Mathieu wrote, the overhead of disabled LTTng-UST tracepoints is pretty
>> > much unnoticeable. You can use the tracepoint_enabled() and
>> > do_tracepoint()
>> > macros to perform some computation, conversions, and prepare stuff
>> > specifically for the payload of the event.
>> > 
>> >> When in use, the overhead of the tracepoints should also be minimal.
>> >> Allocating memory for a temporary UTF-16 to UTF-8 conversion alone has a
>> >> large overhead, and then converting the data to UTF-8 also adds on top
>> >> of that. Thus, if at all possible, I would like to prevent that.
>> > 
>> > There's no way to support UTF-16 as of the current versions of LTTng and
>> > CTF. Even if you find a way to store the string as is, for example using a
>> > sequence of bytes, the existing CTF viewers and analyzers won't recognize
>> > the field as a string.
>> > 
>> > However we could think about a way to support UTF-16 in the future, and
>> > possibly other Unicode encodings too.
>> > 
>> >> Similarly, I am looking for a way to put URLs into tracepoints, and
>> >> converting a QUrl to string data is far from cheap.
>> > 
>> > Yes, I see that this code is executed:
>> > <https://github.com/qt/qtbase/blob/dev/src/corelib/io/qurl.cpp#L3279>.
>> > 
>> > You could allocate one tracepoint field for each individual attribute of
>> > the QUrl object, for example the scheme, the path, the query string, the
>> > fragment, etc. But I guess you'd still have the UTF-16 issue.
>> > 
>> >> At this point, I don't see a way around only enabling tracepoints
>> >> optionally. Independent of the mechanism in use for the actual
>> >> tracepoints (i.e. lttng-ust or sdt). The conditional to check whether
>> >> tracing is enabled may not be too bad. But then in the conditional it
>> >> looks like $some code will be required to massage the data into a form
>> >> that the tracepoints accept them. This increase in code size negatively
>> >> influences code caches and I don't see any way around that.
>> > 
>> > I leave this part for Mathieu ;-).
>> 
>> This "extra code" can be implemented within the tracepoint provider,
>> which is a cache cold function, not used at all when tracing is disabled.
> 
> This sounds excellent. Can you tell me how? Could you maybe add an example to
> lttng-ust. Also note how http://lttng.org/man/3/lttng-ust/v2.7 says:
> 
>	if (tracepoint_enabled(ust_tests_hello, tptest)) {
>		/* prepare arguments */
>		do_tracepoint(ust_tests_hello, tptest, i, netint, values,
>			text, strlen(text), dbl, flt);
>	}
> 
> If I understood you correctly, then I could do something like
> 
>	if (tracepoint_enabled(ust_tests_hello, tptest)) {
>		/* don't prepare arguments */
>		do_tracepoint(ust_tests_hello, my_complex_data);
>	}
> 
> And then have the "prepare" code somewhere in my TRACEPOINT_EVENT?
> 

Yes. Both approaches can be used.

Note that the second approach you refer to is the same as using a
plain tracepoint() macro and doing the preparation within
the TRACEPOINT_EVENT() macro.

The preparation within TRACEPOINT_EVENT() can currently only be
done as expression evaluation in the TP_FIELDS. LTTng-modules has
more flexibility in that respect, but not lttng-ust yet.

A patch contributing such example to lttng-ust would be welcome :)

Thanks,

Mathieu


> Thanks
> --
> Milian Wolff | milian.wolff at kdab.com | Software Engineer
> KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
> Tel: +49-30-521325470
> KDAB - The Qt Experts

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[lttng-dev] sdt.h tracepoints with unicode data and/or structs

[Philippe Proulx]

>> This sounds excellent. Can you tell me how? Could you maybe add an example to
>> lttng-ust. Also note how http://lttng.org/man/3/lttng-ust/v2.7 says:
>>
>>       if (tracepoint_enabled(ust_tests_hello, tptest)) {
>>               /* prepare arguments */
>>               do_tracepoint(ust_tests_hello, tptest, i, netint, values,
>>                       text, strlen(text), dbl, flt);
>>       }
>>
>> If I understood you correctly, then I could do something like
>>
>>       if (tracepoint_enabled(ust_tests_hello, tptest)) {
>>               /* don't prepare arguments */
>>               do_tracepoint(ust_tests_hello, my_complex_data);
>>       }
>>
>> And then have the "prepare" code somewhere in my TRACEPOINT_EVENT?
>>
>
> Yes. Both approaches can be used.
>
> Note that the second approach you refer to is the same as using a
> plain tracepoint() macro and doing the preparation within
> the TRACEPOINT_EVENT() macro.

Here's an example (C++):

    #define _my_enum_evaluation(_enum, _field, _member)                     \
      ctf_enum(my_provider, _enum, int, _field,                             \
               (_member).is_bound() ?                                       \
                 ((SomeApi::_enum::enum_type) (_member) <
SomeApi::_enum::UNKNOWN_VALUE ? \
                   (int) ((SomeApi::_enum::enum_type) (_member)) : INT_MAX  \
                 ) : INT_MAX)

    TRACEPOINT_ENUM(
      my_provider,
      my_enum,
      TP_ENUM_VALUES(
        ctf_enum_value("unknown", 0)
        ctf_enum_value("apple", 1)
        ctf_enum_value("banana", 2)
        ctf_enum_value("orange", 3)
        ctf_enum_value("strawberry", 4)
      )
    )

    TRACEPOINT_EVENT(
      my_provider,
      my_tracepoint,
      TP_ARGS(
        int something,
        const SomeApi::SomeObject&, object
      ),
      TP_FIELDS(
      ctf_integer(int, int_field, something)
        _my_enum_evaluation(my_enum, enum_field, object.someMember())
      )
    )

You can call functions in there, evaluate conditions using the ternary operator,
etc.

Phil

>
> The preparation within TRACEPOINT_EVENT() can currently only be
> done as expression evaluation in the TP_FIELDS. LTTng-modules has
> more flexibility in that respect, but not lttng-ust yet.
>
> A patch contributing such example to lttng-ust would be welcome :)
>
> Thanks,
>
> Mathieu
>
>
>> Thanks
>> --
>> Milian Wolff | milian.wolff at kdab.com | Software Engineer
>> KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
>> Tel: +49-30-521325470
>> KDAB - The Qt Experts
>
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

[lttng-dev] sdt.h tracepoints with unicode data and/or structs

[Milian Wolff]

On Tuesday, October 4, 2016 11:50:06 AM CEST Philippe Proulx wrote:
> >> This sounds excellent. Can you tell me how? Could you maybe add an
> >> example to>> 
> >> lttng-ust. Also note how http://lttng.org/man/3/lttng-ust/v2.7 says:
> >>       if (tracepoint_enabled(ust_tests_hello, tptest)) {
> >>       
> >>               /* prepare arguments */
> >>               do_tracepoint(ust_tests_hello, tptest, i, netint, values,
> >>               
> >>                       text, strlen(text), dbl, flt);
> >>       
> >>       }
> >> 
> >> If I understood you correctly, then I could do something like
> >> 
> >>       if (tracepoint_enabled(ust_tests_hello, tptest)) {
> >>       
> >>               /* don't prepare arguments */
> >>               do_tracepoint(ust_tests_hello, my_complex_data);
> >>       
> >>       }
> >> 
> >> And then have the "prepare" code somewhere in my TRACEPOINT_EVENT?
> > 
> > Yes. Both approaches can be used.
> > 
> > Note that the second approach you refer to is the same as using a
> > plain tracepoint() macro and doing the preparation within
> > the TRACEPOINT_EVENT() macro.
> 
> Here's an example (C++):
> 
>     #define _my_enum_evaluation(_enum, _field, _member)                    
> \ ctf_enum(my_provider, _enum, int, _field,                             \
> (_member).is_bound() ?                                       \
> ((SomeApi::_enum::enum_type) (_member) <
> SomeApi::_enum::UNKNOWN_VALUE ? \
>                    (int) ((SomeApi::_enum::enum_type) (_member)) : INT_MAX 
> \ ) : INT_MAX)
> 
>     TRACEPOINT_ENUM(
>       my_provider,
>       my_enum,
>       TP_ENUM_VALUES(
>         ctf_enum_value("unknown", 0)
>         ctf_enum_value("apple", 1)
>         ctf_enum_value("banana", 2)
>         ctf_enum_value("orange", 3)
>         ctf_enum_value("strawberry", 4)
>       )
>     )
> 
>     TRACEPOINT_EVENT(
>       my_provider,
>       my_tracepoint,
>       TP_ARGS(
>         int something,
>         const SomeApi::SomeObject&, object
>       ),
>       TP_FIELDS(
>       ctf_integer(int, int_field, something)
>         _my_enum_evaluation(my_enum, enum_field, object.someMember())
>       )
>     )
> 
> You can call functions in there, evaluate conditions using the ternary
> operator, etc.

Thank you, that is exactly the example that I was looking for.

Cheers

-- 
Milian Wolff | milian.wolff at kdab.com | Software Engineer
KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt Experts
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5903 bytes
Desc: not available
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20161005/f838498a/attachment-0001.bin>