From mboxrd@z Thu Jan 1 00:00:00 1970 From: mathieu.desnoyers@efficios.com (Mathieu Desnoyers) Date: Tue, 4 Oct 2016 15:36:48 +0000 (UTC) Subject: [lttng-dev] sdt.h tracepoints with unicode data and/or structs In-Reply-To: <12621392.supdKG9qFk@milian-kdab2> References: <133954640.uMaX5Hy6f7@agathebauer> <1494937800.28038.1474478555753.JavaMail.zimbra@efficios.com> <12621392.supdKG9qFk@milian-kdab2> Message-ID: <1527185266.45326.1475595408856.JavaMail.zimbra@efficios.com> ----- On Sep 22, 2016, at 8:56 AM, Milian Wolff milian.wolff at kdab.com wrote: > On Wednesday, September 21, 2016 5:22:35 PM CEST Mathieu Desnoyers wrote: >> ----- On Sep 20, 2016, at 7:22 PM, Philippe Proulx eeppeliteloop at gmail.com > wrote: >> >>> >> > Can I pass UTF16 strings? Do they need to be null-terminated? >> >>> >> >> >>> >> You should convert them to UTF8. >> >>> >> >> >>> >> A ctf_string() needs to be null-terminated. A ctf_sequence_text() is >> >>> >> not required to be null-terminated. See >> >>> >> http://lttng.org/docs/#doc-liblttng-ust-tp-fields >> >>> > >> >>> > This sounds like tracing would then incur a huge runtime overhead, >> >>> > when I >> >>> > need to convert all my UTF-16 strings to UTF-8. How does one deal with >> >>> > that? >> >>> >> >>> Are you concerned about the runtime overhead when tracing is disabled, >> >>> or >> >>> enabled ? It would be good to gather some metrics on the overhead of >> >>> this >> >>> conversion. >> >> >> >> I'm mostly concerned about the overhead when tracing is disabled. >> >> Ideally, I would like to see these tracepoints unconditionally compiled >> >> into Qt. But if the overhead is noticeable when they are not in use, I >> >> may have a hard time achieving this goal. Instead, one will then >> >> probably need to recompile Qt with the tracepoints enabled. >> > >> > As Mathieu wrote, the overhead of disabled LTTng-UST tracepoints is pretty >> > much unnoticeable. You can use the tracepoint_enabled() and >> > do_tracepoint() >> > macros to perform some computation, conversions, and prepare stuff >> > specifically for the payload of the event. >> > >> >> When in use, the overhead of the tracepoints should also be minimal. >> >> Allocating memory for a temporary UTF-16 to UTF-8 conversion alone has a >> >> large overhead, and then converting the data to UTF-8 also adds on top >> >> of that. Thus, if at all possible, I would like to prevent that. >> > >> > There's no way to support UTF-16 as of the current versions of LTTng and >> > CTF. Even if you find a way to store the string as is, for example using a >> > sequence of bytes, the existing CTF viewers and analyzers won't recognize >> > the field as a string. >> > >> > However we could think about a way to support UTF-16 in the future, and >> > possibly other Unicode encodings too. >> > >> >> Similarly, I am looking for a way to put URLs into tracepoints, and >> >> converting a QUrl to string data is far from cheap. >> > >> > Yes, I see that this code is executed: >> > . >> > >> > You could allocate one tracepoint field for each individual attribute of >> > the QUrl object, for example the scheme, the path, the query string, the >> > fragment, etc. But I guess you'd still have the UTF-16 issue. >> > >> >> At this point, I don't see a way around only enabling tracepoints >> >> optionally. Independent of the mechanism in use for the actual >> >> tracepoints (i.e. lttng-ust or sdt). The conditional to check whether >> >> tracing is enabled may not be too bad. But then in the conditional it >> >> looks like $some code will be required to massage the data into a form >> >> that the tracepoints accept them. This increase in code size negatively >> >> influences code caches and I don't see any way around that. >> > >> > I leave this part for Mathieu ;-). >> >> This "extra code" can be implemented within the tracepoint provider, >> which is a cache cold function, not used at all when tracing is disabled. > > This sounds excellent. Can you tell me how? Could you maybe add an example to > lttng-ust. Also note how http://lttng.org/man/3/lttng-ust/v2.7 says: > > if (tracepoint_enabled(ust_tests_hello, tptest)) { > /* prepare arguments */ > do_tracepoint(ust_tests_hello, tptest, i, netint, values, > text, strlen(text), dbl, flt); > } > > If I understood you correctly, then I could do something like > > if (tracepoint_enabled(ust_tests_hello, tptest)) { > /* don't prepare arguments */ > do_tracepoint(ust_tests_hello, my_complex_data); > } > > And then have the "prepare" code somewhere in my TRACEPOINT_EVENT? > Yes. Both approaches can be used. Note that the second approach you refer to is the same as using a plain tracepoint() macro and doing the preparation within the TRACEPOINT_EVENT() macro. The preparation within TRACEPOINT_EVENT() can currently only be done as expression evaluation in the TP_FIELDS. LTTng-modules has more flexibility in that respect, but not lttng-ust yet. A patch contributing such example to lttng-ust would be welcome :) Thanks, Mathieu > Thanks > -- > Milian Wolff | milian.wolff at kdab.com | Software Engineer > KDAB (Deutschland) GmbH&Co KG, a KDAB Group company > Tel: +49-30-521325470 > KDAB - The Qt Experts -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com