LTTng kernel tracer writes packetized (binary) metadata instead of plain text on Ubuntu 22.04 5.15.0-140/141-generic with LTTng 2.13.18

[Shreya Tripathi via lttng-dev <lttng-dev@lists.lttng.org>]

[-- Attachment #1: Type: text/plain, Size: 3078 bytes --]

Component(s) involved:

  *   LTTng-modules (kernel tracing)

  *   LTTng-tools

________________________________
Component version:

  *   lttng-tools: 2.13.15 (lttng --version)

  *   lttng-modules-dkms: 2.13.18 (modinfo lttng_tracer | grep version)

  *   Babeltrace2: 2.0.4 (babeltrace2 --version)

________________________________
System information:

  *   Architecture: x86_64, 4 CPUs

  *   OS: Ubuntu 22.04.4 LTS

  *   Kernel:

  *   Linux shrtripathi-vm 5.15.0-140-generic #150-Ubuntu SMP Sat Apr 12 06:00:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

  *   Also tested: 5.15.0-141-generic

  *   Distribution: Ubuntu 22.04.4 LTS

Expected results: After running a kernel trace session, the file /tmp/my-kernel-trace/kernel/metadata should be plain text, starting with /* CTF 1.8 */, and be readable by Babeltrace2 and the Python bt2 bindings.
________________________________
Actual results:

  *   The metadata file is written in "packetized" (binary) format, as detected by the file command:
text
Apply
  /tmp/my-kernel-trace/kernel/metadata: Common Trace Format (CTF) packetized metadata (LE), v1.8

  *   The first line is binary garbage, not /* CTF 1.8 */.

  *   Babeltrace2 and the Python bt2 bindings cannot parse the trace, reporting errors about invalid metadata.

  *   Example of the first lines of the metadata file:
 W�u�kzM�W{L����^l�typealias integer { size = 8; align = 8; signed = false; } := uint8_t;
  typealias integer { size = 16; align = 8; signed = false; } := uint16_t;
  ...

Logs and context:

  *   Babeltrace2 error:

  At line 146 in metadata stream: Invalid character: char="", val=0x1d
  At line 146 in metadata stream: syntax error, unexpected CTF_ERROR, expecting CTF_TYPEASSIGN or CTF_EQUAL: token=""


  *   All LTTng kernel modules and user-space tools are version 2.13.18.

  *   All modules are loaded from /lib/modules/5.15.0-140-generic/updates/dkms/ and match the running kernel.

  *   No old or mismatched modules are present.

Loaded kernel modules:

lsmod | grep lttng
lttng_clock            16384  0
lttng_kretprobes       16384  0
lttng_kprobes          16384  0
lttng_uprobes          16384  0
lttng_statedump       749568  0
lttng_wrapper          16384  1 lttng_statedump
...

Additional context:

  *   The issue persists across multiple kernel versions (5.15.0-140-generic, 5.15.0-141-generic).

  *   User-space tracing works as expected (metadata is plain text).

  *   The issue is not fixable by editing the file, as the format is fundamentally binary.

________________________________
What should happen:
The kernel tracer should write plain text CTF metadata, starting with /* CTF 1.8 */, so that Babeltrace2 and the Python bt2 bindings can parse the trace.What happens:The kernel tracer writes packetized (binary) metadata, which is not usable for analysis.
________________________________
Thank you for your help! Please let me know if you need any more logs or information.

Regards
Shreya Tripathi

[-- Attachment #2: Type: text/html, Size: 62928 bytes --]