Hi Philippe, Great proposal, I have some _quick_ feedback. I tried the mental exercise of reading the example 8 and being a matthew-based-parser. It would have been much easier if you make a table, let's say 3.1 where the description column is the first column, or the key of table 3. Example: Name Type Description Required? Default type JSON string Type of F.The value of this property MUST be one of: "fixed-length-bit-array" F is a fixed-length bit array field class. "fixed-length-boolean" F is a fixed-length boolean field class. Yes Would make a new table (in CSV) Value, Type, Description fixes-length-bit-array, JSON string, A fixed-length bit array field class describes fixed-length bit array fields. fixes-length-boolean-field, JSON string, A fixed-length boolean field class is a fixed-length bit array field class which describes fixed-length boolean fields. Potentially extremely out of scope Another general feedback would be that CTF is a data transport layer, it is information rich in the sense that it will tell you what the type of data is though. Would it be interesting to have a protocol ABOVE CTF to define a common way to specify data and make visualizing it easier? If you look at the open tracing API or trace event (google), they have examples, but in the case of CTF, I suggest considering the following with the equivalent UI element: * stack entry-exit, (nested are guaranteed to be smaller) * request begin-end (nested may be any size) * lifespan create-destroy, (cannot nest) * message begin-end (link stacks or requests) * instant (add a tick) * frame start-frame end (contiguous global info, cannot have overlapping, think fps) * stack (link to code) BR Matthew ________________________________ From: tracecompass-dev-bounces@eclipse.org on behalf of tracecompass developer discussions Sent: Wednesday, November 18, 2020 3:30 PM To: diamon-discuss Cc: lttng-dev ; tracecompass-dev Subject: [tracecompass-dev] CTF2-PROP-2.0: Proposal for a major revision of the Common Trace Format, version 1.8 Hello, people of the tracing world! This is a major revision of the proposal for a major revision of the Common Trace Format 1.8 (to CTF 2). This document, named CTF2-PROP-2.0, is available here: It's a revision of CTF2-PROP-1.0 which I sent to this mailing list on October 25 2016: I'm copying the AsciiDoc source here so that you can inline-comment it. If you do so, _please_ keep only a few lines of context (only what's necessary) above and below any comment, because the source has about 6000 lines of text. Thank you for your comments! Philippe Proulx EfficiOS Inc. https://protect2.fireeye.com/v1/url?k=28b9d28c-7722e8c9-28b99217-867b36d1634c-2745ea8ed85463a9&q=1&e=698f127a-baf5-4d0a-8f64-50168d8d98c6&u=http%3A%2F%2Fwww.efficios.com%2F - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - // Please render with Asciidoctor = **CTF2-PROP-2.0**: Proposal for a major revision of the Common Trace Format, version 1.8 Philippe Proulx v2.0, 18 November 2020 :attribute-missing: warn :icons: font :nofooter: :sectnums: :sectnumlevels: 5 :toc: left :toclevels: 3 :nbh: ‑ :minus: − :times: × :noteq: ≠ :ieee754: https://standards.ieee.org/standard/754-2008.html[IEEE 754-2008]'s binary interchange format :ctf1-nl: CTF{nbsp}1 :ctf1: https://protect2.fireeye.com/v1/url?k=32b6763d-6d2d4c78-32b636a6-867b36d1634c-8fa7e342363b715f&q=1&e=698f127a-baf5-4d0a-8f64-50168d8d98c6&u=https%3A%2F%2Fdiamon.org%2Fctf%2Fv1.8.3%2F%5B%257Bctf1-nl}] :ctf2: CTF{nbsp}2 :fl-ba: fixed-length bit array :c-fl-ba: Fixed-length bit array :vl-ba: variable-length bit array :c-vl-ba: Variable-length bit array :fl-bool: fixed-length boolean :c-fl-bool: Fixed-length boolean :fl-int: fixed-length integer :c-fl-int: Fixed-length integer :fl-uint: fixed-length unsigned integer :c-fl-uint: Fixed-length unsigned integer :fl-sint: fixed-length signed integer :c-fl-sint: Fixed-length signed integer :vl-int: variable-length integer :c-vl-int: Variable-length integer :vl-uint: variable-length unsigned integer :c-vl-uint: Variable-length unsigned integer :vl-sint: variable-length signed integer :c-vl-sint: Variable-length signed integer :fl-enum: fixed-length enumeration :c-fl-enum: Fixed-length enumeration :fl-uenum: fixed-length unsigned enumeration :c-fl-uenum: Fixed-length unsigned enumeration :fl-senum: fixed-length signed enumeration :c-fl-senum: Fixed-length signed enumeration :vl-enum: variable-length enumeration :c-vl-enum: Variable-length enumeration :vl-uenum: variable-length unsigned enumeration :c-vl-uenum: Variable-length unsigned enumeration :vl-senum: variable-length signed enumeration :c-vl-senum: Variable-length signed enumeration :fl-fp: fixed-length floating point number :c-fl-fp: Fixed-length floating point number :str: null-terminated string :c-str: Null-terminated string :sl-array: static-length array :c-sl-array: Static-length array :sl-str: static-length string :c-sl-str: Static-length string :dl-array: dynamic-length array :c-dl-array: Dynamic-length array :dl-str: dynamic-length string :c-dl-str: Dynamic-length string :sl-blob: static-length BLOB :c-sl-blob: Static-length BLOB :dl-blob: dynamic-length BLOB :c-dl-blob: Dynamic-length BLOB :fl-ba-fc: <> :c-fl-ba-fc: <> :vl-ba-fc: <> :c-vl-ba-fc: <> :fl-bool-fc: <> :c-fl-bool-fc: <> :fl-int-fc: <> :c-fl-int-fc: <> :fl-uint-fc: <> :c-fl-uint-fc: <> :fl-sint-fc: <> :c-fl-sint-fc: <> :vl-int-fc: <> :c-vl-int-fc: <> :vl-uint-fc: <> :c-vl-uint-fc: <> :vl-sint-fc: <> :c-vl-sint-fc: <> :fl-enum-fc: <> :c-fl-enum-fc: <> :fl-uenum-fc: <> :c-fl-uenum-fc: <> :fl-senum-fc: <> :c-fl-senum-fc: <> :vl-enum-fc: <> :c-vl-enum-fc: <> :vl-uenum-fc: <> :c-vl-uenum-fc: <> :vl-senum-fc: <> :c-vl-senum-fc: <> :fl-fp-fc: <> :c-fl-fp-fc: <> :str-fc: <> :c-str-fc: <> :sl-array-fc: <> :c-sl-array-fc: <> :sl-str-fc: <> :c-sl-str-fc: <> :dl-array-fc: <> :c-dl-array-fc: <> :dl-str-fc: <> :c-dl-str-fc: <> :sl-blob-fc: <> :c-sl-blob-fc: <> :dl-blob-fc: <> :c-dl-blob-fc: <> :struct-fc: <> :c-struct-fc: <> :opt-fc: <> :c-opt-fc: <> :var-fc: <> :c-var-fc: <> :diamon: https://protect2.fireeye.com/v1/url?k=c6f67cca-996d468f-c6f63c51-867b36d1634c-ccf548966b1a6170&q=1&e=698f127a-baf5-4d0a-8f64-50168d8d98c6&u=https%3A%2F%2Fdiamon.org%2F%5BDiaMon Workgroup] :c-bo: https://en.wikipedia.org/wiki/Endianness[Byte order] :must: pass:q[__MUST__] :must-not: pass:q[__MUST NOT__] :required: pass:q[__REQUIRED__] :should: pass:q[__SHOULD__] :should-not: pass:q[__SHOULD NOT__] :may: pass:q[__MAY__] :optional: pass:q[__OPTIONAL__] :var-f: pass:q[__**F**__] :var-o: pass:q[__**O**__] :var-v: pass:q[__**V**__] :var-p: pass:q[__**P**__] :var-s: pass:q[__**S**__] :var-po: pass:q[__**PO**__] :var-o-minus-po: pass:q[__**O**__ − __**PO**__] :doc-id: pass:q[__**CTF2-PROP-2.0**__] This document is an informal proposal for the next major revision of the **Common Trace Format** (CTF) version{nbsp}2 (hereafter named _{ctf2}_). This is _not_ a formal reference. Some parts of this document, however, may be formal enough to be elligible for a specification document. .RFC 2119 IMPORTANT: The key words {must}, {must-not}, {required}, {should}, {should-not}, {may}, and {optional} in this document, when emphasized, are to be interpreted as described in https://www.ietf.org/rfc/rfc2119.txt[RFC{nbsp}2119]. == Revision history Significant changes since https://protect2.fireeye.com/v1/url?k=98714279-c7ea783c-987102e2-867b36d1634c-8e327483f8f20aba&q=1&e=698f127a-baf5-4d0a-8f64-50168d8d98c6&u=https%3A%2F%2Fdiamon.org%2Fctf%2Ffiles%2FCTF2-PROP-1.0.html%5B_%2A%2ACTF2-PROP-1.0%2A%2A_]: * Written in https://protect2.fireeye.com/v1/url?k=84af5bb3-db3461f6-84af1b28-867b36d1634c-fd327bcde7e315c5&q=1&e=698f127a-baf5-4d0a-8f64-50168d8d98c6&u=https%3A%2F%2Fasciidoctor.org%2F%5BAsciidoctor%5D%27s AsciiDoc syntax. + The original AsciiDoc project is pretty much deprecated. + Asciidoctor is also what GitHub, GitLab, and more use to render AsciiDoc documents. * Terminology update. + See the <>. * Mostly written from the consumer's perspective, especially the <>. * Metadata types are removed: the document uses JSON directly now. + This makes the specification easier to read, removing one nonessential abstraction layer. * Constant integer JSON object is removed: we can use plain JSON integers. * Splitting the specification into three layers is removed: the document specifies the whole format monolithically. + This makes the specification easier to read. * **Format simplifications**: ** <>'s default/native byte order property is removed: a {fl-ba-fc} {must} have an explicit byte order property. ** Not all <> have an alignment property: only a {fl-ba-fc}, and all the field classes which inherit it, need one. + To remain backward compatible with {ctf1}, a {struct-fc} still needs a _minimum_ alignment property. ** Relative field location (field path in https://protect2.fireeye.com/v1/url?k=25738848-7ae8b20d-2573c8d3-867b36d1634c-956299cfe47af8a5&q=1&e=698f127a-baf5-4d0a-8f64-50168d8d98c6&u=https%3A%2F%2Fdiamon.org%2Fctf%2Ffiles%2FCTF2-PROP-1.0.html%5B_%2A%2ACTF2-PROP-1.0%2A%2A_]) is removed. + You can always use absolute <>. ** Field class aliases are removed. ** "`Tags`" replaced with <> attached to <>. + It makes sense for a field class to have a role now that each field class is unique (no field class aliases). ** A <> {may} have a single <>. Therefore, a <> {may} have a single, default <>. + This is enough to satisfy the current use cases. ** Null, variable-length boolean, and union field classes are removed: we don't have use cases for them. * {c-opt-fc} added. + This is why https://protect2.fireeye.com/v1/url?k=e8634e5c-b7f87419-e8630ec7-867b36d1634c-f650031c27b0f5b1&q=1&e=698f127a-baf5-4d0a-8f64-50168d8d98c6&u=https%3A%2F%2Fdiamon.org%2Fctf%2Ffiles%2FCTF2-PROP-1.0.html%5B_%2A%2ACTF2-PROP-1.0%2A%2A_] introduced the null field class (removed in {doc-id}): in the end, a {var-fc} with a null field class option and another option really is an optional field class. * <> added. + A BLOB field contains a https://en.wikipedia.org/wiki/Binary_large_object[BLOB]. + Having dedicated static- and dynamic-length BLOB field classes instead of using static- and dynamic-length <> makes it possible for consumer APIs to provide the values of such fields as byte buffers. * An <> {may} have a preferred display base, even if it's insignificant to <>, because this property exists in {ctf1} and makes the transition easier. * An <> {may} have a name and other properties which exist in {ctf1-nl}, but which are insignificant to <>, because it makes the transition easier. * <> feature added. + This is how we plan to add core features and modify existing ones in the future without having to rely on a minor version. * <> feature added. == CTF publication authority The {diamon}, a https://www.linuxfoundation.org/[Linux Foundation] workgroup which creates de facto open standards and tools for tracing, monitoring, and diagnostics, oversees the Common Trace Format design and the publication of official CTF documents. The DiaMon Workgroup is also responsible for providing a platform for interested parties to comment on CTF-related proposals. == {ctf2} document identification We suggest that all official {ctf2} documents bear a unique **document identifier** (ID) having the following format: [verse] `CTF2-__SHORTNAME__-__MAJOR__.__MINOR__[r__REV__]` [%header%autowidth] .Descriptions and roles of {ctf2} document ID parts |=== |Part |Description |Bump {may} introduce new concepts, procedures, and formats? |Bump {may} remove or change existing concepts, procedures, and formats? |`__SHORTNAME__` |The capitalized short name of the document, unique amongst all the official {ctf2} documents. |N/A |N/A |`__MAJOR__` |The major version number of the document. |Yes |Yes |`__MINOR__` |The minor version number of the document. |Yes |No |`__REV__` |The revision letter of the document (from `A` to `Z`, then `ZA` to `ZZ`, and so on). We'll use document revisions to add examples, clarify existing concepts, or fix grammar/content mistakes, for example. |No |No |=== As an example, the short name of this document is _PROP_, for _proposal_, and its full document ID is {doc-id}. The next revision would be _**CTF2-PROP-2.0rA**_, and the following would be _**CTF2-PROP-2.0rB**_. A {ctf2} document {must} refer to another {ctf2} document using only its ID, for example: > This concept is further explained in _**CTF2-SOMEID-1.2**_. There's no need to refer to a specific revision: a reference always targets the latest document's revision. We suggest the following IDs for the initial documents: [%header,cols="d,d"] |=== |{ctf2} document ID |Description |_**CTF2-DOCID-1.0**_ |{ctf2} document identifier format. |_**CTF2-FS-2.0**_ |Layout of a {ctf2} trace when stored on a file system. |_**CTF2-PMETA-2.0**_ |{ctf2} metadata stream packet format. |_**CTF2-SPEC-2.0**_ |Common Trace Format (CTF), version 2. We don't plan to ever bump this document's minor version as we can use the proposed <> mechanism to add or modify core features. |=== == Why {ctf2}? Why do we need a major version bump of the CTF specification? A major version bump is never an easy choice when it comes to revisiting a software library, a communication protocol, a file format, or anything else that serves as a contract between a producer and a consumer. When such a decision is taken, it must be justified by solid arguments, since it makes it impossible for old consumers to consume the product of new producers. In this proposal, for instance, {ctf2} traces are _not_ backward compatible with {ctf1} traces. Although the <> binary format is compatible, the <> is not: it's written in a different language (JSON instead of TSDL). {ctf1-nl} has been used and tested for over ten years now, by different producers and consumers. Over that time, we have noted a few gaps in the trace format, gaps that make it hard to extend {ctf1-nl} as much as we'd like, amongst other things. This specification proposition designs {ctf2} to overcome those gaps, to the best of our knowledge, and to be flexible enough to gracefully accept future additions while avoiding another major version bump in the upcoming years. [[design-goals]] === Design goals The design goals of {ctf2} are as follows, in order, beginning with the most important: . **{ctf2} data streams {must} be backward compatible with {ctf1} <>.** + Many applications are already written to output valid {ctf1-nl} <>. Modifying the code of those applications to produce different binary packets can be cumbersome, and sometimes impossible if the application passed acceptance tests, for example. + Making sure that applications producing {ctf1-nl} traces can also produce {ctf2} traces only by changing the <> is therefore a hard requirement. . **The {ctf2} data streams {must} be as efficient as possible to produce by a tracer.** + This design goal was also one of the major ones which drove the design of {ctf1-nl}. + In other words, a small embedded system {must} be able to produce {ctf2} data streams natively. Moreover, the tracer {must} be able to copy binary data to the packet buffer of a data stream without altering it. . **A {ctf2} metadata stream {must} be extensible by users (including the {diamon}) of the specification.** + {ctf1-nl}'s TSDL grammar is pretty restrictive when it comes to customizing existing blocks with user-defined attributes. + Many protocols and declarative languages support custom user data in their payload. For example, HTML5 allows any element to have user attributes with the `data-` prefix. + A {ctf2} producer {must} be able to add custom attributes to almost any specified metadata object. This makes it possible for standard consumers to read any {ctf2} trace and ignore unknown user attributes, providing a "`bland`", yet complete view of the trace fields, while you can write a special consumer (or extend an existing one) to interpret specific user attributes and use them to render a meaningful visualization. + This design goal also means that an "`old`" {ctf2} consumer {must} be able to either: + ** Decode a "`new`" {ctf2} trace completely. ** Indicate that it cannot decode a "`new`" {ctf2} trace completely. . **{ctf2}'s specification {must} focus on use cases known on its publication date.** + The previous design goal indicates that a {ctf2} metadata stream {must} be extensible by users. Knowing this, the {ctf2} specification itself {must} focus on _currently known_ use cases, leaving anything else to future extensions to be described in other {ctf2} documents. + As per design goal{nbsp}1, those known use cases include everything available in {ctf1-nl}. . **{ctf2}'s specification {must-not} specify how to transport or store a trace.** + In the {ctf2} specification, a {ctf2} trace {must} be defined as a set of streams, without specifying how you transport or store said streams. + Other official {ctf2} documents published by the {diamon} can define standard ways to transport and store {ctf2} streams for targeted use cases. Trace producers and consumers can choose to implement one or more transport/storage strategies following the other documents. . **A {ctf2} trace {should} be as easy as possible to consume.** + {ctf1-nl} focuses on being easy to produce, which is a good idea since producers are often tracers in this context, and as per design goal{nbsp}2, a minimal tracer {must} be able to produce a correct CTF trace with minimal code. + However, because a {ctf1-nl} <> is written in TSDL, a custom, declarative, C-like DSL designed for CTF, writing a minimal consumer of {ctf1-nl} is not an easy task. TSDL is an intricate language, with many special cases and features, many of which are borrowed from the C language, which you can't ignore when writing a consumer supporting all its features. TSDL was developed to ease the _manual_ (human) production of metadata streams. + Over time, we realized that, while producing traces is important, consuming them to solve problems with <> analysis is just as important, if not more. + This is why {ctf2} {should} encourage the development of CTF consumers in any programming language by reducing the number of special cases, as well as by using a very simple, yet well-known descriptive language for the <>. + {ctf1-nl} tries to accomodate other trace formats, which can be converted to CTF without changing the data streams by writing the matching metadata stream. This is also a source of special cases. {ctf2} {should} build on binary trace conversion (from another, non-CTF trace format to {ctf2}) rather than trying to accomodate other formats. . **{ctf2}'s model {should} be similar to {ctf1-nl}'s.** + Some APIs are already written to deal with {ctf1-nl} "`objects`", or concepts (<>, <>, <>, and <>, for example). + The model of {ctf2} {should} be, as much as possible, compatible with the model of {ctf1-nl}, so that those existing APIs can operate on {ctf2} objects too without requiring significant upgrades. == Changes since {ctf1-nl} Here is a brief summary of the changes, from {ctf1} to {ctf2}, as introduced by this proposal. * The terminology of the specification and the binary layouts of the <> are _completely detached_ from the C language and from the behaviour of any C compiler. + {ctf2} is a programming language-agnostic trace format. * [[term-update]]**Terminology update**: + [%header%autowidth] |=== |{ctf1-nl} term |Equivalent {ctf2} term |Absolute (clock class property) |Origin is Unix epoch |Array (field class) |{c-sl-array} |Base (integer field class property) |Preferred display base |Binary stream |Data stream |Clock (when it names a block of metadata which describes actual clocks) |Clock class |Content size |Packet's content size |Declaration |Field class |Enumeration (field class) |{c-fl-enum} |Event (when it names a block of metadata which describes event records) |Event record class |Event (when it names an actual recorded event contained in a packet) |Event record |Event context |Event record specific context |Event fields |Event record payload |Event ID |Event record class ID |Events discarded |Discarded event record counter |Field (structure field class) |Structure field member class |Field (structure field) |Structure field member |Floating point (field class) |{c-fl-fp} |Integer (field class) |{c-fl-int} |Packet size |Packet's total size |Sequence (field class) |{c-dl-array} |Size (integer field class property) |Length |Stream (when it names a block of metadata which describes data streams) |Data stream class |Stream event context |Event record common context |Stream event header |Event record header |Stream ID |Data stream class ID |Stream instance ID |Data stream ID |Stream packet context |Packet context |String (field class) |{c-str} |Tag (variant field class) |Selector |Trace (when it names a block of metadata which describes traces) |Trace class |Trace packet header |Packet header |=== * The <> is written in JSON, as specified by https://protect2.fireeye.com/v1/url?k=3e8816cf-61132c8a-3e885654-867b36d1634c-bbf4be023e44dd4b&q=1&e=698f127a-baf5-4d0a-8f64-50168d8d98c6&u=https%3A%2F%2Fwww.ecma-international.org%2Fpublications%2Ffiles%2FECMA-ST%2FECMA-404.pdf%5BECMA-404]. * The {ctf2} specification doesn't specify a "`packetized`" metadata stream format. + This is a back-end-specific way of wrapping a <> as defined in this document. + Another document specifies the packetized metadata stream format, which trace storage and transport documents can use if needed. * Most objects of a {ctf2} metadata stream {may} contain <>. * Field class aliases are removed. + This removes indirection, simplifying the format. * **New field classes**: ** A __{fl-ba-fc}__ describes the most primitive fields ({fl-ba} fields) of a <> which actually have values. + A {fl-int-fc} is now specified as a {fl-ba} field class with an {optional} preferred display base property and integer type semantics. + A {fl-enum-fc} is now specified as a {fl-int-fc} with mappings. + A {fl-fp-fc} is now specified as a {fl-ba-fc}: its length property is enough to determine its encoding under {ieee754}. ** A __{fl-bool-fc}__, which is a {fl-ba-fc} with a special meaning, describes {fl-bool} fields. + When all the bits of a {fl-bool} field are cleared (zero), the field's value is said to be _false_. Otherwise, the field's value is said to be _true_. ** A __{vl-ba-fc}__ describes {vl-ba} fields. + Each byte of a {vl-ba} field has its most significant bit set if one more byte is part of the field. + The seven low-order bits of each byte are concatenated in a specific way to form a final, equivalent {fl-ba} field. ** A __{vl-int-fc}__, which is to a {vl-ba-fc} what a {fl-int-fc} is to a {fl-ba-fc}, describes {vl-int} fields. ** A __{vl-enum-fc}__, which is to a {vl-int-fc} what a {fl-enum-fc} is to a {fl-int-fc}, describes {vl-enum} fields. ** A __{sl-str-fc}__ and a __{dl-str-fc}__ describe static- and dynamic-length string fields, which {may} not be null-terminated. + Such string fields hold _possibly_ null-terminated UTF-8 string values (they {may} also hold an exact number of UTF-8 bytes without a terminating null byte). ** An __{opt-fc}__ describes an optional field. + It is similar to a variant field class with two options: the optional field class and an "`empty`" field class. ** A __{sl-blob-fc}__ and a __{dl-blob-fc}__ describe static- and dynamic-length sequences of contiguous bytes to represent https://en.wikipedia.org/wiki/Binary_large_object[BLOBs]. * **Modified field classes**: ** The default alignment property of a <> field (and therefore also of fixed-length <>, <>, <>, and <> fields) is{nbsp}1. + In {ctf1-nl}, it's{nbsp}8 if the field's class's size property is a multiple of{nbsp}8, and{nbsp}1 otherwise. ** The _encoding_ property _is removed_ from the {fl-int-fc}. + The encoding property in {ctf1-nl} indicates that a byte is a UTF-8 character, for example, but since there are UTF-8 characters which are encoded on more than one byte, this property actually makes no sense at the {fl-int-fc} level. + You can use the new {sl-str-fc} and {dl-str-fc} to achieve the same result. ** The _exponent_ and _mantissa size_ properties _are removed_ from the {fl-fp-fc}. + A {fl-fp-fc} is now defined as a {fl-ba-fc}, which has a single length property to indicate its total, fixed length, in bits. + A {ctf2} consumer can <> encoded following {ieee754} knowing only this parameter--the storage width of the bit array--from which it can deduce other parameters according to the standard. + In other words, in {ctf1-nl}, only specific pairs of exponent and mantissa size properties are valid: 8 and 24, 11 and 53, 15 and 113, and so on. ** The _encoding_ property _is removed_ from the {str-fc}. + {ctf2} {str} fields always contain a sequence of bytes which encode a {str} with UTF-8. If a {str} field is encoded with the ASCII encoding in a {ctf1-nl} data stream, it's still valid in {ctf2} since an ASCII string is a UTF-8 string. * _Relative_ field locations (variant field class's tag/selector and dynamic-length field class's length) are removed. + In {ctf2}, you can specify a location to any valid field with an absolute <>. + This simplifies the format. * "`Special`" <>, such as the <> magic number member class, the <> ID member class, and the packet's total size member class, _can have any name_: a <> {may} have one or more <> instead to indicate the purpose of its instances. + This means a {ctf2} consumer doesn't need to rely on reserved names like {ctf1-nl}'s `magic`, `stream_id`, and `packet_size`. + This new approach also makes it possible for a given field class to have multiple roles. * The _native byte order_ property _is removed_ from the <>. + In {ctf2}, each {fl-ba-fc} {must} explicitly indicate the byte order of its instances. + This removes indirection, simplifying the format. * The _EMF URI_ property _is removed_ from the <>. + This Eclipse Modeling Framework property is not common enough for the Common Trace Format. + A producer can still write such an URI as an event record class's <>. * The _log level_ property _is removed_ from the <>. + Log level integral values are tracer-specific. + A producer can still write such a log level value as an event record class's <>. * A {ctf2} trace {may} have zero or more <>. + An auxiliary stream contains structured information related to a specific trace which doesn't fit the <> model, whereas a {ctf2} metadata stream now describes a class of traces (multiple traces can share the same metadata stream). * The "`call site`" block feature is removed. + We don't know any producer using this. == Common definitions Common definitions for this specification: Byte:: A group of eight bits operated on as a unit. Class:: A set of values (instances) which share common properties. + For example, a {fl-uint-fc} with an 8-bit length is the set of the all the {fl-uint} fields from `00000000` to `11111111` (values 0{nbsp}to{nbsp}255). . + This document often states that some class _describes_ instances. For example, an <> describes <>. Consumer:: A software or hardware system which consumes (reads) the streams of a <>. + A trace consumer is often a _trace viewer_ or a _trace analyzer_. [[ns-def]]Namespace:: A string of which the purpose is to avoid naming conflicts. + This document doesn't specify the format of a namespace. It is _recommended_ to use a URI, or at least to include a domain name owned by the organization defining the objects under a namespace. + IMPORTANT: The `std` namespace is reserved for the {ctf2} specification. Producer:: A software or hardware system which produces (writes) the streams of a <>. + A trace producer is often a _tracer_. Sequence:: A set of related items that follow each other in a particular order. Stream:: A sequence of bytes. [[trace]] == Trace composition A trace is: * One <>. * One or more <>. * Zero or more <>. As a reminder, a stream is defined as a sequence of bytes. NOTE: This document doesn't specify how to transport or store {ctf2} streams. A producer could serialize all streams as a single file on the file system, or it could send the streams over the network using TCP, to name a few examples. [[metadata-stream-overview]] === Metadata stream (overview) A metadata stream describes trace <> with JSON objects. A metadata stream describes things such as: * The <> of the data stream <>. * The names of <>. * The <> of event record fields. Multiple traces {may} share the same metadata stream: a given trace {may} contain specific information in its own <>. See <> for the full metadata stream specification. [[ds]] === Data stream A _data stream_ is a sequence of one or more data <>: image::ctf-trace-all.svg[] In the <>, a <> describes data streams. A packet {must} contain one or more bytes of data. Although a packet {may} contain padding at the end itself, from the data stream's point of view, there's no padding between packets. In other words, the byte following the last byte of a packet is the first byte of the next packet. A data stream {may} have, conceptually: [[def-clk]]One default, monotonic clock:: Described by a <> in the metadata stream. + <> and <> {may} contain _snapshots_, named _timestamps_, of their data stream's default clock. [[disc-er-counter]]One counter of discarded event records:: Indicates the number of event records which the producer needed to discard for different reasons. + For example, a tracer could discard an event record when it doesn't fit some buffer and there's no other available buffer. + A packet {may} contain a snapshot of this counter. See <> to learn how to decode a {ctf2} data stream. [[pkt]] ==== Packet A _packet_ is a part of a <>. A packet contains a sequence of data _fields_ or padding (garbage data). In the metadata stream, <> describe data fields. A packet {var-p}, contained in a data stream{nbsp}{var-s}, contains, in this order: . [[pkt-header]]{optional}: A **header** <> field, described at the <> level in the <>, which contains: .. {optional}: A packet magic number field (0xc1fc1fc1, or 3254525889). .. Order _not_ significant: *** {optional}: A trace class UUID field. *** {optional}: One or more fields which contain the numeric ID of the <> of{nbsp}{var-s}. *** {optional}: One or more fields which contain the numeric ID of{nbsp}{var-s}. . [[pkt-ctx]]{optional}: A **context** <> field, described at the <> level in the metadata stream, which contains (order _not_ significant): ** {optional}: A field which contains the total size of{nbsp}{var-p}, in bits (always a multiple of 8). ** {optional}: A field which contains the content size of{nbsp}{var-p}, in bits. ** {optional}: A field which contains the beginning timestamp of{nbsp}{var-p}. ** {optional}: A field which contains the end timestamp of{nbsp}{var-p}. ** {optional}: A field which contains a snapshot of the <> of{nbsp}{var-s} at the end of{nbsp}{var-p}. ** {optional}: A field which contains the sequence number of{nbsp}{var-p} within{nbsp}{var-s}. ** {optional}: User fields. . Zero or more <>. A packet {must} contain one or more bytes of data. A packet {may} have padding (garbage data) after its _last_ event record. The size of this padding is the difference between its total size and its content size (as found in its <>). Packets are independent of each other: if you remove a packet from a data stream, a consumer can still decode the whole data stream. This is why: * Packets {may} contain _snapshots_ of their data stream's <>. * Packets and event records {may} contain _timestamps_ which are snapshots of their data stream's <>. If the <> fields of a data stream's packets contain a <> field, a consumer can recognize missing packets. See <> to learn how to decode a {ctf2} packet. [[er]] ==== Event record An _event record_ is the result of a producer writing a record with {optional} user data when an event occurs during its execution. A <> contains zero or more event records. An <> describes the specific parts of event records. An event record _**E**_, contained in a <>{nbsp}{var-s}, contains, in this order: . [[er-header]]{optional}: A **header** <> field, described at the <> level in the metadata stream, which contains (order _not_ significant): ** {optional}: One or more fields which contain the numeric ID of the <> of{nbsp}__**E**__ which has the class of{nbsp}{var-s} as its parent. ** {optional}: One or more fields which contain a timestamp or a partial timestamp. . [[er-common-ctx]]{optional}: A **common context** <> field, described at the data stream class level in the metadata stream, which contains user fields. . [[er-spec-ctx]]{optional}: A **specific context** <> field, described at the event record class level in the metadata stream, which contains user fields. . [[er-payload]]{optional}: A **payload** <> field, described at the event record class level in the metadata stream, which contains user fields. An event record {must} contain one or more bits of data. The <> timestamp of an event record, that is, the value of its <>'s default clock _after_ its <>, if any, is encoded/decoded {must} be greater than or equal to the default clock timestamp of the previous event record, if any, within the _same_ data stream. See <> to learn how to decode a {ctf2} event record. [[aux-stream]] === Auxiliary stream An auxiliary stream is a JSON object, as specified by https://protect2.fireeye.com/v1/url?k=af4f3d31-f0d40774-af4f7daa-867b36d1634c-fd4033714bb458fb&q=1&e=698f127a-baf5-4d0a-8f64-50168d8d98c6&u=https%3A%2F%2Fwww.ecma-international.org%2Fpublications%2Ffiles%2FECMA-ST%2FECMA-404.pdf%5BECMA-404], which contains extra, structured information about the trace which doesn't fit the <> model. An auxiliary stream has a single property: [horizontal] Name:: Auxiliary stream's <>. Value:: A JSON value. Two auxiliary streams of a given trace {must-not} have the same namespace. .Auxiliary stream with the `my.tracer` namespace. ==== [source,json] ---- { "my.tracer": { "version": [1, 3, 2], "session-name": "hanon" } } ---- ==== .Auxiliary stream of which the value is just `42`. ==== [source,json] ---- { "328c7a2d-a959-4f60-bd22-cca74359326f": 42 } ---- ==== [[env]] ==== Trace environment To remain backward compatible with {ctf1}, a trace {may} contain an auxiliary stream having the `std` namespace which contains trace environment variables under the `environment` property. The trace environment variables are a single JSON object where each property is: [horizontal] Name:: Trace environment variable name. Value:: Trace environment variable value (any JSON value). This document doesn't specify trace environment variable names. .`std` auxiliary stream with trace environment variables. ==== [source,json] ---- { "std": { "environment": { "hostname": "hanon", "domain": "kernel", "sysname": "Linux", "kernel_release": "4.12.12-1-ARCH", "kernel_version": "#1 SMP PREEMPT Sun Sep 10 09:41:14 CEST 2017", "tracer_name": "lttng-modules", "tracer_major": 2, "tracer_minor": 10, "tracer_patchlevel": 0 } } } ---- ==== [[metadata-stream]] == Metadata stream A metadata stream is a JSON array, as specified by https://protect2.fireeye.com/v1/url?k=92ff4d35-cd647770-92ff0dae-867b36d1634c-404c4d62482533d6&q=1&e=698f127a-baf5-4d0a-8f64-50168d8d98c6&u=https%3A%2F%2Fwww.ecma-international.org%2Fpublications%2Ffiles%2FECMA-ST%2FECMA-404.pdf%5BECMA-404], of _fragments_. Together, the fragments of a metadata stream contain all the information about the <> of one or more <>. [[frag]]A _fragment_ is a JSON object; its allowed properties depend on its `type` property. .Common properties of a fragment {var-f}. [%header%autowidth,cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be one of: `"preamble"`:: {var-f} is a <>. `"trace-class"`:: {var-f} is a <>. `"clock-class"`:: {var-f} is a <>. `"data-stream-class"`:: {var-f} is a <>. `"event-record-class"`:: {var-f} is a <>. |Yes | |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. For any fragment except a <>, any extension which exists under this property must also be declared in the metadata stream's preamble fragment. |No |`+{}+` |=== The metadata stream is designed as a flat JSON array of fragments instead of a single JSON object containing nested objects to enable real-time, or "`live`", tracing: a consumer can always decode <> having known <> while a producer can always add new event record classes to a <> by appending additional fragments to the metadata stream. Once a producer appends a fragment to a metadata stream, the fragment is considered "`frozen`", in that it never needs to change. A metadata stream: * {must} start with a preamble fragment. * {must} contain exactly one <>. * {may} contain one <>. * {must} contain one or more <> which {must} follow the trace class fragment, if any. * {may} contain one or more <> which {must} follow their parent data stream class, if any. .Partial metadata stream. ==== [source,json] ---- [ { "type": "preamble", "version": 2 }, … ] ---- ==== [NOTE] ==== This section doesn't specify how a metadata stream translates into <> encoding and decoding rules; it only describes objects and their properties. See <> to learn how to decode a data stream. ==== [[uuid]] === UUID Both a <> and a <> {may} have a https://en.wikipedia.org/wiki/Universally_unique_identifier[_UUID_] property. Within a metadata stream, a UUID is a JSON string of which the value is the canonical textual representation of the UUID. .UUID. ==== [source,json] ---- "e53e0ab8-50a1-4f0a-b710-b5f0bba9c4ac" ---- ==== [[ext]] === Extensions A producer {may} add _extensions_ to many metadata stream JSON objects. The purpose of an extension is to add core features to {ctf2} or to modify existing core features, as specified by this document. In other words, an extension {may} alter the format itself. This document doesn't specify what an extension exactly is. The metadata stream's <> contains _extension declarations_: * Any extension in metadata stream objects {must} be declared, by namespace and name, in the preamble fragment. + Declaring an extension is said to _enable_ it. * If a consumer doesn't support _any_ declared extension, it {must-not} consume the <>'s <>. + The consumer {should} report unsupported extensions as an error. Extensions are a single JSON object, where each property is: [horizontal] Name:: A <> Value:: A <> [[ns-exts-obj]]A _namespaced extensions object_ is a JSON object, where each property is: [horizontal] Name:: An extension name Value:: A JSON value The metadata stream JSON objects which {may} contain extensions as their `extensions` property are: * Any <>. + An extension in the <> also makes it _declared_. * Any <>. * A <>. * A <>. .Three extensions under two namespaces. ==== [source,json] ---- { "my.tracer": { "piano": { "keys": 88, "temperament": "equal" }, "ramen": 23 }, "abc/xyz": { "sax": { "variant": "alto" } } } ---- ==== [[user-attrs]] === User attributes A producer {may} add custom _user attributes_ to many metadata stream JSON objects. This document doesn't specify what a user attribute exactly is. Unlike <>, a consumer {must-not} consider user attributes to decode <>. User attributes are a single JSON object, where each property is: [horizontal] Name:: A <> Value:: A JSON value The metadata stream JSON objects which {may} contain user attributes as their `user-attributes` property are: * Any <>. * Any <>. * A <>. * A <>. .User attributes under two namespaces. ==== [source,json] ---- { "my.tracer": { "max-count": 45, "module": "sys" }, "abc/xyz": true } ---- ==== [[fc]] === Field classes A _field class_ describes fields, that is, sequences of bits as found in a <>. A field class contains all the properties a consumer needs to <> a given field. A _field_ is a field class instance. This document specifies the following types of field classes: Abstract field classes:: You cannot use the following field classes directly: they are bases for other, concrete field classes: + * <> * <> * <> * <> Fixed/static-length field classes:: + * {c-fl-ba-fc} * {c-fl-bool-fc} * {c-fl-int-fc} * {c-fl-enum-fc} * {c-fl-fp-fc} * {c-sl-str-fc} * {c-sl-blob-fc} Variable/dynamic-length field classes:: + * {c-vl-ba-fc} * {c-vl-int-fc} * {c-vl-enum-fc} * {c-str-fc} * {c-dl-str-fc} * {c-dl-blob-fc} Compound field classes:: The following field classes contain one or more field classes. + * {c-struct-fc} * {c-sl-array-fc} * {c-dl-array-fc} * {c-opt-fc} * {c-var-fc} A field class is a JSON object; its properties depend on its `type` property. .Common properties of a field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be one of: `"fixed-length-bit-array"`:: {var-f} is a {fl-ba-fc}. `"fixed-length-boolean"`:: {var-f} is a {fl-bool-fc}. `"fixed-length-unsigned-integer"`:: `"fixed-length-signed-integer"`:: {var-f} is a {fl-int-fc}. `"fixed-length-unsigned-enumeration"`:: `"fixed-length-signed-enumeration"`:: {var-f} is a {fl-enum-fc}. `"fixed-length-floating-point-number"`:: {var-f} is a {fl-fp-fc}. `"variable-length-bit-array"`:: {var-f} is a {vl-ba-fc}. `"variable-length-unsigned-integer"`:: `"variable-length-signed-integer"`:: {var-f} is a {vl-int-fc}. `"variable-length-unsigned-enumeration"`:: `"variable-length-signed-enumeration"`:: {var-f} is a {vl-enum-fc}. `"null-terminated-string"`:: {var-f} is a {str-fc}. `"static-length-string"`:: {var-f} is a {sl-str-fc}. `"static-length-blob"`:: {var-f} is a {sl-blob-fc}. `"dynamic-length-string"`:: {var-f} is a {dl-str-fc}. `"dynamic-length-blob"`:: {var-f} is a {dl-blob-fc}. `"structure"`:: {var-f} is a {struct-fc}. `"static-length-array"`:: {var-f} is a {sl-array-fc}. `"dynamic-length-array"`:: {var-f} is a {dl-array-fc}. `"optional"`:: {var-f} is a {opt-fc}. `"variant"`:: {var-f} is a {var-fc}. |Yes | |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== The following <> properties require a {struct-fc} as their value: <>:: `packet-header-field-class` <>:: + * `packet-context-field-class` * `event-record-header-field-class` * `event-record-common-context-field-class` <>:: + * `specific-context-field-class` * `payload-field-class` [[field-loc]] ==== Field location A _field location_ is a means for a consumer to find a field which it needs to decode another field. A consumer needs to find another field to decode instances of the following <>: {c-dl-array-fc}:: {c-dl-str-fc}:: {c-dl-blob-fc}:: Needs a <> or <> length field. {c-opt-fc}:: Needs a <>, <>, or <> selector field. {c-var-fc}:: Needs a <> or <> selector field. A field location is a JSON array, where, in this order: . The first element is the name (JSON string) of a root field from where to start the lookup, amongst: + [horizontal] `"packet-header"`::: Packet header `"packet-context"`::: Packet context `"event-record-header"`::: Event record header `"event-record-common-context"`::: Event record common context `"event-record-specific-context"`::: Event record specific context `"event-record-payload"`::: Event record payload . The following elements are <> field member names (JSON strings) to follow to find the target field. The length of a field location {must} be greater than or equal to two. Let _**T**_ be a field which a consumer needs to decode another field{nbsp}{var-s}: * If {var-s} is in a packet header, then{nbsp}__**T**__ {must} be in the _same_ packet header. * If {var-s} is in a packet context, then{nbsp}__**T**__ {must} be in one of: ** The packet header the _same_ packet. ** The _same_ packet context. * If {var-s} is in an event record header, then{nbsp}__**T**__ {must} be in one of: ** The packet header of the _same_ packet. ** The packet context of the _same_ packet. ** The _same_ event record header. * If {var-s} is in an event record common context, then{nbsp}__**T**__ {must} be in one of: ** The packet header of the _same_ packet. ** The packet context of the _same_ packet. ** The event record header of the _same_ event record. ** The _same_ event record common context. * If {var-s} is in an event record specific context, then{nbsp}__**T**__ {must} be in one of: ** The packet header of the _same_ packet. ** The packet context of the _same_ packet. ** The event record header of the _same_ event record. ** The event record common context of the _same_ event record. ** The _same_ event record specific context. * If {var-s} is in an event record payload, then{nbsp}__**T**__ {must} be in one of: ** The packet header of the _same_ packet. ** The packet context of the _same_ packet. ** The event record header of the _same_ event record. ** The event record common context of the _same_ event record. ** The event record common specific of the _same_ event record. ** The _same_ event record payload. * If {var-s} and _**T**_ are _not_ in the same root field, then{nbsp}__**T**__ {must-not} be in any <> or <> field. * If {var-s} and _**T**_ are in the same root field, then: ** If {var-s} is in an array field, then{nbsp}__**T**__ must be in the _same_ array field element. ** If {var-s} is in an optional field, then{nbsp}__**T**__ must be in the _same_ optional field. If any structure member name{nbsp}__**N**__ of a field location{nbsp}__**L**__ names a <>, then: If _**N**_ is _not_ the last element of{nbsp}__**L**__:: The variant field {must} select a structure field, from which the lookup process can continue, recursively. If _**N**_ is the last element of{nbsp}__**L**__:: The variant field {must} select the target field (<>, <>, or <>), recursively. In both cases, _all_ the options of the variant field class {must} make it possible for the lookup process to continue. .<> field and its length field in the same root field. ==== The following JSON object is an event record payload {struct-fc}. [source,json] ---- { "type": "structure", "members": [ { "name": "corn", <3> "field-class": { "type": "fixed-length-unsigned-integer", "length": 32, "byte-order": "little-endian" } }, { "name": "inside", "field-class": { "type": "fixed-length-unsigned-integer", "length": 16, "byte-order": "little-endian" } }, { "name": "carbon", "field-class": { "type": "dynamic-length-array", <1> "length-field-location": ["event-record-payload", "corn"], <2> "element-field-class": { "type": "null-terminated-string" } } } ] } ---- <1> {c-dl-array-fc}. <2> {c-dl-array-fc}'s length field location. <3> Length member class. ==== .{c-dl-array} field and its length field in the same root field, within the same array field element. ==== The following JSON object is an event record payload {struct-fc}. Both the {dl-array} field and its length field exist within the same element of the <> field named `nature`. [source,json] ---- { "type": "structure", "members": [ { "name": "norm", "field-class": { "type": "null-terminated-string" } }, { "name": "nature", "field-class": { "type": "static-length-array", "length": 43, "element-field-class": { "type": "structure", "members": [ { "name": "laser", <3> "field-class": { "type": "variable-length-unsigned-integer" } }, { "name": "joystick", "field-class": { "type": "dynamic-length-array", <1> "length-field-location": [ <2> "event-record-payload", "nature", "laser" ], "element-field-class": { "type": "null-terminated-string" } } } ] } } } ] } ---- <1> {c-dl-array-fc}. <2> {c-dl-array-fc}'s length field location. <3> Length member class. ==== .{c-dl-array} and its length field in the same root field, within the same <> field. ==== The following JSON object is an event record payload {struct-fc}. Both the {dl-array} field and its length field exist within the same option of the <> field named `clinic`. Moreover, the selector field of the `clinic` variant field is the `lawyer` field. [source,json] ---- { "type": "structure", "members": [ { "name": "lawyer", <5> "field-class": { "type": "fixed-length-signed-integer", "length": 16, "byte-order": "little-endian" } }, { "name": "clinic", "field-class": { "type": "variant", "selector-field-location": ["event-record-payload", "lawyer"], <4> "options": [ { "selector-field-ranges": [[0, 0]], "field-class": { "type": "null-terminated-string" } }, { "selector-field-ranges": [[1, 4]], "field-class": { "type": "structure", "members": [ { "name": "lemon", <3> "field-class": { "type": "fixed-length-unsigned-integer", "length": 8, "byte-order": "big-endian" } }, { "name": "joystick", "field-class": { "type": "dynamic-length-array", <1> "length-field-location": [ <2> "event-record-payload", "clinic", "lemon" ], "element-field-class": { "type": "null-terminated-string" } } } ] } }, { "selector-field-ranges": [[5, 5], [7, 7]], "field-class": { "type": "fixed-length-boolean", "length": 8, "byte-order": "little-endian" } } ] } } ] } ---- <1> {c-dl-array-fc}. <2> {c-dl-array-fc}'s length field location. <3> Length member class. <4> Variant field class's selector field location. <5> Selector member class. ==== .{c-dl-array} and its length field in the same root field; length field is a variant field. ==== The following JSON object is an event record payload {struct-fc}. The length field of the {dl-array} field is a variant field: it can be an 8-bit, a 16-bit, or a 32-bit <> field, depending on the variant field's selection. Moreover, the selector field of the variant field is located in another root field (event record specific context). [source,json] ---- { "type": "structure", "members": [ { "name": "glass", <3> "field-class": { "type": "variant", "selector-field-location": ["event-record-specific-context", "sel"], "options": [ { "selector-field-ranges": [[0, 0]], "field-class": { "type": "fixed-length-unsigned-integer", <4> "length": 8, "byte-order": "little-endian" } }, { "selector-field-ranges": [[1, 1]], "field-class": { "type": "fixed-length-unsigned-integer", <4> "length": 16, "byte-order": "little-endian" } }, { "selector-field-ranges": [[2, 2]], "field-class": { "type": "fixed-length-unsigned-integer", <4> "length": 32, "byte-order": "little-endian" } } ] } }, { "name": "margin", "field-class": { "type": "dynamic-length-array", <1> "length-field-location": ["event-record-payload", "glass"], <2> "element-field-class": { "type": "null-terminated-string" } } } ] } ---- <1> {c-dl-array-fc}. <2> {c-dl-array-fc}'s length field location. <3> Length member class. <4> Possible length field class. ==== .{c-dl-array} and its length field in the same root field; structure field containing length field is a variant field. ==== The following JSON object is an event record payload {struct-fc}. The length field of the {dl-array} field is within a structure field which is a variant field. Moreover: * The selector field of the variant field is located in another root field (event record common context). * The field class of the third option of the `glass` variant field class contains a {dl-blob-fc} (`lock` member); the length field of its instance is the previous member (`eagle`) within the same structure field. [source,json] ---- { "type": "structure", "members": [ { "name": "glass", "field-class": { "type": "variant", "selector-field-location": ["event-record-common-context", "sel"], "options": [ { "selector-field-ranges": [[0, 0]], "field-class": { "type": "structure", "members": [ { "name": "eagle", "field-class": { "type": "fixed-length-unsigned-integer", <3> "length": 16, "byte-order": "little-endian" } }, { "name": "road", "field-class": { "type": "null-terminated-string" } } ] } }, { "selector-field-ranges": [[32, 172]], "field-class": { "type": "structure", "members": [ { "name": "nuance", "field-class": { "type": "null-terminated-string" } }, { "name": "eagle", "field-class": { "type": "fixed-length-unsigned-integer", <3> "length": 24, "byte-order": "big-endian" } } ] } }, { "selector-field-ranges": [[5, 5]], "field-class": { "type": "structure", "members": [ { "name": "eagle", <5> "field-class": { "type": "variable-length-unsigned-integer" <3> } }, { "name": "lock", "field-class": { "type": "dynamic-length-blob", "length-field-location": [ <4> "event-record-payload", "glass", "eagle" ] } } ] } } ] } }, { "name": "margin", "field-class": { "type": "dynamic-length-array", <1> "length-field-location": [ <2> "event-record-payload", "glass", "eagle" ], "element-field-class": { "type": "null-terminated-string" } } } ] } ---- <1> {c-dl-array-fc}. <2> {c-dl-array-fc}'s length field location. <3> Possible length field class. <4> {c-dl-blob-fc}'s length field location. <5> Length field class for {dl-blob-fc}. Note that both the {dl-array} and {dl-blob} field classes have the same length field location. ==== .{c-dl-array} and its length field in another root field. ==== The following JSON objects are the event record specific context and payload <> of the same <>. The length field of the event record payload's {dl-array} field is within the event record specific context. .Event record specific context field class. [source,json] ---- { "type": "structure", "members": [ { "name": "cook", "field-class": { "type": "fixed-length-floating-point-number", "length": 64, "byte-order": "little-endian" } }, { "name": "vegetable", <1> "field-class": { "type": "variable-length-unsigned-integer" } } ] } ---- <1> Length member class. .Event record payload field class. [source,json] ---- { "type": "structure", "members": [ { "name": "avenue", "field-class": { "type": "dynamic-length-array", <1> "length-field-location": [ <2> "event-record-specific-context", "vegetable" ], "element-field-class": { "type": "null-terminated-string" } } }, { "name": "railroad", "field-class": { "type": "null-terminated-string" } } ] } ---- <1> {c-dl-array-fc}. <2> {c-dl-array-fc}'s length field location. ==== [[int-range-set]] ==== Integer range set An _integer range set_ is a JSON array of integer ranges. An integer range set {must} contain one or more integer ranges. An _integer range_ is a JSON array of two elements: . The range's lower bound (JSON integer, included). . The range's upper bound (JSON integer, included). An integer range represents all the integer values from the range's lower bound to its upper bound. An integer range's upper bound {must} be greater than or equal to its lower bound. If both the lower and upper bounds of an integer range are equal, then the integer range represents a single integer value. .Integer ranges. ==== [source,json] ---- [3, 67] ---- [source,json] ---- [-45, 101] ---- .Single integer value. [source,json] ---- [42, 42] ---- ==== .Integer range set containing three integer ranges. ==== [source,json] ---- [[3, 67], [-45, 1], [42, 42]] ---- ==== [[roles]] ==== Roles Some <> instances can have _roles_. A role is specific semantics attached to the fields (instances) of a field class. For example, the `packet-magic-number` role of a {fl-int-fc} indicates that the value of its instances {must} be the <> magic number (0xc1fc1fc1). Roles are a JSON array of role names (JSON strings). See <> and <> which indicate accepted roles for their root field classes. [[fl-ba-fc]] ==== {c-fl-ba} field class A _{fl-ba}_ field class describes _{fl-ba}_ fields. A {fl-ba} field is a simple array of contiguous bits, without any attached integer type semantics. The length, or number of bits, of a {fl-ba} field is a property (`length`) of its class. A {fl-ba} field class acts as a base of a {fl-bool-fc}, a {fl-int-fc}, and a {fl-fp-fc}. .Common properties of a {fl-ba} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"fixed-length-bit-array"`. |Yes | |`length` |JSON integer |Number of bits of an instance of{nbsp}{var-f}. The value of this property {must} be greater than zero. |Yes | |`byte-order` |JSON string |{c-bo} of an instance of{nbsp}{var-f}. The value of this property {must} be one of: `"big-endian"`:: Big-endian. `"little-endian"`:: Little-endian. |Yes | |`alignment` |JSON integer |Alignment of the first bit of an instance of{nbsp}{var-f} relative to the beginning of the <> which contains this instance. The value of this property {must} be a positive power of two. |No |`1` |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Minimal {fl-ba} field class. ==== [source,json] ---- { "type": "fixed-length-bit-array", "length": 16, "byte-order": "little-endian" } ---- ==== .{c-fl-ba} field class with a 32-bit alignment. ==== [source,json] ---- { "type": "fixed-length-bit-array", "length": 48, "byte-order": "big-endian", "alignment": 32 } ---- ==== .{c-fl-ba} field class with <>. ==== [source,json] ---- { "type": "fixed-length-bit-array", "length": 16, "byte-order": "little-endian", "user-attributes": { "my.tracer": { "is-nice": true } } } ---- ==== [[fl-bool-fc]] ==== {c-fl-bool} field class A _{fl-bool}_ field class is a {fl-ba-fc} which describes _{fl-bool}_ fields. A {fl-bool} field is a {fl-ba} field which has the following semantics: If all the bit array field's bits are cleared (zero):: The {fl-bool} field's value is _false_. Otherwise:: The {fl-bool} field's value is _true_. .Properties of a {fl-bool} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"fixed-length-boolean"`. |Yes | |`length` |JSON integer |Number of bits of an instance of{nbsp}{var-f}. The value of this property {must} be greater than zero. Property inherited from the {fl-ba-fc}. |Yes | |`byte-order` |JSON string |{c-bo} of an instance of{nbsp}{var-f}. The value of this property {must} be one of: `"big-endian"`:: Big-endian. `"little-endian"`:: Little-endian. Property inherited from the {fl-ba-fc}. |Yes | |`alignment` |JSON integer |Alignment of the first bit of an instance of{nbsp}{var-f} relative to the beginning of the <> which contains this instance. The value of this property {must} be a positive power of two. Property inherited from the {fl-ba-fc}. |No |`1` |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Minimal {fl-bool} field class. ==== [source,json] ---- { "type": "fixed-length-boolean", "length": 16, "byte-order": "little-endian" } ---- ==== .{c-fl-bool} field class with a 32-bit alignment. ==== [source,json] ---- { "type": "fixed-length-boolean", "length": 48, "byte-order": "big-endian", "alignment": 32 } ---- ==== .{c-fl-bool} field class with <>. ==== [source,json] ---- { "type": "fixed-length-boolean", "length": 16, "byte-order": "little-endian", "user-attributes": { "my.tracer": { "is-nice": true } } } ---- ==== [[int-fc]] ==== Abstract integer field class An _abstract integer_ field class is a base of a {fl-int-fc} and a {vl-int-fc}. This field class is abstract in that it only exists to show the relation between different integer field classes in this document: a <> cannot contain an abstract integer field. .Common property of an integer field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`preferred-display-base` |JSON integer |Preferred base to display the value of an instance of{nbsp}{var-f}. The value of this property {must} be one of: `2`:: Binary base. `8`:: Octal base. `10`:: Decimal base. `16`:: Hexadecimal base. |No |`10` |=== [[fl-int-fc]] ==== {c-fl-int} field class A _{fl-int}_ field class is both an <> and a {fl-ba-fc} which describes _{fl-int}_ fields. A {fl-int} field is a {fl-ba} field which has integer semantics. If the value of a {fl-int} field class's `type` property is `"fixed-length-signed-integer"`, then its instances have the two's complement format. A {fl-int} field class acts as a base of a {fl-enum-fc}. .Common properties of a {fl-int} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be one of: `"fixed-length-unsigned-integer"`:: The instances of{nbsp}{var-f} are {fl-uint} fields. `"fixed-length-signed-integer"`:: The instances of{nbsp}{var-f} are {fl-sint} fields. |Yes | |`length` |JSON integer |Number of bits of an instance of{nbsp}{var-f}. The value of this property {must} be greater than zero. Property inherited from the {fl-ba-fc}. |Yes | |`byte-order` |JSON string |{c-bo} of an instance of{nbsp}{var-f}. The value of this property {must} be one of: `"big-endian"`:: Big-endian. `"little-endian"`:: Little-endian. Property inherited from the {fl-ba-fc}. |Yes | |`alignment` |JSON integer |Alignment of the first bit of an instance of{nbsp}{var-f} relative to the beginning of the <> which contains this instance. The value of this property {must} be a positive power of two. Property inherited from the {fl-ba-fc}. |No |`1` |`preferred-display-base` |JSON integer |Preferred base to display the value of an instance of{nbsp}{var-f}. The value of this property {must} be one of: `2`:: Binary base. `8`:: Octal base. `10`:: Decimal base. `16`:: Hexadecimal base. Property inherited from the <>. |No |`10` |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Minimal {fl-uint} field class. ==== [source,json] ---- { "type": "fixed-length-unsigned-integer", "length": 16, "byte-order": "little-endian" } ---- ==== .{c-fl-sint} field class with a 32-bit alignment. ==== [source,json] ---- { "type": "fixed-length-signed-integer", "length": 48, "byte-order": "big-endian", "alignment": 32 } ---- ==== .{c-fl-uint} field class with a preferred hexadecimal display base. ==== [source,json] ---- { "type": "fixed-length-unsigned-integer", "length": 48, "byte-order": "big-endian", "preferred-display-base": 16 } ---- ==== .{c-fl-sint} field class with <>. ==== [source,json] ---- { "type": "fixed-length-signed-integer", "length": 16, "byte-order": "little-endian", "user-attributes": { "my.tracer": { "is-nice": true } } } ---- ==== [[enum-fc]] ==== Abstract enumeration field class An _abstract enumeration_ field class is a base of a {fl-enum-fc} and a {vl-enum-fc}. This field class is abstract in that it only exists to show the relation between different enumeration field classes in this document: a <> cannot contain an abstract enumeration field. An abstract enumeration field class is an <>. An enumeration field is an integer field which {may} have one or more associated names thanks to its class's `mappings` property. .Common property of an enumeration field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`preferred-display-base` |JSON integer |Preferred base to display the value of an instance of{nbsp}{var-f}. The value of this property {must} be one of: `2`:: Binary base. `8`:: Octal base. `10`:: Decimal base. `16`:: Hexadecimal base. Property inherited from the <>. |No |`10` |`mappings` |<> |Mappings of{nbsp}{var-f}. The value of this property {must} contain one or more properties. |Yes | |=== [[enum-fc-mappings]] ===== Enumeration field class mappings _Enumeration field class mappings_ map names to <>. Enumeration field class mappings are a JSON object, where each property is: [horizontal] Name:: Mapping name. Value:: Mapped ranges of integers (<>). The integer ranges of two given mappings {may} overlap. Enumeration field class mappings {must} contain one or more properties. .Enumeration field class mappings with three mappings. ==== In this example, the `fortune` and `building` mappings overlap with the values 4 and 5, and the `building` and `journal` mappings overlap with the value 80. [source,json] ---- { "fortune": [[3, 67], [-45, 1], [84, 84]], "building": [[4, 5], [75, 82]], "journal": [[100, 2305], [80, 80]] } ---- ==== [[fl-enum-fc]] ==== {c-fl-enum} field class A _{fl-enum}_ field class is both an <> and a {fl-int-fc} which describes _{fl-enum}_ fields. A {fl-enum} field is a {fl-int} field which {may} have one or more associated names thanks to its class's `mappings` property. If the value of a {fl-enum} field class's `type` property is `"fixed-length-signed-enumeration"`, then its instances have the two's complement format. .Properties of a {fl-enum} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be one of: `"fixed-length-unsigned-enumeration"`:: The instances of{nbsp}{var-f} are {fl-uenum} fields. `"fixed-length-signed-enumeration"`:: The instances of{nbsp}{var-f} are {fl-senum} fields. |Yes | |`length` |JSON integer |Number of bits of an instance of{nbsp}{var-f}. The value of this property {must} be greater than zero. Property inherited from the {fl-ba-fc}. |Yes | |`byte-order` |JSON string |{c-bo} of an instance of{nbsp}{var-f}. The value of this property {must} be one of: `"big-endian"`:: Big-endian. `"little-endian"`:: Little-endian. Property inherited from the {fl-ba-fc}. |Yes | |`alignment` |JSON integer |Alignment of the first bit of an instance of{nbsp}{var-f} relative to the beginning of the <> which contains this instance. The value of this property {must} be a positive power of two. Property inherited from the {fl-ba-fc}. |No |`1` |`preferred-display-base` |JSON integer |Preferred base to display the value of an instance of{nbsp}{var-f}. The value of this property {must} be one of: `2`:: Binary base. `8`:: Octal base. `10`:: Decimal base. `16`:: Hexadecimal base. Property inherited from the <>. |No |`10` |`mappings` |<> |Mappings of{nbsp}{var-f}. The value of this property {must} contain one or more properties. Property inherited from the <>. |Yes | |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Minimal {fl-uenum} field class. ==== [source,json] ---- { "type": "fixed-length-unsigned-enumeration", "length": 16, "byte-order": "little-endian", "mappings": { "apple": [[1, 19]] } } ---- ==== .{c-fl-senum} field class with a 32-bit alignment. ==== [source,json] ---- { "type": "fixed-length-signed-enumeration", "length": 48, "byte-order": "big-endian", "alignment": 32, "mappings": { "banana": [[-27399, -1882], [8, 199], [101, 101]], "orange": [[67, 67], [43, 1534]] } } ---- ==== .{c-fl-uenum} field class with a preferred hexadecimal display base. ==== [source,json] ---- { "type": "fixed-length-unsigned-enumeration", "length": 8, "byte-order": "big-endian", "preferred-display-base": 16, "mappings": { "lime": [[3, 3]], "kiwi": [[8, 8]], "blueberry": [[11, 11]] } } ---- ==== .{c-fl-senum} field class with <>. ==== [source,json] ---- { "type": "fixed-length-signed-enumeration", "length": 16, "byte-order": "little-endian", "mappings": { "mango": [[23, 42]] }, "user-attributes": { "my.tracer": { "is-nice": true } } } ---- ==== [[fl-fp-fc]] ==== {c-fl-fp} field class A _{fl-fp}_ field class is a {fl-ba-fc} which describes _{fl-fp}_ fields. A {fl-fp} field is a {fl-ba} field which has floating point number semantics. .Properties of a {fl-fp} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"fixed-length-floating-point-number"`. |Yes | |`length` |JSON integer |Number of bits of an instance of{nbsp}{var-f}. The value of this property {must} be one of: `16`:: The instances of{nbsp}{var-f} are binary16 floating point numbers, as per {ieee754}. `32`:: The instances of{nbsp}{var-f} are binary32 floating point numbers. `64`:: The instances of{nbsp}{var-f} are binary64 floating point numbers. `128`:: The instances of{nbsp}{var-f} are binary128 floating point numbers. _**K**_, where _**K**_ is greater than{nbsp}128 and a multiple of{nbsp}32:: The instances of{nbsp}{var-f} are binary__**K**__ floating point numbers. Property inherited from the {fl-ba-fc}. |Yes | |`byte-order` |JSON string |{c-bo} of an instance of{nbsp}{var-f}. The value of this property {must} be one of: `"big-endian"`:: Big-endian. `"little-endian"`:: Little-endian. Property inherited from the {fl-ba-fc}. |Yes | |`alignment` |JSON integer |Alignment of the first bit of an instance of{nbsp}{var-f} relative to the beginning of the <> which contains this instance. The value of this property {must} be a positive power of two. Property inherited from the {fl-ba-fc}. |No |`1` |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Minimal binary32 {fl-fp} field class. ==== [source,json] ---- { "type": "fixed-length-floating-point-number", "length": 32, "byte-order": "little-endian" } ---- ==== .binary64 {fl-fp} field class with a 32-bit alignment. ==== [source,json] ---- { "type": "fixed-length-floating-point-number", "length": 64, "byte-order": "big-endian", "alignment": 32 } ---- ==== .binary192 {fl-fp} field class with <>. ==== [source,json] ---- { "type": "fixed-length-floating-point-number", "length": 192, "byte-order": "little-endian", "user-attributes": { "my.tracer": { "is-nice": true } } } ---- ==== [[vl-ba-fc]] ==== {c-vl-ba} field class A _{vl-ba}_ field class describes _{vl-ba}_ fields. A {vl-ba} field is a sequence of bytes with a variable length which contains an array of bits of which the length is a multiple of{nbsp}7. A {vl-ba} field is encoded as per https://en.wikipedia.org/wiki/LEB128[LEB128]. A {vl-ba} field class acts as a base of a {vl-int-fc}. .Common properties of a {vl-ba} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"variable-length-bit-array"`. |Yes | |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Minimal {vl-ba} field class. ==== [source,json] ---- { "type": "variable-length-bit-array" } ---- ==== .{c-vl-ba} field class with <>. ==== [source,json] ---- { "type": "variable-length-bit-array", "user-attributes": { "my.tracer": { "is-nice": true } } } ---- ==== [[vl-int-fc]] ==== {c-vl-int} field class A _{vl-int}_ field class is both an <> and a {vl-ba-fc} which describes _{vl-int}_ fields. A {vl-int} field is a {vl-ba} field which has integer semantics. If the value of a {vl-int} field class's `type` property is `"variable-length-signed-integer"`, then its instances have the two's complement format. A {vl-int} field class acts as a base of a {vl-enum-fc}. .Common properties of a {vl-int} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be one of: `"variable-length-unsigned-integer"`:: The instances of{nbsp}{var-f} are {vl-uint} fields. `"variable-length-signed-integer"`:: The instances of{nbsp}{var-f} are {vl-sint} fields. |Yes | |`preferred-display-base` |JSON integer |Preferred base to display the value of an instance of{nbsp}{var-f}. The value of this property {must} be one of: `2`:: Binary base. `8`:: Octal base. `10`:: Decimal base. `16`:: Hexadecimal base. Property inherited from the <>. |No |`10` |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Minimal {vl-uint} field class. ==== [source,json] ---- { "type": "variable-length-unsigned-integer" } ---- ==== .{c-vl-sint} field class with a preferred hexadecimal display base. ==== [source,json] ---- { "type": "variable-length-signed-integer", "preferred-display-base": 16 } ---- ==== .{c-vl-uint} field class with <>. ==== [source,json] ---- { "type": "variable-length-unsigned-integer", "user-attributes": { "my.tracer": { "is-nice": true } } } ---- ==== [[vl-enum-fc]] ==== {c-vl-enum} field class A _{vl-enum}_ field class is both an <> and a {vl-int-fc} which describes _{vl-enum}_ fields. A {vl-enum} field is a {vl-int} field which {may} have one or more associated names thanks to its class's `mappings` property. If the value of a {vl-enum} field class's `type` property is `"variable-length-signed-enumeration"`, then its instances have the two's complement format. .Properties of a {vl-enum} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be one of: `"variable-length-unsigned-enumeration"`:: The instances of{nbsp}{var-f} are {vl-uenum} fields. `"variable-length-signed-enumeration"`:: The instances of{nbsp}{var-f} are {vl-senum} fields. |Yes | |`preferred-display-base` |JSON integer |Preferred base to display the value of an instance of{nbsp}{var-f}. The value of this property {must} be one of: `2`:: Binary base. `8`:: Octal base. `10`:: Decimal base. `16`:: Hexadecimal base. Property inherited from the <>. |No |`10` |`mappings` |<> |Mappings of{nbsp}{var-f}. The value of this property {must} contain one or more properties. Property inherited from the <>. |Yes | |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Minimal {vl-uenum} field class. ==== [source,json] ---- { "type": "variable-length-unsigned-enumeration", "mappings": { "apple": [[1, 19]] } } ---- ==== .{c-vl-uenum} field class with a preferred hexadecimal display base. ==== [source,json] ---- { "type": "variable-length-unsigned-enumeration", "preferred-display-base": 16, "mappings": { "lime": [[3, 3]], "kiwi": [[8, 8]], "blueberry": [[11, 11]] } } ---- ==== .{c-vl-senum} field class with <>. ==== [source,json] ---- { "type": "variable-length-signed-enumeration", "mappings": { "banana": [[-27399, -1882], [8, 199], [101, 101]], "orange": [[67, 67], [43, 1534]] }, "user-attributes": { "my.tracer": { "is-nice": true } } } ---- ==== [[str-fc]] ==== {c-str} field class A _{str}_ field class describes _{str}_ fields. A {str} field is, in this order: . Zero or more contiguous non-null (non-zero) bytes which form a UTF-8-encoded string. . One null (zero) byte. .Properties of a {str} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"null-terminated-string"`. |Yes | |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Minimal {str} field class. ==== [source,json] ---- { "type": "null-terminated-string" } ---- ==== .{c-str} field class with <>. ==== [source,json] ---- { "type": "null-terminated-string", "user-attributes": { "my.tracer": { "is-nice": true } } } ---- ==== [[sl-str-fc]] ==== {c-sl-str} field class A _{sl-str}_ field class describes _{sl-str}_ fields. A {sl-str} field is a sequence of zero or more contiguous bytes. All the bytes of a {sl-str} before the first null (zero) byte, if any, form a UTF-8-encoded string. All the bytes after the first null (zero) byte, if any, are garbage data. The length, or number of bytes, of a {sl-str} field is a property (`length`) of its class. .Properties of a {sl-str} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"static-length-string"`. |Yes | |`length` |JSON integer |Number of bytes contained in an instance of{nbsp}{var-f}. The value of this property {must} be greater than or equal to zero. |Yes | |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Empty {sl-str} field class. ==== [source,json] ---- { "type": "static-length-string", "length": 0 } ---- ==== .{c-sl-str} field class with instances having 100{nbsp}bytes. ==== [source,json] ---- { "type": "static-length-string", "length": 100 } ---- ==== .{c-sl-str} field class with <>. ==== [source,json] ---- { "type": "static-length-string", "length": 13, "user-attributes": { "my.tracer": null } } ---- ==== [[dl-str-fc]] ==== {c-dl-str} field class A _{dl-str}_ field class describes _{dl-str}_ fields. A {dl-str} field is a sequence of zero or more contiguous bytes. All the bytes of a {dl-str} before the first null (zero) byte, if any, form a UTF-8-encoded string. All the bytes after the first null (zero) byte, if any, are garbage data. The length, or number of bytes, of a {dl-str} field is the value of another, anterior length field. A consumer can find this length field thanks to the {dl-str} field class's `length-field-location` property. .Properties of a {dl-str} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"static-length-string"`. |Yes | |`length-field-location` |<> |Location of the field of which the value is the number of bytes contained in an instance of{nbsp}{var-f}. The class of the length field {must} be one of: * {c-fl-uint-fc} * {c-vl-uint-fc} |Yes | |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .{c-dl-str} field class. ==== [source,json] ---- { "type": "dynamic-length-string", "length-field-location": ["event-record-payload", "length"] } ---- ==== .{c-dl-str} field class with <>. ==== [source,json] ---- { "type": "dynamic-length-string", "length-field-location": ["event-record-common-context", "name-length"], "user-attributes": { "my.tracer": 177 } } ---- ==== [[blob-fc]] ==== Abstract BLOB field class An _abstract https://en.wikipedia.org/wiki/Binary_large_object[BLOB]_ field class is a base of a {sl-blob-fc} and a {dl-blob-fc}. This field class is abstract in that it only exists to show the relation between different BLOB field classes in this document: a <> cannot contain an abstract BLOB field. .Common properties of a BLOB field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`media-type` |JSON string |https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types[IANA media type] of an instance of{nbsp}{var-f}. |No |`"application/octet-stream"` |=== [[sl-blob-fc]] ==== {c-sl-blob} field class A _{sl-blob}_ field class is an <> which describes _{sl-blob}_ fields. A {sl-blob} field is a sequence of zero or more contiguous bytes with an associated IANA media type (given by its class's `media-type` property). The length, or number of bytes, of a {sl-blob} field is a property (`length`) of its class. .Properties of a {sl-blob} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"static-length-blob"`. |Yes | |`length` |JSON integer |Number of bytes contained in an instance of{nbsp}{var-f}. The value of this property {must} be greater than or equal to zero. |Yes | |`media-type` |JSON string |https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types[IANA media type] of an instance of{nbsp}{var-f}. Property inherited from the <>. |No |`"application/octet-stream"` |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Empty {sl-blob} field class with a default IANA media type. ==== [source,json] ---- { "type": "static-length-blob", "length": 0 } ---- ==== .Static-length TIFF BLOB field class with instances having 511,267{nbsp}bytes. ==== [source,json] ---- { "type": "static-length-blob", "length": 511267, "media-type": "image/tif" } ---- ==== .Static-length CSV BLOB field class with <>. ==== [source,json] ---- { "type": "static-length-blob", "length": 2400, "media-type": "text/csv", "user-attributes": { "my.tracer": { "csv-cols": 12 } } } ---- ==== [[dl-blob-fc]] ==== {c-dl-blob} field class A _{dl-blob}_ field class is an <> which describes _{dl-blob}_ fields. A {dl-blob} field is a sequence of zero or more contiguous bytes with an associated IANA media type. The length, or number of bytes, of a {dl-blob} field is the value of another, anterior length field. A consumer can find this length field thanks to the {dl-blob} field class's `length-field-location` property. .Properties of a {dl-blob} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"dynamic-length-blob"`. |Yes | |`length-field-location` |<> |Location of the field of which the value is the number of bytes contained in an instance of{nbsp}{var-f}. The class of the length field {must} be one of: * {c-fl-uint-fc} * {c-vl-uint-fc} |Yes | |`media-type` |JSON string |https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types[IANA media type] of an instance of{nbsp}{var-f}. Property inherited from the <>. |No |`"application/octet-stream"` |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .{c-dl-blob} field class with a default IANA media type. ==== [source,json] ---- { "type": "dynamic-length-blob", "length-field-location": ["event-record-payload", "length"] } ---- ==== .Dynamic-length JPEG BLOB field class with <>. ==== [source,json] ---- { "type": "dynamic-length-blob", "length-field-location": ["event-record-common-context", "length"], "media-type": "image/jpeg", "user-attributes": { "my.tracer": { "quality": 85 } } } ---- ==== [[struct-fc]] ==== Structure field class A _structure field class_ describes _structure fields_. A structure field is a sequence of zero or more structure field _members_. A structure field member is a named field. .Properties of a structure field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"structure"`. |Yes | |`members` |JSON array of <> |Classes of the members of an instance of{nbsp}{var-f}. Each member class's `name` property must be unique within this field class's member class names. |No |`+[]+` |`minimum-alignment` |JSON integer |Minimum alignment of the first bit of an instance of{nbsp}{var-f} relative to the beginning of the <> which contains this instance. The value of this property {must} be a positive power of two. The <> of the first bit of an instance of{nbsp}{var-f} {may} be greater than the value of this property. |No |`1` |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Empty structure field class (no member classes). ==== [source,json] ---- { "type": "structure" } ---- ==== .Structure field class with three member classes. ==== [source,json] ---- { "type": "structure", "members": [ { "name": "Villeray", "field-class": { "type": "null-terminated-string" } }, { "name": "Berri", "field-class": { "type": "fixed-length-unsigned-integer", "length": 32, "byte-order": "little-endian", "preferred-display-base": 2 }, "user-attributes": { "my.tracer": { "is-mask": true } } }, { "name": "Faillon", "field-class": { "type": "fixed-length-boolean", "length": 8, "byte-order": "little-endian" } } ] } ---- ==== .Structure field class with a minimum alignment. ==== [source,json] ---- { "type": "structure", "members": [ { "name": "St-Denis", "field-class": { "type": "null-terminated-string" } }, { "name": "Lajeunesse", "field-class": { "type": "fixed-length-unsigned-integer", "length": 32, "byte-order": "big-endian", "alignment": 32 } } ], "minimum-alignment": 64 } ---- ==== .Structure field class with <>. ==== [source,json] ---- { "type": "structure", "members": [ { "name": "Henri-Julien", "field-class": { "type": "fixed-length-signed-integer", "length": 48, "byte-order": "little-endian" } }, { "name": "Casgrain", "field-class": { "type": "static-length-string", "length": 32 } } ], "user-attributes": { "my.tracer": { "version": 4 } } } ---- ==== [[struct-member-cls]] ===== Structure field member class A _structure field member class_ describes _structure field members_. A structure field member class is a JSON object. .Properties of a structure field member class _**M**_. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`name` |JSON string |Name of{nbsp}__**M**__. |Yes | |`field-class` |<> |Field class of{nbsp}__**M**__. |Yes | |`user-attributes` |<> |User attributes of{nbsp}__**M**__. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}__**M**__. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .<> member class named `cat`. ==== [source,json] ---- { "name": "cat", "field-class": { "type": "null-terminated-string" } } ---- ==== .{c-vl-sint-fc} member class named `dog` with <>. ==== [source,json] ---- { "name": "dog", "field-class": { "type": "variable-length-signed-integer", "preferred-display-base": 8 }, "user-attributes": { "my.tracer": { "uuid": "f36100b8-ec36-4861-8d6b-a9d6ab8973c9", "is-pid": true } } } ---- ==== [[array-fc]] ==== Abstract array field class An _abstract array_ field class is a base of a {sl-array-fc} and a {dl-array-fc}. This field class is abstract in that it only exists to show the relation between different array field classes in this document: a <> cannot contain an abstract array field. .Common properties of an array field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`element-field-class` |<> |Class of the element fields contained in an instance of{nbsp}{var-f}. |Yes | |=== [[sl-array-fc]] ==== {c-sl-array} field class A _{sl-array}_ field class is an <> which describes _{sl-array}_ fields. A {sl-array} field is a sequence of zero or more element fields. The length, or number of element fields, of a {sl-array} field is a property (`length`) of its class. .Properties of a {sl-array} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"static-length-array"`. |Yes | |`element-field-class` |<> |Class of the element fields contained in an instance of{nbsp}{var-f}. Property inherited from the <>. |Yes | |`length` |JSON integer |Number of element fields contained in an instance of{nbsp}{var-f}. The value of this property {must} be greater than or equal to zero. |Yes | |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Empty {sl-array} field class. ==== [source,json] ---- { "type": "static-length-array", "element-field-class": { "type": "fixed-length-signed-integer", "length": 16, "byte-order": "little-endian", "alignment": 16 }, "length": 0 } ---- ==== .{c-sl-array} field class with instances having 100{nbsp}<> fields. ==== [source,json] ---- { "type": "static-length-array", "element-field-class": { "type": "null-terminated-string" }, "length": 100 } ---- ==== .{c-sl-array} field class with <>. ==== [source,json] ---- { "type": "static-length-array", "element-field-class": { "type": "variable-length-unsigned-integer" }, "length": 13, "user-attributes": { "my.tracer": true } } ---- ==== [[dl-array-fc]] ==== {c-dl-array} field class A _{dl-array}_ field class is an <> which describes _{dl-array}_ fields. A {dl-array} field is a sequence of zero or more element fields. The length, or number of element fields, of a {dl-array} field is the value of another, anterior length field. A consumer can find this length field thanks to the {dl-array} field class's `length-field-location` property. .Properties of a {dl-array} field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"dynamic-length-array"`. |Yes | |`element-field-class` |<> |Class of the element fields contained in an instance of{nbsp}{var-f}. Property inherited from the <>. |Yes | |`length-field-location` |<> |Location of the field of which the value is the number of element fields contained in an instance of{nbsp}{var-f}. The class of the length field {must} be one of: * {c-fl-uint-fc} * {c-vl-uint-fc} |Yes | |`roles` |<> |Roles of an instance of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .{c-dl-array} field class. ==== [source,json] ---- { "type": "dynamic-length-array", "element-field-class": { "type": "fixed-length-unsigned-integer", "length": 32, "byte-order": "big-endian", "alignment": 16 }, "length-field-location": ["event-record-payload", "length"] } ---- ==== .{c-dl-array} field class with <>. ==== [source,json] ---- { "type": "dynamic-length-array", "element-field-class": { "type": "variable-length-unsigned-integer" }, "length-field-location": ["packet-context", "common-length"], "user-attributes": { "my.tracer": 177 } } ---- ==== [[opt-fc]] ==== Optional field class An _optional_ field class describes _optional_ fields. An optional field is, depending on the value of another, anterior selector field, one of: * An instance of a given field class (optional field class's `field-class` property). + In this case, the optional field is said to be _enabled_. * A zero-bit field (no field). + In this case, the optional field is said to be _disabled_. A consumer can find the selector field thanks to the optional field class's `selector-field-location` property. .Properties of an optional field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"optional"`. |Yes | |`field-class` |<> |Class of an instance of{nbsp}{var-f} when it's enabled. |Yes | |`selector-field-location` |<> |Location of the field of which the value indicates whether or not an instance of{nbsp}{var-f} is enabled. The selector field {must} be an instance of one of: {c-fl-bool-fc}:: An instance of{nbsp}{var-f} is enabled when the selector field is true. {c-fl-int-fc}:: {c-vl-int-fc}:: An instance of{nbsp}{var-f} is enabled when the selector field's value is an element of any of the integer ranges of{nbsp}{var-f}'s `selector-field-ranges` property. |Yes | |`selector-field-ranges` |<> |Ranges of integers which the value of a selector field {must} be an element of to enable an instance of{nbsp}{var-f}. |Yes, if the selector field is an instance of a {fl-int-fc} or a {vl-int-fc}. |None if the selector field is an instance of a {fl-bool-fc}. |`roles` |<> |Roles of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Optional {sl-array-fc} with a <> selector field class. ==== [source,json] ---- { "type": "optional", "selector-field-location": ["event-record-payload", "has-ip"], "field-class": { "type": "static-length-array", "element-field-class": { "type": "fixed-length-unsigned-integer", "length": 8, "byte-order": "little-endian", "alignment": 8 }, "length": 16 } } ---- ==== .Optional {sl-array-fc} with a <> selector field class. ==== [source,json] ---- { "type": "optional", "selector-field-location": ["event-record-payload", "has-ip"], "selector-field-ranges": [[-12, -12], [-5, 0], [15, 35]], "field-class": { "type": "static-length-array", "element-field-class": { "type": "fixed-length-unsigned-integer", "length": 8, "byte-order": "little-endian", "alignment": 8 }, "length": 16 } } ---- ==== [[var-fc]] ==== Variant field class A _variant_ field class describes _variant_ fields. A variant field is, depending on the value of another, anterior selector field, the instance of a specific, effective field class amongst one or more _variant field class options_. A consumer can find the selector field thanks to the variant field class's `selector-field-location` property. .Properties of a variant field class {var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"variant"`. |Yes | |`options` |JSON array of <> |Options containing the possible effective classes of an instance of{nbsp}{var-f}. This array {must} contain one or more elements. Each option's `name` property, if it's set, must be unique within this field class's option names. The integer ranges (`selector-field-ranges` property) of two given options {must-not} intersect. |Yes | |`selector-field-location` |<> |Location of the field of which the value indicates which option of{nbsp}{var-f} contains the effective class of an instance of{nbsp}{var-f}. The selector field {must} be an instance of one of: * {c-fl-int-fc} * {c-vl-int-fc} |Yes | |`roles` |<> |Roles of{nbsp}{var-f}. See <> and <> which indicate accepted roles for their root field classes. |No |`+[]+` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Variant field class with two options. ==== [source,json] ---- { "type": "variant", "selector-field-location": ["event-record-payload", "sel"], "options": [ { "field-class": { "type": "null-terminated-string" }, "selector-field-ranges": [[5, 5]] }, { "field-class": { "type": "fixed-length-signed-integer", "length": 16, "byte-order": "little-endian", "preferred-display-base": 8 }, "selector-field-ranges": [[8, 8]] } ] } ---- ==== .Variant field class within an {opt-fc} which share the same selector field location. ==== This example shows that an optional field class and a contained variant field class {may} share the same selector field location. In this example, depending on the selector field's value: [horizontal] 0:: Optional field is _not_ enabled. 1:: Optional field is enabled and is a variant field. + Variant field is an instance of a {str-fc} (effective class). 2:: Optional field is enabled and is a variant field. + Variant field is an instance of a {vl-sint-fc} (effective class). [source,json] ---- { "type": "optional", "selector-field-location": ["event-record-payload", "sel"], "selector-field-ranges": [[1, 255]], "field-class": { "type": "variant", "selector-field-location": ["event-record-payload", "sel"], "options": [ { "field-class": { "type": "null-terminated-string" }, "selector-field-ranges": [[1, 1]] }, { "field-class": { "type": "variable-length-signed-integer", "preferred-display-base": 16 }, "selector-field-ranges": [[2, 2]] } ] } } ---- ==== .Variant field class with <>. ==== [source,json] ---- { "type": "variant", "selector-field-location": ["event-record-specific-context", "sel"], "options": [ { "field-class": { "type": "static-length-string", "length": 20 }, "selector-field-ranges": [[5, 5], [10, 10], [15, 15]] }, { "field-class": { "type": "fixed-length-floating-point-number", "length": 32, "byte-order": "big-endian" }, "selector-field-ranges": [[0, 4], [6, 9], [11, 14], [16, 127]] } ], "user-attributes": { "my.tracer": { "owner": "Jimmy", "id": 199990 } } } ---- ==== [[var-fc-opt]] ===== Variant field class option A _variant field class option_ contains a possible effective class of a variant field. A variant field class option _**O**_ also contains the ranges of integer values (`selector-field-ranges` property) of which a selector field's value {must} be an element of for a variant field's effective class to be the field class of {var-o}. A variant field class option is a JSON object. .Properties of a variant field class option {var-o} contained in a variant field class{nbsp}{var-f}. [%header%autowidth, cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`field-class` |<> |Field class of{nbsp}{var-o}. |Yes | |`selector-field-ranges` |<> |Ranges of integers which the value of a selector field {must} be an element of for the effective class of an instance of{nbsp}{var-f} to be the field class (`field-class` property) of{nbsp}{var-o}. |Yes | |`name` |JSON string |Name of{nbsp}{var-o}. This property exists to remain backward compatible with {ctf1}. |No |{var-o} is unnamed |`user-attributes` |<> |User attributes of{nbsp}{var-o}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-o}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== .Unnamed {str-fc} option. ==== [source,json] ---- { "field-class": { "type": "null-terminated-string" }, "selector-field-ranges": [[3, 9]] } ---- ==== .{c-vl-sint-fc} option named `juice` with <>. ==== [source,json] ---- { "name": "juice", "field-class": { "type": "variable-length-signed-integer", "preferred-display-base": 16 }, "selector-field-ranges": [[-4, 4], [9, 9], [100, 200]], "user-attributes": { "my.tracer": { "uuid": "f36100b8-ec36-4861-8d6b-a9d6ab8973c9", "is-did": true } } } ---- ==== [[preamble-frag]] === Preamble fragment A _preamble fragment_ indicates: * The {ctf2} major version (2). + {ctf2} doesn't have a minor version: all new changes are introduced as <>. * <> declarations. + An extension declaration is an initial extension of which the purpose is to declare that it's _enabled_ within the <>. + Because an extension {may} alter the {ctf2} format itself, and because a preamble fragment is always the first metadata stream fragment, those extension declarations make it possible for a consumer to decline the trace's <> gracefully if it doesn't support _any_ declared extension. The first fragment of a metadata stream {must} be a preamble fragment. .Properties of a preamble fragment {var-f}. [%header%autowidth] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"preamble"`. |Yes | |`version` |JSON integer |{ctf2} major version. The value of this property {must} be `2`. |Yes | |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extension declarations of{nbsp}{var-f}. The name of each property is a <> and its value is a <>. Within a <>, an extension named{nbsp}__**N**__ is _declared_ when it exists as a property named{nbsp}__**N**__, whatever the property's value. |No |`+{}+` |=== .Minimal preamble fragment. ==== [source,json] ---- { "type": "preamble", "version": 2 } ---- ==== .Preamble fragment with <> declarations. ==== The following preamble fragment declares the `piano` and `ramen` extensions under the `my.tracer` namespace. [source,json] ---- { "type": "preamble", "version": 2, "extensions": { "my.tracer": { "piano": { "keys": 88, "temperament": "equal" }, "ramen": null } } } ---- ==== [[tc-frag]] === Trace class fragment A _trace class_ describes <>. Within a metadata stream, a trace class fragment {must} occur before any <>. .Properties of a trace class fragment {var-f}. [%header%autowidth,cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"trace-class"`. |Yes | |`uuid` |<> |UUID of{nbsp}{var-f}. |No |{var-f}{nbsp}has no UUID |`packet-header-field-class` |{c-struct-fc} |Class of all the <> of an instance of{nbsp}{var-f}. Any <> within this property's value {must} satisfy at least one of: * Have at least one valid <>. * Be a {struct-fc}. * Be an {opt-fc}. * Be a {var-fc}. * Be the class of a field which is the selector field of an optional or variant field. |No |{var-f}{nbsp}has no packet header field class |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== ==== Roles [[pkt-header-roles]]If the `packet-header-field-class` property of a trace class fragment exists, then its inner <> {may} have the following <>: .Roles of field classes within a packet header field class. [%header%autowidth,cols="d,d,a,a"] |=== |Name |Description |Field class ({var-f}) constraints |Other constraints |`packet-magic-number` |<> magic number. The purpose of a packet magic number field is to confirm the beginning of a {ctf2} packet. |{c-fl-uint-fc} with the following property value: [horizontal] `length`:: `32` |An instance of{nbsp}{var-f} {must} be the packet header structure field's _first_ member. The value of an instance of{nbsp}{var-f} value {must} be 0xc1fc1fc1 (3254525889). |`trace-class-uuid` |Trace class UUID. The purpose of a trace class UUID field is to confirm the association between a <> and a <>. |{c-sl-blob-fc} with the following property value: [horizontal] `length`:: `16` |The `uuid` property of the trace class must exist. The value of an instance of{nbsp}{var-f} {must} be the binary representation of the `uuid` property of the trace class. |`data-stream-class-id` |Data stream class ID. The purpose of a data stream class ID field is to set the current ID of the class of the packet's data stream. |{c-fl-uint-fc} or {vl-uint-fc}. | |`data-stream-id` |Data stream ID. The purpose of a data stream ID field is to set the current ID of the packet's data stream. Combined with the ID of its class, such a field makes it possible to uniquely identify a data stream within a <>. |{c-fl-uint-fc} or {vl-uint-fc}. | |=== .Trace class fragment. ==== [source,json] ---- { "type": "trace-class", "uuid": "1ec96494-e402-4546-93db-e9222bee6cc7", "packet-header-field-class": { "type": "structure", "members": [ { "name": "the magic!", "field-class": { "type": "fixed-length-unsigned-integer", "length": 32, "byte-order": "little-endian", "preferred-display-base": 16, "roles": ["packet-magic-number"] } }, { "name": "the UUID", "field-class": { "type": "static-length-blob", "length": 16, "roles": ["trace-class-uuid"] } }, { "name": "my data stream class ID", "field-class": { "type": "fixed-length-unsigned-integer", "length": 8, "byte-order": "little-endian", "roles": ["data-stream-class-id"] } }, { "name": "my data stream ID", "field-class": { "type": "variable-length-unsigned-integer", "roles": ["data-stream-id"] } } ] } } ---- ==== [[cc-frag]] === Clock class fragment A _clock class_ describes _clocks_. A <> {may} have a <>. Within a metadata stream, a clock class fragment {must} occur before any <> which refers to it by name with its `default-clock-class-name` property. .Properties of a clock class fragment {var-f}. [%header%autowidth,cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"clock-class"`. |Yes | |`frequency` |JSON integer |Frequency of an instance of{nbsp}{var-f} (Hz). The value of this property {must} be greater than zero. |Yes | |`name` |JSON string |Name of{nbsp}{var-f}. |Yes | |`description` |JSON string |Textual description of{nbsp}{var-f}. This property exists to remain backward compatible with {ctf1}. |No |{var-f}{nbsp}has no textual description |`uuid` |<> |UUID of{nbsp}{var-f}. This property exists to remain backward compatible with {ctf1}. |No |{var-f}{nbsp}has no UUID |`origin-is-unix-epoch` |JSON boolean |Whether or not the origin of an instance of{nbsp}{var-f} is the https://en.wikipedia.org/wiki/Unix_time[Unix epoch]. If the value of this property is `false`, then the origin of an instance of{nbsp}{var-f} is unknown. |No |`true` |`offset` |<> |Offset of an instance of{nbsp}{var-f} relative to its origin. Let: * _**H**_ be the value of {var-f}'s `frequency` property. * {var-o} be the value of this property. * {var-s} be the value of {var-o}'s `seconds` property. * _**C**_ be the value of {var-o}'s `cycles` property. Then the effective offset of an instance of {var-f}, in clock cycles, is{nbsp}{var-s}{nbsp}{times}{nbsp}__**H**__{nbsp}pass:[+]{nbsp}__**C**__. |No |`{"seconds":{nbsp}0, "cycles":{nbsp}0}` |`precision` |JSON integer |Precision of an instance of{nbsp}{var-f} (clock cycles). The value of this property {must} be greater than or equal to zero. Let{nbsp}{var-p} be the value of this property and{nbsp}__**V**__ the value of an instance of{nbsp}{var-f}: the range of possible values of the instance is{nbsp}[__**V**__{nbsp}{minus}{nbsp}{var-p},{nbsp}__**V**__{nbsp}pass:[+]{nbsp}{var-p}]. |No |`0` |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== Within a metadata stream, two given clock class fragments {must-not}: * Share the same `name` property value. * Share the same `uuid` property value. .Minimal clock class fragment (1{nbsp}GHz frequency). ==== [source,json] ---- { "type": "clock-class", "name": "my clock class", "frequency": 1000000000 } ---- ==== .Clock class fragment with a UUID. ==== [source,json] ---- { "type": "clock-class", "name": "my clock class", "frequency": 1000000000, "uuid": "74d2008c-efff-4d03-8163-e9e2866acf20" } ---- ==== .Clock class fragment with an offset. ==== [source,json] ---- { "type": "clock-class", "name": "my clock class", "frequency": 1000000000, "offset": { "seconds": 1605112699, "cycles": 2878388 } } ---- ==== .Clock class fragment with a precision. ==== [source,json] ---- { "type": "clock-class", "name": "my clock class", "frequency": 8000000, "precision": 100 } ---- ==== .Clock class fragment with an origin which is not the Unix epoch. ==== [source,json] ---- { "type": "clock-class", "name": "my clock class", "frequency": 1000000000, "origin-is-unix-epoch": false } ---- ==== .Clock class fragment with <>. ==== [source,json] ---- { "type": "clock-class", "name": "my clock class", "frequency": 16000000, "user-attributes": { "my.tracer": { "sys-name": "SOC23", "bus": { "name": "LMB5", "index": 5 }, "propagation-delay-ps": 177 } } } ---- ==== [[cc-offset]] ==== Clock class offset A _clock class offset_ contains the offset of a <>'s instances relative to their origin. A clock class offset is a JSON object. .Properties of a clock class offset contained in a clock class fragment {var-f}. [%header%autowidth] |=== |Name |Type |Description |Required? |Default |`seconds` |JSON integer |Offset, in seconds, of an instance of{nbsp}{var-f} relative to its origin. |No |`0` |`cycles` |JSON integer |Offset, in cycles, of an instance of{nbsp}{var-f} relative to its origin. The value of this property {must} be greater than or equal to zero. The value of this property {must} be less than the value of{nbsp}{var-f}'s `frequency` property. |No |`0` |=== .Minimal clock class offset. ==== [source,json] ---- {} ---- ==== .Clock class offset with seconds and cycles. ==== [source,json] ---- { "seconds": 1605112699, "cycles": 2878388 } ---- ==== .Clock class offset with seconds only. ==== [source,json] ---- { "seconds": 1605111293 } ---- ==== .Negative clock class offset. ==== This example shows that a clock class offset {may} be negative, that is, _before_ the origin of the clock class instances. [source,json] ---- { "seconds": -18003, "cycles": 11928547 } ---- ==== [[dsc-frag]] === Data stream class fragment A _data stream class_ describes <>. Within a metadata stream, a data stream class fragment {var-f} {must} occur before any <> of which {var-f} is the parent. .Properties of a data stream class fragment {var-f}. [%header%autowidth,cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"data-stream-class"`. |Yes | |`id` |JSON integer |Numeric ID of{nbsp}{var-f}. The value of this property {must} be greater than or equal to zero. |No |`0` |`name` |JSON string |Name of{nbsp}{var-f}. The purpose of this property, combined with the `namespace` property, is to uniquely identify a data stream class amongst many producers. |No |{var-f}{nbsp}is unnamed |`namespace` |JSON string |<> of{nbsp}{var-f}. The purpose of this property, combined with the `name` property, is to uniquely identify a data stream class amongst many producers. |No |{var-f}{nbsp}has no namespace |`default-clock-class-name` |JSON string |Name of the <> of the <> of an instance of{nbsp}{var-f}. Within the metadata stream containing{nbsp}{var-f}, the <> which has this property's value as its `name` property must occur before{nbsp}{var-f}. |No |An instance of{nbsp}{var-f} has no default clock |`packet-context-field-class` |{c-struct-fc} |Class of all the <> of an instance of{nbsp}{var-f}. |No |{var-f}{nbsp}has no packet context field class |`event-record-header-field-class` |{c-struct-fc} |Class of all the <> of an instance of{nbsp}{var-f}. Any field class within this property's value {must} satisfy at least one of: * Have at least one valid <>. * Be a {struct-fc}. * Be an {opt-fc}. * Be a {var-fc}. * Be the class of a field which is the selector field of an optional or variant field. |No |{var-f}{nbsp}has no event record header field class |`event-record-common-context-field-class` |{c-struct-fc} |Class of all the <> of an instance of{nbsp}{var-f}. |No |{var-f}{nbsp}has no event record common context field class |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== Within a metadata stream, two given data stream class fragments {must-not} share the same `id` property value. ==== Roles [[pkt-ctx-roles]]If the `packet-context-field-class` property of a data stream class fragment exists, then its inner <> {may} have the following <>: .Roles of field classes within a packet context field class. [%header%autowidth,cols="d,d,a,a"] |=== |Name |Description |Field class ({var-f}) constraints |Other constraints |`packet-total-size` |Total size (bits) of the <>. This size includes any padding after the packet's content. |{c-fl-uint-fc} or {vl-uint-fc}. |The value of an instance of{nbsp}{var-f} {must} be greater than or equal to the value of an instance of a field class having the `packet-content-size` role within the _same_ packet context field. |`packet-content-size` |Content size (bits) of the packet. |{c-fl-uint-fc} or {vl-uint-fc}. |The value of an instance of{nbsp}{var-f} {must} be less than or equal to the value of an instance of a field class having the `packet-total-size` role within the _same_ packet context field. |`packet-beginning-default-clock-timestamp` |Timestamp of the packet's <>'s <> when the packet begins. |{c-fl-uint-fc} or {vl-uint-fc}. |The timestamps of all the packet's <> {must} be greater than or equal to the value of an instance of{nbsp}{var-f}. The value of an instance of{nbsp}{var-f} {must} be less than or equal to the value of an instance of a field class having the `packet-end-default-clock-timestamp` role within the _same_ packet context field. |`packet-end-default-clock-timestamp` |Timestamp of the packet's data stream's <> when the packet ends. |{c-fl-uint-fc} or {vl-uint-fc}. |The timestamps of all the packet's <> {must} be less than or equal to the value of an instance of{nbsp}{var-f}. The value of an instance of{nbsp}{var-f} {must} be greater than or equal to the value of an instance of a field class having the `packet-beginning-default-clock-timestamp` role within the _same_ packet context field. |`discarded-event-record-counter-snapshot` |Snapshot of the packet's data stream's <> when the packet ends. |{c-fl-uint-fc} or {vl-uint-fc}. | |[[pkt-seq-num-role]]`packet-sequence-number` |Sequence number of the packet within its data stream. |{c-fl-uint-fc} or {vl-uint-fc}. | |=== [[er-header-roles]]If the `event-record-header-field-class` property of a data stream class fragment exists, then its inner <> {may} have the following <>: .Roles of field classes within an event record header field class. [%header%autowidth,cols="d,d,a"] |=== |Name |Description |Field class ({var-f}) constraints |`event-record-class-id` |Event record class ID. The purpose of an event record class ID field is to set the current ID of the class of the event record within its parent <>. |{c-fl-uint-fc} or {vl-uint-fc}. |`default-clock-timestamp` |Current timestamp of the event record's <>'s <> when the event record occurs. |{c-fl-uint-fc} or {vl-uint-fc}. |=== [[erc-frag]] === Event record class fragment An _event record class_ describes <>. The <> of which the `id` property's value matches the value of the `data-stream-class-id` property of an event record class fragment{nbsp}{var-f} is considered the _parent_ of{nbsp}{var-f}. .Properties of an event record class fragment {var-f} having the data stream class{nbsp}{var-p} as its parent. [%header%autowidth,cols="d,d,a,d,d"] |=== |Name |Type |Description |Required? |Default |`type` |JSON string |Type of{nbsp}{var-f}. The value of this property {must} be `"event-record-class"`. |Yes | |`id` |JSON integer |Numeric ID of{nbsp}{var-f} within{nbsp}{var-p}. The value of this property {must} be greater than or equal to zero. |No |`0` |`data-stream-class-id` |JSON integer |Numeric ID of{nbsp}{var-p}. The value of this property {must} be greater than or equal to zero. Within the metadata stream,{nbsp}{var-p} {must} occur before{nbsp}{var-f}. |No |`0` |`name` |JSON string |Name of{nbsp}{var-f}. The purpose of this property, combined with the `namespace` property, is to uniquely identify an event record class amongst many producers. |No |{var-f}{nbsp}is unnamed |`namespace` |JSON string |<> of{nbsp}{var-f}. The purpose of this property, combined with the `name` property, is to uniquely identify an event record class amongst many producers. |No |{var-f}{nbsp}has no namespace |`specific-context-field-class` |{c-struct-fc} |Class of the <> of an instance of{nbsp}{var-f}. |No |{var-f}{nbsp}has no event record specific context field class |`payload-field-class` |{c-struct-fc} |Class of the <> of an instance of{nbsp}{var-f}. |No |{var-f}{nbsp}has no event record payload field class |`user-attributes` |<> |User attributes of{nbsp}{var-f}. |No |`+{}+` |`extensions` |<> |Extensions of{nbsp}{var-f}. Any extension which exists under this property must also be declared in the metadata stream's <>. |No |`+{}+` |=== Within a metadata stream, two given event record class fragments {must-not} share the same `id` property value _and_ the same `data-stream-class-id` property value. [[ds-dec]] == Data stream decoding procedure This section shows how to, procedurally, _decode_ a {ctf2} <>. Decoding a data stream is the responsibility of a consumer. This document doesn't specify how to encode a data stream, as this procedure implies much more freedom than decoding. You can deduce how to encode a data stream from the decoding procedure. A consumer needs to keep a _data stream decoding state_ while decoding a data stream. A data stream decoding state comprises the following _variable_: .Variable needed to decode a data stream{nbsp}{var-s}. [%header%autowidth] |=== |Name |Type |Description |Initial value |{var-o} |Integer |Current decoding offset/position (bits) from the beginning of{nbsp}{var-s}. |0 |=== To decode a data stream {var-s}: * While there's remaining data in {var-s}: ** <>. [[pkt-dec]] === Packet decoding procedure A consumer needs to keep a _packet decoding state_ while decoding a <>. A packet decoding state comprises the following _variables_: .Variables needed to decode a packet{nbsp}{var-p} within a data stream{nbsp}{var-s}. [%header%autowidth] |=== |Name |Type |Description |Initial value |_**DEF_CLK_VAL**_ |Integer |Current value (clock cycles) of the <> of{nbsp}{var-s}, if any. |0 |_**DSC_ID**_ |Integer |Current ID of the <> of{nbsp}{var-s}. |0 |_**DSC**_ |<> |Current class of{nbsp}{var-s}. |None |_**DS_ID**_ |Integer |Current ID of{nbsp}{var-s}. |None |_**PKT_TOTAL_SZ**_ |Integer |Current total size (bits) of{nbsp}{var-p}. |∞ |_**PKT_CONTENT_SZ**_ |Integer |Current content size (bits) of{nbsp}{var-p}. |∞ |_**LAST_BO**_ |String |Byte order of the last <>. |None |=== To decode a packet {var-p} within a data stream {var-s}: . Let {var-po} be the current value of {var-o}. . If the metadata stream's <>'s `packet-header-field-class` property exists, then <> the <> of{nbsp}{var-p} using this property. + During the packet header field decoding procedure, after having decoded a field{nbsp}{var-f} having the class{nbsp}__**C**__ with a `roles` property: + ** If _**C**_ has the role `packet-magic-number`, then validate that the integer value of{nbsp}{var-f} is 0xc1fc1fc1 (3254525889). ** If _**C**_ has the role `trace-type-uuid`, then validate that the value of{nbsp}{var-f} matches the `uuid` property of the trace class fragment. ** If _**C**_ has the role `data-stream-class-id`, then set{nbsp}__**DSC_ID**__ to the integer value of{nbsp}{var-f}. ** If _**C**_ has the role `data-stream-id`, then set{nbsp}__**DS_ID**__ to the integer value of{nbsp}{var-f}. + After having decoded the whole packet header field, if __**DS_ID**__ is set, it contains the ID of{nbsp}{var-s} within its <>. In other words, two data streams {may} have the same ID if they are instances of different data stream classes. . Set _**DSC**_ to the <> having{nbsp}__**DSC_ID**__ as the value of its `id` property. . If the `packet-context-field-class` property of{nbsp}__**DSC**__ exists, then <> the <> of{nbsp}{var-p} using this property. + During the packet context field decoding procedure, after having decoded a field{nbsp}{var-f} having the class{nbsp}__**C**__ with a `roles` property: + ** If _**C**_ has the role `packet-total-size`, then set{nbsp}__**PKT_TOTAL_SZ**__ to the integer value of{nbsp}{var-f}. ** If _**C**_ has the role `packet-content-size`, then set{nbsp}__**PKT_CONTENT_SZ**__ to the integer value of{nbsp}{var-f}. ** If _**C**_ has the role `packet-beginning-default-clock-timestamp`, then set{nbsp}__**DEF_CLK_VAL**__ to the integer value of{nbsp}{var-f}. ** If _**C**_ has the role `packet-end-default-clock-timestamp`, then the integer value of{nbsp}{var-f} is the value of the <> of{nbsp}{var-s} at the end of{nbsp}{var-p}. ** If _**C**_ has the role `discarded-event-record-counter-snapshot`, then the integer value of{nbsp}{var-f} is a snapshot of the <> of{nbsp}{var-s} at the end of{nbsp}{var-p}. ** If _**C**_ has the role `packet-sequence-number`, then the integer value of{nbsp}{var-f} is the sequence number of{nbsp}{var-p} within{nbsp}{var-s}. . If __**PKT_TOTAL_SZ**__ is{nbsp}∞ and __**PKT_CONTENT_SZ**__ is __not__{nbsp}∞, then set{nbsp}__**PKT_TOTAL_SZ**__ to{nbsp}__**PKT_CONTENT_SZ**__. . If __**PKT_CONTENT_SZ**__ is{nbsp}∞ and __**PKT_TOTAL_SZ**__ is __not__{nbsp}∞, then set{nbsp}__**PKT_CONTENT_SZ**__ to{nbsp}__**PKT_TOTAL_SZ**__. . While {var-o}{nbsp}<{nbsp}{var-po}{nbsp}pass:[+]{nbsp}__**PKT_CONTENT_SZ**__ and there's remaining data in{nbsp}{var-s}: ** <>. . If __**PKT_TOTAL_SZ**__ and{nbsp}__**PKT_CONTENT_SZ**__ both are __not__{nbsp}∞, then set {var-o} to {var-po}{nbsp}pass:[+]{nbsp}__**PKT_TOTAL_SZ**__, effectively skipping end-of-packet padding. [[er-dec]] === Event record decoding procedure A consumer needs to keep an _event record decoding state_ while decoding an <>. An event record decoding state comprises the following _variables_: .Variables needed to decode an event record{nbsp}__**E**__ within a data stream{nbsp}{var-s}. [%header%autowidth] |=== |Name |Type |Description |Initial value |_**ERC_ID**_ |Integer |Current ID of the <> of{nbsp}__**E**__ of which the parent is the <> of{nbsp}{var-s}. |0 |_**ERC**_ |<> |Current class of{nbsp}__**E**__. |None |=== To decode an event record _**E**_ within a data stream {var-s}: . If the `event-record-header-field-class` property of{nbsp}__**DSC**__ exists, then <> the <> of{nbsp}__**E**__ using this property. + During the event record header field decoding procedure, after having decoded a field{nbsp}{var-f} having the class{nbsp}__**C**__ with a `roles` property: + ** If _**C**_ has the role `event-record-class-id`, then set{nbsp}__**ERC_ID**__ to the integer value of{nbsp}{var-f}. ** If _**C**_ has the role `default-clock-timestamp`, then update{nbsp}__**DEF_CLK_VAL**__ according to Common Trace Format{nbsp}1.8.3,{nbsp}§8. + NOTE: This proposal refers to {ctf1} to remain brief; the {ctf2} specification will show the complete clock value updating procedure. + After having decoded the whole event record header field, __**DEF_CLK_VAL**__ is the value of{nbsp}{var-s}'s <> when{nbsp}__**E**__ occurs. . Set _**ERC**_ to the <> having: ** __**DSC_ID**__ as the value of its `data-stream-class-id` property. ** __**ERC_ID**__ as the value of its `id` property. . If the `event-record-common-context-field-class` property of{nbsp}__**DSC**__ exists, then <> the <> of{nbsp}__**E**__ using this property. . If the `specific-context-field-class` property of{nbsp}__**ERC**__ exists, then <> the <> of{nbsp}__**E**__ using this property. . If the `payload-field-class` property of{nbsp}__**ERC**__ exists, then <> the <> of{nbsp}__**E**__ using this property. [[field-dec]] === Field decoding procedure The <> of a field contains what's needed to decode it as a _value_. While a field is an actual sequence of bits within a <>, a value is its conceptual interpretation with attached semantics. The types of values are: [%header%autowidth,cols="a,a"] |=== |Value type |Possible values |Boolean |_True_ or _false_. |Integer |Integral quantity. |Real |Continuous quantity. |String |Sequence of https://home.unicode.org/[Unicode] characters. |Array |Sequence of values having the same type. |Structure |Sequence of named values (members) which {may} have different types. |=== To decode an instance of a field class{nbsp}{var-f}, depending on the value of its `type` property: [%header%autowidth,cols="a,a"] |=== |Value of{nbsp}{var-f}'s `type` property |{var-f}'s decoding procedure |<> |<>. |<> |<>. | * <> * <> |<>. | * <> * <> |<>. |<> |<>. |<> |<>. | * <> * <> |<>. | * <> * <> |<>. |<> |<>. |<> |<>. |<> |<>. |<> |<>. |<> |<>. |<> |<>. |<> |<>. |<> |<>. |<> |<>. |<> |<>. |=== [[align-dec]] ==== Alignment procedure The decoding procedure of many fields require {var-o-minus-po} to have a specific _alignment_. The alignment requirement of an instance of a <>{nbsp}{var-f} is, depending on the value of its `type` property: [%header,cols="a,a"] |=== |{var-f}'s `type` property |{var-f}'s alignment requirement | * <> * <> * <> * <> * <> * <> * <> |The value of {var-f}'s `alignment` property. | * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> |8 |<> |The _maximum_ value of: * The value of{nbsp}{var-f}'s `minimum-alignment` property. * The alignment requirements of the instances of the `field-class` property of each <> of{nbsp}{var-f}'s `members` property. | * <> * <> |The alignment requirement of an instance of{nbsp}{var-f}'s `element-field-class` property | * <> * <> |1 |=== To align {var-o-minus-po} to some alignment requirement{nbsp}__**A**__ (bits): * Set {var-o} to {var-po}{nbsp}pass:[+]{nbsp}(({var-o-minus-po}{nbsp}pass:[+]{nbsp}__**A**__{nbsp}{minus}{nbsp}1){nbsp}&{nbsp}{minus}__**A**__), where __&__ is the bitwise _AND_ operator. [[fl-ba-field-dec]] ==== {c-fl-ba} field decoding procedure To decode an instance of a {fl-ba-fc}{nbsp}{var-f}: . Let: ** _**BO**_ be the value of {var-f}'s `byte-order` property. ** _**L**_ be the value of {var-f}'s `length` property. ** {var-v} be an array of booleans of length{nbsp}__**L**__. . <> according to{nbsp}{var-f}. . If (({var-o-minus-po}){nbsp}mod{nbsp}8{nbsp}{noteq}{nbsp}0) and __**LAST_BO**__{nbsp}{noteq}{nbsp}__**BO**__, then fail immediately. . Read __**L**__{nbsp}bits of data from{nbsp}{var-s} at the offset{nbsp}{var-o} following Common Trace Format{nbsp}1.8.3, §4.1.5, as booleans within{nbsp}{var-v}. + NOTE: This proposal refers to {ctf1} to remain brief; the {ctf2} specification will show the complete decoding procedures for big-endian and little-endian {fl-ba} fields. . Set {var-o} to {var-o}{nbsp}pass:[+]{nbsp}__**L**__. . Set _**LAST_BO**_ to{nbsp}__**BO**__. {var-v} is the decoded value. [[fl-bool-field-dec]] ==== {c-fl-bool} field decoding procedure To decode an instance of a {fl-bool-fc}: . Let __**VB**__ be a boolean. . <>. . If all the elements of{nbsp}{var-v} are _false_, then set{nbsp}__**VB**__ to _false_. + Else, set __**VB**__ to _true_. _**VB**_ is the decoded boolean value. [[fl-uint-field-dec]] ==== {c-fl-uint} field decoding procedure To decode an instance of a {fl-uint-fc}: . Let __**VI**__ be an integer. . <>. . Set __**VI**__ as the unsigned integer interpretation of{nbsp}{var-v}. _**VI**_ is the decoded integer value. [[fl-sint-field-dec]] ==== {c-fl-sint} field decoding procedure To decode an instance of a {fl-sint-fc}: . Let __**VI**__ be an integer. . <>. . Set __**VI**__ as the signed integer interpretation, following the two's complement format, of{nbsp}{var-v}. _**VI**_ is the decoded integer value. [[fl-fp-field-dec]] ==== {c-fl-fp} field decoding procedure To decode an instance of a {fl-fp-fc}: . Let __**VR**__ be a real value. . <>. . Set __**VR**__ to the real number interpretation, following {ieee754}, of{nbsp}{var-v}. _**VR**_ is the decoded real value. [[vl-ba-field-dec]] ==== {c-vl-ba} field decoding procedure To decode an instance of a {vl-ba-fc}{nbsp}{var-f}: . Let {var-v} be an empty array of booleans. . <> according to{nbsp}{var-f}. . Read __**N**__{nbsp}bytes of data from{nbsp}{var-s} at the offset{nbsp}{var-o}, as many as needed following the unsigned https://en.wikipedia.org/wiki/LEB128[LEB128] format, appending the decoded bits to{nbsp}{var-v} as booleans. + NOTE: This proposal refers to LEB128 to remain brief; the {ctf2} specification will show the complete decoding procedure. . Set {var-o} to {var-o}{nbsp}pass:[+]{nbsp}__**N**__{nbsp}{times}{nbsp}8. {var-v} is the decoded value. [[vl-uint-field-dec]] ==== {c-vl-uint} field decoding procedure To decode an instance of a {vl-uint-fc}: . Let __**VI**__ be an integer. . <>. . Set __**VI**__ as the unsigned integer interpretation of{nbsp}{var-v}. _**VI**_ is the decoded integer value. [[vl-sint-field-dec]] ==== {c-vl-sint} field decoding procedure To decode an instance of a {vl-sint-fc}: . Let __**VI**__ be an integer. . <>. . Set __**VI**__ as the signed integer interpretation, following the two's complement format, of{nbsp}{var-v}. _**VI**_ is the decoded integer value. [[str-field-dec]] ==== {c-str} field decoding procedure To decode an instance of a {str-fc}{nbsp}{var-f}: . Let: ** _**B**_ be a byte. ** _**A**_ be an empty sequence of bytes. ** {var-v} be a string. . <> according to{nbsp}{var-f}. . Read one byte of data from{nbsp}{var-s} at the offset{nbsp}{var-o} as{nbsp}__**B**__. . Set {var-o} to {var-o}{nbsp}pass:[+]{nbsp}8. . While __**B**__{nbsp}{noteq}{nbsp}0: .. Append __**B**__ to{nbsp}__**A**__. .. Read one byte of data from{nbsp}{var-s} at the offset{nbsp}{var-o} as{nbsp}__**B**__. .. Set {var-o} to {var-o}{nbsp}pass:[+]{nbsp}8. . Decode _**A**_, following UTF-8, as {var-v}. {var-v} is the decoded string value. [[sl-str-field-dec]] ==== {c-sl-str} field decoding procedure To decode an instance of a {sl-str-fc}{nbsp}{var-f}: . Let: ** _**L**_ be {var-f}'s `length` property. ** _**I**_ be an unsigned integer initialized to 0. ** _**B**_ be a byte. ** _**R**_ be a boolean initialized to _true_. ** _**A**_ be an empty sequence of bytes. ** {var-v} be a string. . <> according to{nbsp}{var-f}. . While __**I**__{nbsp}<{nbsp}__**L**__: .. Read one byte of data from{nbsp}{var-s} at the offset{nbsp}{var-o} as{nbsp}__**B**__. .. If __**B**__{nbsp}={nbsp}0, then set _**R**_{nbsp}to _false_. + Else, if _**R**_ is _true_, then append{nbsp}__**B**__ to{nbsp}__**A**__. .. Set {var-o} to {var-o}{nbsp}pass:[+]{nbsp}8. .. Set _**I**_ to __**I**__{nbsp}pass:[+]{nbsp}1. . Decode _**A**_, following UTF-8, as {var-v}. {var-v} is the decoded string value. [[sl-blob-field-dec]] ==== {c-sl-blob} field decoding procedure To decode an instance of a {sl-blob-fc}{nbsp}{var-f}: . Let: ** _**L**_ be {var-f}'s `length` property. ** {var-v} be an array of bytes of length{nbsp}__**L**__. . <> according to{nbsp}{var-f}. . Read _**L**_ bytes of data from{nbsp}{var-s} at the offset{nbsp}{var-o} as{nbsp}{var-v}. . Set {var-o} to {var-o}{nbsp}pass:[+]{nbsp}__**L**__{nbsp}{times}{nbsp}8. {var-v} is the decoded BLOB value. [[dl-str-field-dec]] ==== {c-dl-str} field decoding procedure To decode an instance of a {dl-str-fc}{nbsp}{var-f}: . Let: ** _**L**_ be the value of the previously decoded unsigned integer field of which{nbsp}{var-f}'s `length-field-location` indicates the location. ** _**I**_ be an unsigned integer initialized to 0. ** _**B**_ be a byte. ** _**R**_ be a boolean initialized to _true_. ** _**A**_ be an empty sequence of bytes. ** {var-v} be a string. . <> according to{nbsp}{var-f}. . While __**I**__{nbsp}<{nbsp}__**L**__: .. Read one byte of data from{nbsp}{var-s} at the offset{nbsp}{var-o} as{nbsp}__**B**__. .. If __**B**__{nbsp}={nbsp}0, then set _**R**_{nbsp}to _false_. + Else, if _**R**_ is _true_, then append{nbsp}__**B**__ to{nbsp}__**A**__. .. Set {var-o} to {var-o}{nbsp}pass:[+]{nbsp}8. .. Set _**I**_ to __**I**__{nbsp}pass:[+]{nbsp}1. . Decode _**A**_, following UTF-8, as {var-v}. {var-v} is the decoded string value. [[dl-blob-field-dec]] ==== {c-dl-blob} field decoding procedure To decode an instance of a {dl-blob-fc}{nbsp}{var-f}: . Let: ** _**L**_ be the value of the previously decoded unsigned integer field of which{nbsp}{var-f}'s `length-field-location` indicates the location. ** {var-v} be an array of bytes of length{nbsp}__**L**__. . <> according to{nbsp}{var-f}. . Read _**L**_ bytes of data from{nbsp}{var-s} at the offset{nbsp}{var-o} as{nbsp}{var-v}. . Set {var-o} to {var-o}{nbsp}pass:[+]{nbsp}__**L**__{nbsp}{times}{nbsp}8. {var-v} is the decoded BLOB value. [[struct-field-dec]] ==== Structure field decoding procedure To decode an instance of a {struct-fc}{nbsp}{var-f}: . Let: ** _**M**_ be {var-f}'s `members` property. ** {var-v} be an empty structure. . <> according to{nbsp}{var-f}. . For each member class{nbsp}__**MC**__ of{nbsp}__**M**__: .. Let: *** _**MF**_ be __**MC**__'s `field-class` property. *** _**MN**_ be __**MC**__'s `name` property. .. <> one instance of{nbsp}__**MF**__, appending the resulting value as a member of{nbsp}{var-v} named{nbsp}__**MN**__. {var-v} is the decoded value. [[sl-array-field-dec]] ==== {c-sl-array} field decoding procedure To decode an instance of a {sl-array-fc}{nbsp}{var-f}: . Let: ** _**L**_ be {var-f}'s `length` property. ** _**EF**_ be {var-f}'s `element-field-class` property. ** _**I**_ be an unsigned integer initialized to 0. ** {var-v} be an array of values of length{nbsp}__**L**__. . <> according to{nbsp}{var-f}. . While __**I**__{nbsp}<{nbsp}__**L**__: .. <> one instance of{nbsp}__**EF**__ as element{nbsp}__**I**__ of{nbsp}{var-v}. .. Set _**I**_ to __**I**__{nbsp}pass:[+]{nbsp}1. {var-v} is the decoded value. [[dl-array-field-dec]] ==== {c-dl-array} field decoding procedure To decode an instance of a {dl-array-fc}{nbsp}{var-f}: . Let: ** _**L**_ be the value of the previously decoded unsigned integer field of which{nbsp}{var-f}'s `length-field-location` indicates the location. ** _**EF**_ be {var-f}'s `element-field-class` property. ** _**I**_ be an unsigned integer initialized to 0. ** {var-v} be an array of values of length{nbsp}__**L**__ . <> according to{nbsp}{var-f}. . While __**I**__{nbsp}<{nbsp}__**L**__: .. <> one instance of{nbsp}__**EF**__ as element{nbsp}__**I**__ of{nbsp}{var-v}. .. Set _**I**_ to __**I**__{nbsp}pass:[+]{nbsp}1. {var-v} is the decoded value. [[opt-field-dec]] ==== Optional field decoding procedure To decode an instance of an {opt-fc}{nbsp}{var-f}: . Let: ** _**SEL**_ be the value of the previously decoded <> or <> field of which{nbsp}{var-f}'s `selector-field-location` indicates the location. ** _**OF**_ be {var-f}'s `field-class` property. ** {var-v} be an optional value, initially not set. . If the class of{nbsp}__**SEL**__ is a {fl-bool-fc} and _**SEL**_ is _true_, then: ** <> one instance of{nbsp}__**OF**__ as{nbsp}{var-v}. + Else, if _**SEL**_ is an element of any <> of{nbsp}{var-f}'s `selector-field-ranges` property, then: ** <> one instance of{nbsp}__**OF**__ as{nbsp}{var-v}. If {var-v} is set, then {var-v} is the decoded value. Otherwise, there's no decoded value. [[var-field-dec]] ==== Variant field decoding procedure To decode an instance of an {var-fc}{nbsp}{var-f}: . Let: ** _**SEL**_ be the value of the previously decoded <> field of which{nbsp}{var-f}'s `selector-field-location` indicates the location. ** _**OPTS**_ be {var-f}'s `options` property. ** _**OF**_ be the `field-class` property of the <>{nbsp}__**OPT**__ of{nbsp}__**OPTS**__ of which{nbsp}__**SEL**__ is an element of any <> of{nbsp}__**OPT**__'s `selector-field-ranges` property. ** {var-v} be a value. . <> one instance of{nbsp}__**OF**__ as{nbsp}{var-v}. {var-v} is the decoded value. _______________________________________________ tracecompass-dev mailing list tracecompass-dev@eclipse.org To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/tracecompass-dev