From mboxrd@z Thu Jan 1 00:00:00 1970 From: mathieu.desnoyers@efficios.com (Mathieu Desnoyers) Date: Tue, 31 Aug 2010 12:37:22 -0400 Subject: [ltt-dev] [RFC] Common Trace Format Requirements (v1.3) In-Reply-To: <20100831153650.GI3185@redhat.com> References: <20100831145030.GA18176@Krystal> <20100831153650.GI3185@redhat.com> Message-ID: <20100831163722.GA27279@Krystal> * Frank Ch. Eigler (fche at redhat.com) wrote: > Hi - > > On Tue, Aug 31, 2010 at 10:50:30AM -0400, Mathieu Desnoyers wrote: > > > [...] The goal of the present document is to propose a trace format > > that will suit the needs [...]. It starts by doing an overview of > > the trace format, tracer and trace analyzer requirements to consider > > for a Common Trace Format proposal. > > Was it your intent to limit this document to a listing of abstract > requirements, as opposed to proposing an actual file format? Yep. But I'm trying to state the requirements as clearly as possible so I can leave the justification out of the trace format proposal per se. I'm also only planning to do a proposal for a Linux-specific trace format, which can be seen as a single instance of the trace format model. > > > > 1) Architecture > > > This high-level model is meant to be an industry-wide, common model, > > fulfilling the tracing requirements. It is meant to be application-, > > architecture-, and language-agnostic. > > > [...] > > - Metadata [...] > > - Metadata description language not imposed by standard > > If the metadata is not given in a standard form, then how do envision > general trace analysis tools (those not hard-coded for some particular > trace source) working? We can have a metadata format selector at the beginning of the metadata section, with reserved IDs for metadata formats. We can think of a format generated natively by TRACE_EVENT(), a format generated in some sort of XML. The trace analyzer would need the metadata format parser in order to be able to read the trace. > > > > * Requirements on the Tracers > > > Higher-level tracer requirements that seem appropriate to support > > some of the trace format requirements stated above. [...] > > The list that follows here appear to be wish-list performance > characteristics of the tracing infrastructure ("make it go fast" and > "use good transports"). How does it benefit the tracing *format* > specification to enumerate such particulars here? Requirements like tracer speed influences the format in many ways, e.g.: - requirement for compactness leads to schemes where all information repetition should be eliminated. Thus the need for optional per-section context information. - requirement for speed and streaming leads to zero-copy implementations, which imply that the trace format should be written natively by the tracer. > > > > * Trace Analyzer Requirements > > [...] > > Such requirements are specific to some particular tracing task. It > would not make sense to identify these items as normative. For > example, a trace analyzer tool that can consume the standard format > should not be deemed to violate the standard, if it merely can't grok > 10GB files. I don't think this is the spirit in which I stated this. The idea here is to ensure that the trace format structure makes it *possible* for a trace analyzer to cope with huge traces, not to require that all trace analyzers do so. I'll update the RFC document according to your comments, Thanks! Mathieu > > > - FChE -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From mboxrd@z Thu Jan 1 00:00:00 1970 From: mathieu.desnoyers@efficios.com (Mathieu Desnoyers) Date: Tue, 31 Aug 2010 12:37:22 -0400 Subject: [ltt-dev] [RFC] Common Trace Format Requirements (v1.3) In-Reply-To: <20100831153650.GI3185@redhat.com> References: <20100831145030.GA18176@Krystal> <20100831153650.GI3185@redhat.com> Message-ID: <20100831163722.GA27279@Krystal> Message-ID: <20100831163722.2dSKSuqYzDrjnmYgB5pLe9okTEOlMmlA__SAmw-tn60@z> * Frank Ch. Eigler (fche at redhat.com) wrote: > Hi - > > On Tue, Aug 31, 2010 at 10:50:30AM -0400, Mathieu Desnoyers wrote: > > > [...] The goal of the present document is to propose a trace format > > that will suit the needs [...]. It starts by doing an overview of > > the trace format, tracer and trace analyzer requirements to consider > > for a Common Trace Format proposal. > > Was it your intent to limit this document to a listing of abstract > requirements, as opposed to proposing an actual file format? Yep. But I'm trying to state the requirements as clearly as possible so I can leave the justification out of the trace format proposal per se. I'm also only planning to do a proposal for a Linux-specific trace format, which can be seen as a single instance of the trace format model. > > > > 1) Architecture > > > This high-level model is meant to be an industry-wide, common model, > > fulfilling the tracing requirements. It is meant to be application-, > > architecture-, and language-agnostic. > > > [...] > > - Metadata [...] > > - Metadata description language not imposed by standard > > If the metadata is not given in a standard form, then how do envision > general trace analysis tools (those not hard-coded for some particular > trace source) working? We can have a metadata format selector at the beginning of the metadata section, with reserved IDs for metadata formats. We can think of a format generated natively by TRACE_EVENT(), a format generated in some sort of XML. The trace analyzer would need the metadata format parser in order to be able to read the trace. > > > > * Requirements on the Tracers > > > Higher-level tracer requirements that seem appropriate to support > > some of the trace format requirements stated above. [...] > > The list that follows here appear to be wish-list performance > characteristics of the tracing infrastructure ("make it go fast" and > "use good transports"). How does it benefit the tracing *format* > specification to enumerate such particulars here? Requirements like tracer speed influences the format in many ways, e.g.: - requirement for compactness leads to schemes where all information repetition should be eliminated. Thus the need for optional per-section context information. - requirement for speed and streaming leads to zero-copy implementations, which imply that the trace format should be written natively by the tracer. > > > > * Trace Analyzer Requirements > > [...] > > Such requirements are specific to some particular tracing task. It > would not make sense to identify these items as normative. For > example, a trace analyzer tool that can consume the standard format > should not be deemed to violate the standard, if it merely can't grok > 10GB files. I don't think this is the spirit in which I stated this. The idea here is to ensure that the trace format structure makes it *possible* for a trace analyzer to cope with huge traces, not to require that all trace analyzers do so. I'll update the RFC document according to your comments, Thanks! Mathieu > > > - FChE -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From mboxrd@z Thu Jan 1 00:00:00 1970 From: mathieu.desnoyers@efficios.com (Mathieu Desnoyers) Date: Tue, 31 Aug 2010 12:37:22 -0400 Subject: [ltt-dev] [RFC] Common Trace Format Requirements (v1.3) In-Reply-To: <20100831153650.GI3185@redhat.com> References: <20100831145030.GA18176@Krystal> <20100831153650.GI3185@redhat.com> Message-ID: <20100831163722.GA27279@Krystal> Message-ID: <20100831163722.ixZ7th6jJzRwMHqlhVKzBKDTuVRpdMA1U84lp4qRe5Q@z> * Frank Ch. Eigler (fche at redhat.com) wrote: > Hi - > > On Tue, Aug 31, 2010 at 10:50:30AM -0400, Mathieu Desnoyers wrote: > > > [...] The goal of the present document is to propose a trace format > > that will suit the needs [...]. It starts by doing an overview of > > the trace format, tracer and trace analyzer requirements to consider > > for a Common Trace Format proposal. > > Was it your intent to limit this document to a listing of abstract > requirements, as opposed to proposing an actual file format? Yep. But I'm trying to state the requirements as clearly as possible so I can leave the justification out of the trace format proposal per se. I'm also only planning to do a proposal for a Linux-specific trace format, which can be seen as a single instance of the trace format model. > > > > 1) Architecture > > > This high-level model is meant to be an industry-wide, common model, > > fulfilling the tracing requirements. It is meant to be application-, > > architecture-, and language-agnostic. > > > [...] > > - Metadata [...] > > - Metadata description language not imposed by standard > > If the metadata is not given in a standard form, then how do envision > general trace analysis tools (those not hard-coded for some particular > trace source) working? We can have a metadata format selector at the beginning of the metadata section, with reserved IDs for metadata formats. We can think of a format generated natively by TRACE_EVENT(), a format generated in some sort of XML. The trace analyzer would need the metadata format parser in order to be able to read the trace. > > > > * Requirements on the Tracers > > > Higher-level tracer requirements that seem appropriate to support > > some of the trace format requirements stated above. [...] > > The list that follows here appear to be wish-list performance > characteristics of the tracing infrastructure ("make it go fast" and > "use good transports"). How does it benefit the tracing *format* > specification to enumerate such particulars here? Requirements like tracer speed influences the format in many ways, e.g.: - requirement for compactness leads to schemes where all information repetition should be eliminated. Thus the need for optional per-section context information. - requirement for speed and streaming leads to zero-copy implementations, which imply that the trace format should be written natively by the tracer. > > > > * Trace Analyzer Requirements > > [...] > > Such requirements are specific to some particular tracing task. It > would not make sense to identify these items as normative. For > example, a trace analyzer tool that can consume the standard format > should not be deemed to violate the standard, if it merely can't grok > 10GB files. I don't think this is the spirit in which I stated this. The idea here is to ensure that the trace format structure makes it *possible* for a trace analyzer to cope with huge traces, not to require that all trace analyzers do so. I'll update the RFC document according to your comments, Thanks! Mathieu > > > - FChE -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: