From mboxrd@z Thu Jan 1 00:00:00 1970 From: gbastien+lttng@versatic.net (=?ISO-8859-1?Q?Genevi=E8ve_Bastien?=) Date: Tue, 04 Feb 2014 13:50:58 -0500 Subject: [lttng-dev] Human read-writeable format for CTF traces In-Reply-To: References: <2093535948.9680292.1391443217617.JavaMail.root@polymtl.ca> <52EFCC62.2070508@versatic.net> <814395381.18374.1391447805215.JavaMail.zimbra@efficios.com> <52EFCFD0.1010907@versatic.net> <52F124D3.70507@versatic.net> Message-ID: <52F13692.9090703@versatic.net> By the way, after discussing with Julien, one thing I need to add to understand our point of view: In TMF, we use a custom XML format with the existing custom XML parser to parse an XML trace, so it's really straightforward to use those traces, we don't need to create a new parser for it. Genevi?ve On 02/04/2014 12:46 PM, J?r?mie Galarneau wrote: > On Tue, Feb 4, 2014 at 12:35 PM, Genevi?ve Bastien > wrote: >> Hi Jeremie, >> >> Thanks for your reply. >> >> >> On 02/04/2014 11:19 AM, J?r?mie Galarneau wrote: >>> Hi Genevi?ve, >>> >>> I wonder if we are not overly complicating things here... The Python >>> bindings seem to address your immediate testing concerns and will let >>> you produce test traces with minimal effort. >> Do you have any information, links, examples how to use what exists in >> python to generate traces with minimal effort? Our very immediate need is >> actually to generate traces manually, because we don't need to know how to >> script in python to do this ;-) and we can manually modify the events. But >> since I have no idea (yet) what those python bindings do and how they do it, >> then maybe I miss something here. >> > They are part of the current Babeltrace master branch and will be > included in the next release. An example script is available under > babeltrace/bindings/python/examples/ctf_writer.py. > Let me know if you have any questions. > > Regards, > J?r?mie > >> Thanks, >> Genevi?ve >> >>> While I agree that it doesn't solve the problem of testing analysis on >>> Windows, I must ask if this really is a primary concern at this point. >>> >>> I would personally start by writing tests in Python using the >>> CTF-Writer bindings and then, as the analysis feature gains traction, >>> work with users to determine the best testing strategy. I somehow >>> doubt that having an external dependancy on Babeltrace would really be >>> a problem to these power users... >>> >>> Sorry for the late reply, >>> J?r?mie >>> >>> On Mon, Feb 3, 2014 at 12:20 PM, Genevi?ve Bastien >>> wrote: >>>> On 02/03/2014 12:16 PM, Mathieu Desnoyers wrote: >>>>> ----- Original Message ----- >>>>>> From: "Genevi?ve Bastien" >>>>>> To: "Michel Dagenais" , "Mathieu Desnoyers" >>>>>> >>>>>> Cc: lttng-dev at lists.lttng.org >>>>>> Sent: Monday, February 3, 2014 12:05:38 PM >>>>>> Subject: Re: [lttng-dev] Human read-writeable format for CTF traces >>>>>> >>>>>> Ok, I'll wait for J?r?mie's answer for more details. As I said, my >>>>>> concern is to have something fully standalone in TMF. But if one has >>>>>> access to babeltrace and eventual plugins to read-write a CTF trace to >>>>>> XML, then all the better. We could then import an XML generated by a >>>>>> python script into TMF, edit it there and then use it to test analyses. >>>>>> >>>>>> All we have to settle on is the intermediate format that should be >>>>>> used. >>>>>> I'd go for XML because of the possibility to validate it and have >>>>>> visual >>>>>> editors. >>>>> Michel's idea of going for Python seems even better to generate test >>>>> suites. >>>>> It would allow importing and combining test "patterns" very easily, thus >>>>> allowing to create tests by construction without having to copy-paste >>>>> huge >>>>> XML files. >>>>> >>>>> I don't clearly see why having external dependencies on other tools >>>>> for a TMF CI test suite would be an issue. What would be the main >>>>> arguments >>>>> for having all those tests stand-alone in TMF for the test-suite ? >>>> It is not just for test suite. XML-defined analysis will need test traces >>>> as >>>> well, and that is in main TMF, not in unit tests (one idea of the XML >>>> analysis is to allow end-user to develop their own analysis without >>>> writing >>>> a single line of code or requiring the TMF development environment). And >>>> the >>>> user of these analysis and test traces may not have access to babeltrace >>>> or >>>> even to a Linux command line. >>>> >>>> Thanks, >>>> Genevi?ve >>>> >>>>> Thanks, >>>>> >>>>> Mathieu >>>>> >>>>>> On 02/03/2014 11:00 AM, Michel Dagenais wrote: >>>>>>>> I would expect that the ctf writer API recently added to babeltrace >>>>>>>> (currently in master branch), along with the Python bindings that >>>>>>>> cover >>>>>>>> trace read and write APIs, should allow you to implement things like: >>>>>>>> >>>>>>>> - A plugin to read a CTF trace, and output it in an intermediate >>>>>>>> format >>>>>>>> to facilitate edits (e.g. XML as you propose), >>>>>>>> - A plugin to read this XML format and output a CTF trace. >>>>>>> Yes, this would indeed be extremely helpful, in XML and/or JSON. >>>>>>> >>>>>>>> You could also generate the XML trace completely by hand if you like, >>>>>>>> and >>>>>>>> then convert it to CTF with the second plugin I'm relating to above. >>>>>>> The likely scenario is to add a few events by hand. >>>>>>> >>>>>>>> Another possibility is that the XML description also allows >>>>>>>> describing what the trace contains at a slightly higher level. For >>>>>>>> instance, if you >>>>>>>> have a periodic event happening for a certain amount of time, it >>>>>>>> would >>>>>>>> be described in XML, and then "generated" by the XML-to-CTF >>>>>>>> converter. >>>>>>> Do we want to describe this in XML or in Python? We could have "CTF" >>>>>>> to >>>>>>> "Python statements" generating XML. Then we could add loops by hand. >>>>>>> We >>>>>>> could also have CTF to XML, with hooks to merge Python generated >>>>>>> events. >>>>>> Indeed being able to script a trace would be extremely helpful and >>>>>> convert it either directly to CTF or to the intermediate format. Some >>>>>> scenarios for unit test would be to script a custom trace then change a >>>>>> few events for the test purpose, then either import it in TMF or >>>>>> convert >>>>>> it to CTF. >>>>>> >>>>>> Thanks, >>>>>> Genevi?ve >>>>>>> In addition, TMF may also want to offer similar functionality, an XML >>>>>>> dump >>>>>>> of events and an XML events reader. Indeed, TMF supports a few formats >>>>>>> other than CTF. >>>>>> >>>> _______________________________________________ >>>> lttng-dev mailing list >>>> lttng-dev at lists.lttng.org >>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >>> >>> > >