From mboxrd@z Thu Jan 1 00:00:00 1970 From: jeremie.galarneau@efficios.com (=?ISO-8859-1?Q?J=E9r=E9mie_Galarneau?=) Date: Tue, 4 Feb 2014 12:46:05 -0500 Subject: [lttng-dev] Human read-writeable format for CTF traces In-Reply-To: <52F124D3.70507@versatic.net> References: <2093535948.9680292.1391443217617.JavaMail.root@polymtl.ca> <52EFCC62.2070508@versatic.net> <814395381.18374.1391447805215.JavaMail.zimbra@efficios.com> <52EFCFD0.1010907@versatic.net> <52F124D3.70507@versatic.net> Message-ID: On Tue, Feb 4, 2014 at 12:35 PM, Genevi?ve Bastien wrote: > Hi Jeremie, > > Thanks for your reply. > > > On 02/04/2014 11:19 AM, J?r?mie Galarneau wrote: >> >> Hi Genevi?ve, >> >> I wonder if we are not overly complicating things here... The Python >> bindings seem to address your immediate testing concerns and will let >> you produce test traces with minimal effort. > > Do you have any information, links, examples how to use what exists in > python to generate traces with minimal effort? Our very immediate need is > actually to generate traces manually, because we don't need to know how to > script in python to do this ;-) and we can manually modify the events. But > since I have no idea (yet) what those python bindings do and how they do it, > then maybe I miss something here. > They are part of the current Babeltrace master branch and will be included in the next release. An example script is available under babeltrace/bindings/python/examples/ctf_writer.py. Let me know if you have any questions. Regards, J?r?mie > Thanks, > Genevi?ve > >> While I agree that it doesn't solve the problem of testing analysis on >> Windows, I must ask if this really is a primary concern at this point. >> >> I would personally start by writing tests in Python using the >> CTF-Writer bindings and then, as the analysis feature gains traction, >> work with users to determine the best testing strategy. I somehow >> doubt that having an external dependancy on Babeltrace would really be >> a problem to these power users... >> >> Sorry for the late reply, >> J?r?mie >> >> On Mon, Feb 3, 2014 at 12:20 PM, Genevi?ve Bastien >> wrote: >>> >>> On 02/03/2014 12:16 PM, Mathieu Desnoyers wrote: >>>> >>>> ----- Original Message ----- >>>>> >>>>> From: "Genevi?ve Bastien" >>>>> To: "Michel Dagenais" , "Mathieu Desnoyers" >>>>> >>>>> Cc: lttng-dev at lists.lttng.org >>>>> Sent: Monday, February 3, 2014 12:05:38 PM >>>>> Subject: Re: [lttng-dev] Human read-writeable format for CTF traces >>>>> >>>>> Ok, I'll wait for J?r?mie's answer for more details. As I said, my >>>>> concern is to have something fully standalone in TMF. But if one has >>>>> access to babeltrace and eventual plugins to read-write a CTF trace to >>>>> XML, then all the better. We could then import an XML generated by a >>>>> python script into TMF, edit it there and then use it to test analyses. >>>>> >>>>> All we have to settle on is the intermediate format that should be >>>>> used. >>>>> I'd go for XML because of the possibility to validate it and have >>>>> visual >>>>> editors. >>>> >>>> Michel's idea of going for Python seems even better to generate test >>>> suites. >>>> It would allow importing and combining test "patterns" very easily, thus >>>> allowing to create tests by construction without having to copy-paste >>>> huge >>>> XML files. >>>> >>>> I don't clearly see why having external dependencies on other tools >>>> for a TMF CI test suite would be an issue. What would be the main >>>> arguments >>>> for having all those tests stand-alone in TMF for the test-suite ? >>> >>> It is not just for test suite. XML-defined analysis will need test traces >>> as >>> well, and that is in main TMF, not in unit tests (one idea of the XML >>> analysis is to allow end-user to develop their own analysis without >>> writing >>> a single line of code or requiring the TMF development environment). And >>> the >>> user of these analysis and test traces may not have access to babeltrace >>> or >>> even to a Linux command line. >>> >>> Thanks, >>> Genevi?ve >>> >>>> Thanks, >>>> >>>> Mathieu >>>> >>>>> On 02/03/2014 11:00 AM, Michel Dagenais wrote: >>>>>>> >>>>>>> I would expect that the ctf writer API recently added to babeltrace >>>>>>> (currently in master branch), along with the Python bindings that >>>>>>> cover >>>>>>> trace read and write APIs, should allow you to implement things like: >>>>>>> >>>>>>> - A plugin to read a CTF trace, and output it in an intermediate >>>>>>> format >>>>>>> to facilitate edits (e.g. XML as you propose), >>>>>>> - A plugin to read this XML format and output a CTF trace. >>>>>> >>>>>> Yes, this would indeed be extremely helpful, in XML and/or JSON. >>>>>> >>>>>>> You could also generate the XML trace completely by hand if you like, >>>>>>> and >>>>>>> then convert it to CTF with the second plugin I'm relating to above. >>>>>> >>>>>> The likely scenario is to add a few events by hand. >>>>>> >>>>>>> Another possibility is that the XML description also allows >>>>>>> describing what the trace contains at a slightly higher level. For >>>>>>> instance, if you >>>>>>> have a periodic event happening for a certain amount of time, it >>>>>>> would >>>>>>> be described in XML, and then "generated" by the XML-to-CTF >>>>>>> converter. >>>>>> >>>>>> Do we want to describe this in XML or in Python? We could have "CTF" >>>>>> to >>>>>> "Python statements" generating XML. Then we could add loops by hand. >>>>>> We >>>>>> could also have CTF to XML, with hooks to merge Python generated >>>>>> events. >>>>> >>>>> Indeed being able to script a trace would be extremely helpful and >>>>> convert it either directly to CTF or to the intermediate format. Some >>>>> scenarios for unit test would be to script a custom trace then change a >>>>> few events for the test purpose, then either import it in TMF or >>>>> convert >>>>> it to CTF. >>>>> >>>>> Thanks, >>>>> Genevi?ve >>>>>> >>>>>> In addition, TMF may also want to offer similar functionality, an XML >>>>>> dump >>>>>> of events and an XML events reader. Indeed, TMF supports a few formats >>>>>> other than CTF. >>>>> >>>>> >>> >>> _______________________________________________ >>> lttng-dev mailing list >>> lttng-dev at lists.lttng.org >>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >> >> >> > -- J?r?mie Galarneau EfficiOS Inc. http://www.efficios.com