From mboxrd@z Thu Jan 1 00:00:00 1970 From: matthew.khouzam@ericsson.com (Matthew Khouzam) Date: Fri, 4 Jan 2013 11:43:32 -0500 Subject: [lttng-dev] Unable to import big LTTng traces In-Reply-To: <50CA1992.9020706@voxpopuli.im> References: <50CA1992.9020706@voxpopuli.im> Message-ID: <50E706B4.7090509@ericsson.com> There is a limit to the actual size of the trace being imported depending on your ram, the parser reads an index and keeps it in memory. This requires approx 0.004% of the trace size using default settings. This means if you load a 1 tb trace, you need 400 mb of ram. 400 mb of ram needs about 16k of ram plus overhead so let's say 1-2 mb. Is it a kernel trace? if so then you're also building the state system at the same time and the statistics... this will take a short while. Normally under 2 minutes (I have a core 2 duo laptop with a 5400 rpm hdd and it takes 2min for 433 mb with no state system) , so a coffee/tea break, when you're back, you will have lightning fast views. :) Maybe we should put a more descriptive please wait message. On 12-12-13 01:08 PM, Alexandre Montplaisir wrote: > On 12-12-12 12:59 AM, Radhika Chowdary wrote: >> Hi >> >> I am using linuxtools plugins for importing LTTng traces. I have a trace >> folder of size 400MB and when I try to import it in the Traces folder it >> takes for ever to import it. At the bottom left corner I see Indexing... >> Events view gets populated soon but not others. > Hi, > > It is expected that the indexing phase takes a while for very large > trace. It could probably be optimized a bit too, since we haven't spent > much time on performance improvements. Good news is that once indexed, > it will be faster to reopen the same trace later on. > > We have not yet implemented a percentage progression monitor for the > indexing job. This is not trivial since it has to be implemented > differently depending on the trace type, but would definitely be nice to > have. > > If you use the latest master branch, the timestamps for the start and > end of the histogram are now correctly formatted, so it is possible to > know roughly where the indexing is at, provided you know the end time of > your trace. I'd suggest trying that out to make sure it's not stopping > at some point, which would be a bug. > >> Is there a limit on the size of traces that it can import? > There is no theoretical limit on the trace size that can be imported. It > is capped only by your patience ;) and the amount of available disk > space, since the index files can be quite big too. > > > Cheers, > Alex > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev