From mboxrd@z Thu Jan 1 00:00:00 1970 From: vjanandr85@gmail.com (Vijay Anand) Date: Mon, 30 May 2016 12:46:46 +0530 Subject: [lttng-dev] scale numbers for LTTNG In-Reply-To: <177841807.20574.1464291977964.JavaMail.zimbra@efficios.com> References: <177841807.20574.1464291977964.JavaMail.zimbra@efficios.com> Message-ID: Hi Mathieu, Jonathan, I have written python scripts to simulate our requirements. https://github.com/vjanandr/sampleP/tree/master/lttng Please refer to the README file for further details about the scripts. Mathieu, I understand your recommendation to use per UID buffers instead of per PID buffers. But I dont seem to understand your suggestion to use filters... >>>> Why ? You can always collect trace data into buffers shared across processes (per-uid buffers) and filter after the fact. <<<<< Did you mean to say write all traces to a file but filter them while reading based on maybe process ID ? Regards, Vijay On Fri, May 27, 2016 at 1:16 AM, Mathieu Desnoyers < mathieu.desnoyers at efficios.com> wrote: > > > ----- On May 26, 2016, at 8:23 AM, Vijay Anand > wrote: > > Could anyone please let us know how do we go about this ? > > Regards, > Vijay > > On Tue, May 24, 2016 at 11:31 AM, Vijay Anand > wrote: > >> Hello Folks, >> >> We have been evaluating LTTNG for use in our production systems for user >> space tracing. We have evaluated most of the features supported by LTTNG >> and very much see a value add LTTNG brings into our debugging >> infrastructure. >> >> - Our current requirement is to trace Userspace programs running on >> linux. >> - Each of the linux processes define their own tracepoint providers. >> >> > Sounds like a good design. > > >> - >> - We would like to trace event histories of each process >> independently. >> >> > Why ? You can always collect trace data into buffers shared across > processes (per-uid buffers) > and filter after the fact. > > >> - >> - We could potentially have 1000s of such processes running >> simultaneously. >> >> > Especially with that many processes, having per-process buffers will > degrade your cache > locality. > > >> - >> - We concluded on using a session/channel to trace one tracepoint >> provider corresponding to a unique process. >> - But I understand we could also create one system wide session >> and use channels to trace each of the providers. Either of the approaches >> seems to work for us. >> >> Both approach will kill you cache locality with that many process. I > don't recommend either > of the two approaches you refer to above. You might want to consider > sharing your buffers > across processes. > > >> - But upon evaluating I see that we could create only 25 active >> process-sessions and not traces from all the processes are logged. >> >> We will need much more details on your tracing configuration setup (exact > list of commands you > issue to create your tracing sessions). > > Also please detail what you mean by "not traces from all the processes > are logged". > What commands do you issue, what is the exact setup, and what is the output > you observe (exact listing) which leads you to reach this conclusion. > > Thanks, > > Mathieu > > >> - Please note I have tried increasing the buffer size and the number >> of buffers. This doesn't help. >> - Each of the process trace 52 events at and interval of 1 second >> each. >> - I have evaluated this with lttng in session,live and snapshot modes >> and I have not been getting favourable. >> >> Could you folks share the scale numbers that LTTNG supports, especially >> when it comes to tracing user space programs ? >> >> Regards, >> Vijay >> ​​ >> > > > -- > Mathieu Desnoyers > EfficiOS Inc. > http://www.efficios.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: