[lttng-dev] scale numbers for LTTNG

[mathieu.desnoyers@efficios.com (Mathieu Desnoyers)]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2878 bytes --]

----- On May 26, 2016, at 8:23 AM, Vijay Anand <vjanandr85 at gmail.com> wrote: 

> Could anyone please let us know how do we go about this ?

> Regards,
> Vijay

> On Tue, May 24, 2016 at 11:31 AM, Vijay Anand < vjanandr85 at gmail.com > wrote:

>> Hello Folks,

>> We have been evaluating LTTNG for use in our production systems for user space
>> tracing. We have evaluated most of the features supported by LTTNG and very
>> much see a value add LTTNG brings into our debugging infrastructure.

>>     * Our current requirement is to trace Userspace programs running on linux.
>>     * Each of the linux processes define their own tracepoint providers.

Sounds like a good design. 

>>     *
>>     * We would like to trace event histories of each process independently.

Why ? You can always collect trace data into buffers shared across processes (per-uid buffers) 
and filter after the fact. 

>>     *
>>     * We could potentially have 1000s of such processes running simultaneously.

Especially with that many processes, having per-process buffers will degrade your cache 
locality. 

>>     *
>>    * We concluded on using a session/channel to trace one tracepoint provider
>>     corresponding to a unique process.

>>        * But I understand we could also create one system wide session and use channels
>>         to trace each of the providers. Either of the approaches seems to work for us.

Both approach will kill you cache locality with that many process. I don't recommend either 
of the two approaches you refer to above. You might want to consider sharing your buffers 
across processes. 

>>    * But upon evaluating I see that we could create only 25 active process-sessions
>>     and not traces from all the processes are logged.

We will need much more details on your tracing configuration setup (exact list of commands you 
issue to create your tracing sessions). 

Also please detail what you mean by " not traces from all the processes are logged". 
What commands do you issue, what is the exact setup, and what is the output 
you observe (exact listing) which leads you to reach this conclusion. 

Thanks, 

Mathieu 

>>        * Please note I have tried increasing the buffer size and the number of buffers.
>>         This doesn't help.
>>         * Each of the process trace 52 events at and interval of 1 second each.
>>     *
>> I have evaluated this with lttng in session,live and snapshot modes and I have
>> not been getting favourable.

>> Could you folks share the scale numbers that LTTNG supports, especially when it
>> comes to tracing user space programs ?

>> Regards,
>> Vijay
>> ​​

-- 
Mathieu Desnoyers 
EfficiOS Inc. 
http://www.efficios.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20160526/c7ad1eb7/attachment-0001.html>