[lttng-dev] major variations in perfomance figures

[lttng-dev] major variations in perfomance figures

[Jesper Derehag]

An HTML attachment was scrubbed...
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20150403/dd17dd51/attachment.html>

[lttng-dev] major variations in perfomance figures

[Jérémie Galarneau]

On Tue, Mar 31, 2015 at 3:46 PM, Wolfgang Rostek <wolfgang.rostek at gmx.de> wrote:
> Hi all,
>
> I did a first perfomance test extending the demo example
> by tracing 30 simple integers in a loop.
>
> For a similar CPU (i5 quad core 2.8GHz) I saw values
> mentioned around 250ns in the forum.
>
> I've tried the test several time and could come down
> to about 350ns. The machine was almost idle for all
> tests.
>
> What makes me wonder is a large variation with frequent
> runs giving me 750-800ns.
>
> Not the absolute time but the variations isn't clear
> to me. From my understanding the caller path is more
> or less straight to shared memory. Why do different
> runs show more than double the time for the traces?

Good question. Can you share the benchmark's code?
Also, make sure frequency scaling is disabled on your system; it could
explain these kinds of variations.

J?r?mie

>
> Wolfgang R.
>
>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev



-- 
J?r?mie Galarneau
EfficiOS Inc.
http://www.efficios.com

[lttng-dev] major variations in perfomance figures

[Wolfgang Rostek]

Hi J?r?mie,

thanks for your feedback.

I've added only a loop to the end of the demo programm.

Frequence scaling I can't imagine on my dektop machine. 
It is a jump between 350ns and 800ns. In the loop below
all the events of one test run are of equal time distance, 
either slow or fast.

Wolfgang R.


https://github.com/lttng/lttng-ust/blob/master/doc/examples/demo/demo.c
...
tracepoint(ust_tests_demo, starting, 123);
for (i = 0; i < 5; i++) {
netint = htonl(i);
tracepoint(ust_tests_demo2, loop, i, netint, values,
text, strlen(text), dbl, flt);
}
tracepoint(ust_tests_demo, done, 456);
tracepoint(ust_tests_demo3, done, 42);

for (i = 0; i < 300; i++) {
if(i % 10 == 0) {
tracepoint(ust_tests_demo, done, i);         <=========
}
}

fprintf(stderr, " done.\n");
return 0;
}

> Gesendet: Freitag, 03. April 2015 um 16:22 Uhr
> Von: "J?r?mie Galarneau" <jeremie.galarneau at efficios.com>
> An: "Wolfgang Rostek" <wolfgang.rostek at gmx.de>
> Cc: "lttng-dev at lists.lttng.org" <lttng-dev at lists.lttng.org>
> Betreff: Re: [lttng-dev] major variations in perfomance figures
>
> On Tue, Mar 31, 2015 at 3:46 PM, Wolfgang Rostek <wolfgang.rostek at gmx.de> wrote:
> > Hi all,
> >
> > I did a first perfomance test extending the demo example
> > by tracing 30 simple integers in a loop.
> >
> > For a similar CPU (i5 quad core 2.8GHz) I saw values
> > mentioned around 250ns in the forum.
> >
> > I've tried the test several time and could come down
> > to about 350ns. The machine was almost idle for all
> > tests.
> >
> > What makes me wonder is a large variation with frequent
> > runs giving me 750-800ns.
> >
> > Not the absolute time but the variations isn't clear
> > to me. From my understanding the caller path is more
> > or less straight to shared memory. Why do different
> > runs show more than double the time for the traces?
> 
> Good question. Can you share the benchmark's code?
> Also, make sure frequency scaling is disabled on your system; it could
> explain these kinds of variations.
> 
> J?r?mie
> 
> >
> > Wolfgang R.
> >
> >
> >
> > _______________________________________________
> > lttng-dev mailing list
> > lttng-dev at lists.lttng.org
> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> 
> 
> 
> -- 
> J?r?mie Galarneau
> EfficiOS Inc.
> http://www.efficios.com
>

[lttng-dev] major variations in perfomance figures

[Jérémie Galarneau]

On Fri, Apr 3, 2015 at 10:47 AM, Wolfgang Rostek <wolfgang.rostek at gmx.de> wrote:
> Hi J?r?mie,
>
> thanks for your feedback.
>
> I've added only a loop to the end of the demo programm.
>
> Frequence scaling I can't imagine on my dektop machine.
> It is a jump between 350ns and 800ns. In the loop below
> all the events of one test run are of equal time distance,
> either slow or fast.

Frequency scaling can still definitely play a role, even on a desktop machine.
Also, Core i5 chips support Turbo Boost which should be disabled to
perform benchmarks.

I'm guessing you get your time measurements from the timestamps in the
trace?

J?r?mie

>
> Wolfgang R.
>
>
> https://github.com/lttng/lttng-ust/blob/master/doc/examples/demo/demo.c
> ...
> tracepoint(ust_tests_demo, starting, 123);
> for (i = 0; i < 5; i++) {
> netint = htonl(i);
> tracepoint(ust_tests_demo2, loop, i, netint, values,
> text, strlen(text), dbl, flt);
> }
> tracepoint(ust_tests_demo, done, 456);
> tracepoint(ust_tests_demo3, done, 42);
>
> for (i = 0; i < 300; i++) {
> if(i % 10 == 0) {
> tracepoint(ust_tests_demo, done, i);         <=========
> }
> }
>
> fprintf(stderr, " done.\n");
> return 0;
> }
>
>> Gesendet: Freitag, 03. April 2015 um 16:22 Uhr
>> Von: "J?r?mie Galarneau" <jeremie.galarneau at efficios.com>
>> An: "Wolfgang Rostek" <wolfgang.rostek at gmx.de>
>> Cc: "lttng-dev at lists.lttng.org" <lttng-dev at lists.lttng.org>
>> Betreff: Re: [lttng-dev] major variations in perfomance figures
>>
>> On Tue, Mar 31, 2015 at 3:46 PM, Wolfgang Rostek <wolfgang.rostek at gmx.de> wrote:
>> > Hi all,
>> >
>> > I did a first perfomance test extending the demo example
>> > by tracing 30 simple integers in a loop.
>> >
>> > For a similar CPU (i5 quad core 2.8GHz) I saw values
>> > mentioned around 250ns in the forum.
>> >
>> > I've tried the test several time and could come down
>> > to about 350ns. The machine was almost idle for all
>> > tests.
>> >
>> > What makes me wonder is a large variation with frequent
>> > runs giving me 750-800ns.
>> >
>> > Not the absolute time but the variations isn't clear
>> > to me. From my understanding the caller path is more
>> > or less straight to shared memory. Why do different
>> > runs show more than double the time for the traces?
>>
>> Good question. Can you share the benchmark's code?
>> Also, make sure frequency scaling is disabled on your system; it could
>> explain these kinds of variations.
>>
>> J?r?mie
>>
>> >
>> > Wolfgang R.
>> >
>> >
>> >
>> > _______________________________________________
>> > lttng-dev mailing list
>> > lttng-dev at lists.lttng.org
>> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>
>>
>> --
>> J?r?mie Galarneau
>> EfficiOS Inc.
>> http://www.efficios.com
>>



-- 
J?r?mie Galarneau
EfficiOS Inc.
http://www.efficios.com

[lttng-dev] major variations in perfomance figures

[Wolfgang Rostek]

...
> > Frequence scaling I can't imagine on my dektop machine.
> > It is a jump between 350ns and 800ns. In the loop below
> > all the events of one test run are of equal time distance,
> > either slow or fast.
> 
> Frequency scaling can still definitely play a role, even on a desktop machine.
> Also, Core i5 chips support Turbo Boost which should be disabled to
> perform benchmarks.
> 
> I'm guessing you get your time measurements from the timestamps in the
> trace?
Yes

I'm not hunting for a detailed benchmark. Only such a large variance makes
me thoughtful. For such a test I did expect some 10-30% but not this more 
than 200%.

Wolfgang R.