[ltt-dev] LTTng-UST vs SystemTap userspace tracing benchmarks

[mjw@redhat.com (Mark Wielaard)]

On Wed, 2011-02-16 at 20:55 +0000, Stefan Hajnoczi wrote:
> On Wed, Feb 16, 2011 at 6:50 PM, Roland McGrath <roland at redhat.com> wrote:
> > It's indeed the cast that what Systemtap uses today is a
> > sometimes-inserted normal breakpoint instruction, which is indeed a
> > software interrupt that requires kernel mediation.  When disabled, there
> > is as close to zero overhead as you can have, being a tiny placeholder
> > instruction sequence (currently just one nop), so the runtime overhead
> > is under a cycle and the i-cache pollution is the smallest possible unit
> > (one instruction, being just one byte on x86).
> 
> Thanks for the explanations everyone.
> 
> I remember that DTrace also uses the software breakpoint method for
> userspace probes.  I think the key reason they choose this method is
> that it is the least invasive and does not require target process
> cooperation.

Yes, it prevents slowing down the target process if no probes are ever
triggered (the normal case). Then it is as if no instrumentation was
ever inserted. And the software interrupt is just one implementation,
that is currently used. So you could also look at using dynamic patching
the marker probe location to insert a jump instruction to an in user
handler function (this might need a small tweak to the sdt.h markers so
they leave enough space for that). Then there would also be minimal
overhead when the probes are enabled.

Cheers,

Mark