Hello,
 
I've recently profiled the latency of LTTng tracepoints on arm platforms,
using the follow sample program:
 
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
static inline uint64_t get_time_nsec(void)
{
        struct timespec ts;
 
        if (caa_unlikely(clock_gettime(CLOCK_MONOTONIC, &ts))) {
                ts.tv_sec = 0;
                ts.tv_nsec = 0;
        }
        return ((uint64_t) ts.tv_sec * 1000000000ULL) + ts.tv_nsec;
}

 
int main(int argc, char *argv[])
{
    unsigned int i;
    int tp_num = 0;
    uint64_t total_time = 0;
    uint64_t now, nowz;
 
    if (argc > 1) {
        sscanf (argv[1],"%d",&tp_num);
    }
    for (i = 0; i < tp_num; i++) {
        now = get_time_nsec();
        lttng_ust_tracepoint(hello_world, my_first_tracepoint,
                             i, "some_str");
        nowz = get_time_nsec();
        total_time += (nowz - now);
    }
 
    if (tp_num) {
        printf("---------------------------Average TP time is %"PRIu64"---------------------------\n", total_time / tp_num);      
    }
}
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 

I observed a big average latency variance on different platforms when tracing a high number (many thousands to millions) of tracepoints:
 
  • [platform 1] with CPU info running a linux kernel based on Buildroot (4.19.273 aarch64 GNU/Linux):
 
BogoMIPS        : 187.50
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer   : 0x41
CPU architecture: 8
CPU variant       : 0x0
CPU part  : 0xd08
CPU revision      : 3
 
    •  Saw an average latency of 2-3usec
  • [platform 2] with CPU info running a linux kernel based on Amazon Linux (4.14.294-220.533.amzn2.aarch64 aarch64 GNU/Linux):
 
BogoMIPS        : 243.75
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs
CPU implementer   : 0x41
CPU architecture: 8
CPU variant       : 0x3
CPU part  : 0xd0c
CPU revision      : 1
 
    • Saw an average latency of ~0.5usec
 
Are there any suggestions to root cause the high latency and potentially improve it on platform 1?
 
Thanks and best regards,
Anas.