From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id ZCpqLCqw5WD6XwAAWB0awg (envelope-from ) for ; Wed, 07 Jul 2021 09:46:18 -0400 Received: by simark.ca (Postfix, from userid 112) id 9F3B21F1F2; Wed, 7 Jul 2021 09:46:18 -0400 (EDT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HTML_MESSAGE,MAILING_LIST_MULTI,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.2 Received: from lists.lttng.org (lists.lttng.org [167.114.26.123]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id 2F4641F1EE for ; Wed, 7 Jul 2021 09:46:17 -0400 (EDT) Received: from lists-lttng01.efficios.com (localhost [IPv6:::1]) by lists.lttng.org (Postfix) with ESMTP id 4GKggX26xSzSwB; Wed, 7 Jul 2021 09:46:16 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lists.lttng.org; s=default; t=1625665576; bh=yPUP0o8HBbUJnZAVTaMEQT0LTg+tS8J0TgQQnJa8L8s=; h=Date:To:Cc:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=Fuha6Gu3pJY81Da4GsJFH6ZQZXoMDBFiiPoWpoDV1CwkP9bbI2zPey4mBft09oSBD jPGgM9PAXZRuSoxneNJNhf5yuP8J6BLR5UW4bYxfB4eHvGZqFIhkY7317VA7r/M912 /lZfCDQOtJkiuWXV+uklJHgp3BI6MFRKvu9UDLzMrDPLF5EkckBJXl6DFDmhWcQLp9 oi4XGCXS5Su47Ukw0WeDo16pEfjS5hql/Xe9A5LMvm0vMWnPAZtWutt47EueED3GWk twBzLIyyNi5RdEWwJs1UK4UzEgUgB+g1xX12cXxCYZAXjo3hyyTM/gha+rNWWj4/lk bX5HcTaStja1g== Received: from mail.efficios.com (mail.efficios.com [167.114.26.124]) by lists.lttng.org (Postfix) with ESMTPS id 4GKggW3ZpSzSw9 for ; Wed, 7 Jul 2021 09:46:15 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mail.efficios.com (Postfix) with ESMTP id C12B234B921 for ; Wed, 7 Jul 2021 09:46:09 -0400 (EDT) Received: from mail.efficios.com ([127.0.0.1]) by localhost (mail03.efficios.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id rVMSW_kqVosq; Wed, 7 Jul 2021 09:46:05 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mail.efficios.com (Postfix) with ESMTP id C984634B919; Wed, 7 Jul 2021 09:46:05 -0400 (EDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.efficios.com C984634B919 X-Virus-Scanned: amavisd-new at efficios.com Received: from mail.efficios.com ([127.0.0.1]) by localhost (mail03.efficios.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id F8K6QoVRgrj4; Wed, 7 Jul 2021 09:46:05 -0400 (EDT) Received: from mail03.efficios.com (mail03.efficios.com [167.114.26.124]) by mail.efficios.com (Postfix) with ESMTP id B868F34B37C; Wed, 7 Jul 2021 09:46:05 -0400 (EDT) Date: Wed, 7 Jul 2021 09:46:05 -0400 (EDT) To: "zhenyu.ren" Cc: lttng-dev Message-ID: <1613735014.8733.1625665565618.JavaMail.zimbra@efficios.com> In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [167.114.26.124] X-Mailer: Zimbra 8.8.15_GA_4059 (ZimbraWebClient - FF89 (Linux)/8.8.15_GA_4059) Thread-Topic: what will happen during tracing if getcpu() doesn't return correct cpu number occasionally? Thread-Index: mHanKvsgQlE1eBDaRSpDFE0iDxnlXw== Subject: Re: [lttng-dev] what will happen during tracing if getcpu() doesn't return correct cpu number occasionally? X-BeenThere: lttng-dev@lists.lttng.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: LTTng development list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mathieu Desnoyers via lttng-dev Reply-To: Mathieu Desnoyers Content-Type: multipart/mixed; boundary="===============2187317392108577367==" Errors-To: lttng-dev-bounces@lists.lttng.org Sender: "lttng-dev" --===============2187317392108577367== Content-Type: multipart/alternative; boundary="=_88cef2e3-38e0-4c6a-ab13-a3b4d08e9f3c" --=_88cef2e3-38e0-4c6a-ab13-a3b4d08e9f3c Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit ----- On Jul 6, 2021, at 10:37 PM, lttng-dev wrote: > Hi, > I know lttng-ust tracepoint process uses per cpu lockless ringbuffer algorithm > for high performance so that it relies on getcpu() to return the cpu number on > which the app is running. > Unfortunately, I am working on arm that linux kernel does not support vdso > getcpu() implemention and one getcpu() will take 200ns!!! > My question is : > 1. do you have any advice for that? You might want to try wiring up the "rseq" system call in user-space to provide an accurate cpu_id field in a __rseq_abi TLS variable. It is always kept up to date by the kernel. The rseq system call is implemented on ARM. However the __rseq_abi TLS is a shared resource across libraries, and we have not agreed with glibc people on how exactly it must be shared within a process. > 2. If I implement a cache-version for getcpu()(just like getcpu() implemention > before kernel 2.6.23 ), what will happen during tracing process? You'd have to give more details on how this "cache-version" works. > Since use of the cache could speed getcpu() calls, at the cost that there was a > very small chance that the returned cpu number would be out of date, I am not > sure whether the "wrong" cpu number will result in the tracing app crashing? LTTng-UST always has to expect that it can be migrated at any point between getcpu and writes to per-cpu data. Therefore, it always relies on atomic operations when interacting with the ring buffer, and there is no expectation that it runs on the "right" CPU compared to the ring buffer data structure for consistency. Therefore, you won't experience crashes nor corruption even if the CPU number is wrong once in a while, as long as it belongs to the "possible CPUs". This behavior is selected by lttng's libringbuffer "RING_BUFFER_SYNC_GLOBAL" configuration option internally, as selected by lttng-ust. Note that the kernel tracer instead selects "RING_BUFFER_SYNC_PER_CPU", which is faster, but requires that preemption (or migration) be disabled between the "smp_processor_id()" and writes to the ring buffer per-cpu data structures. Thanks, Mathieu > Thanks > zhenyu.ren > _______________________________________________ > lttng-dev mailing list > lttng-dev@lists.lttng.org > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com --=_88cef2e3-38e0-4c6a-ab13-a3b4d08e9f3c Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

----- On Jul 6, 2021, at 10:37 PM, lttng= -dev <lttng-dev@lists.lttng.org> wrote:
Hi,=  
   I know lttng-u= st tracepoint process uses per cpu lockless ringbuffer algorithm for h= igh performance so that it relies on getcpu() to return the cpu number on w= hich the app is running.
  &n= bsp;Unfortunately, I am working on arm that linux kerne= l does not support vdso getcpu() implemention and one getcpu= () will take 200ns!!!
   = ;My question is :
   1. = do you have any advice for that?

You might want to try wiring up the "rseq" system call in user-spa= ce to provide an accurate cpu_id
field i= n a __rseq_abi TLS variable. It is always kept up to date by the kernel. Th= e rseq system call
is implemented on ARM= . However the __rseq_abi TLS is a shared resource across libraries, and
we have not agreed with glibc people on how= exactly it must be shared within a process.
=

   2. If I implement a cache-version for getcpu()(= just like getcpu() implemention before kernel 2.6.23 ), what will happen du= ring tracing process? 
You'd have= to give more details on how this "cache-version" works.


LTTng-UST always has = to expect that it can be migrated at any point between getcpu and writes to=
per-cpu data. Therefore, it always reli= es on atomic operations when interacting with the ring buffer,
and there is no expectation that it runs on the "rig= ht" CPU compared to the ring buffer data structure
=
for consistency. Therefore, you won't experience crashes nor cor= ruption even if the CPU number is
wrong once in a while, as long = as it belongs to the "possible CPUs".
This behavior is selected by lttng's libr= ingbuffer "RING_BUFFER_SYNC_GLOBAL" configuration
<= /div>
option internally, as selected by lttng-ust.

Note that the kernel tra= cer instead selects "RING_BUFFER_SYNC_PER_CPU", which is faster, but
requires that preemption (or migration) be dis= abled between the "smp_processor_id()" and writes to
the ring buf= fer per-cpu data structures.

Thanks,

Mathieu


Thanks
zhenyu.ren

_______________________________________________
lttn= g-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/= cgi-bin/mailman/listinfo/lttng-dev

--
Mathieu Desnoyers
EfficiOS Inc.http://www.efficios.com
--=_88cef2e3-38e0-4c6a-ab13-a3b4d08e9f3c-- --===============2187317392108577367== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev --===============2187317392108577367==--