From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id CGTAHk8zwV9ZUAAAWB0awg (envelope-from ) for ; Fri, 27 Nov 2020 12:11:43 -0500 Received: by simark.ca (Postfix, from userid 112) id 7BC071F0AB; Fri, 27 Nov 2020 12:11:43 -0500 (EST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,MAILING_LIST_MULTI,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.2 Received: from lists.lttng.org (lists.lttng.org [167.114.26.123]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id C82471E58D for ; Fri, 27 Nov 2020 12:11:42 -0500 (EST) Received: from lists-lttng01.efficios.com (localhost [IPv6:::1]) by lists.lttng.org (Postfix) with ESMTP id 4CjLl227Llz1PlJ; Fri, 27 Nov 2020 12:11:42 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lists.lttng.org; s=default; t=1606497102; bh=9n4wlY3qjxmaNDzAlAcaQjPmtuGOghQa2VtMfRp6vcI=; h=Date:To:Cc:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=yh6J0wzkUGhPPmsSg5oOTX16p42SD7+nmJ1xYAz6UzwJVViU4xp3Hftje5+gyW0Bt hswRl22auxlK9KDv1EYN/vV9843jpTMNhCQqcBGSk389zW72vlkhv5n6HiGnhnKWRx 4B3KMRqUeA0cazkpwMiLwkZluAGPGvogdjHv7K3R8N6FmwOygw5UeIqTgDHQTBcFJH FeCiKj8WlmfTMzLbUCAV/GMnkAKjTSeMZqdWqah2JVZ6koQi4x1WmIY7vKGOHPOh5N dv3oU91d3hem+tCMmZJ9yVcuWEuK6tMKEMMWCTpX+SSXTDVKrIKQS583/ewKziwSP0 pVjXcsvCRUniw== Received: from mail.efficios.com (mail.efficios.com [167.114.26.124]) by lists.lttng.org (Postfix) with ESMTPS id 4CjLl12ZcSz1PlH for ; Fri, 27 Nov 2020 12:11:41 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mail.efficios.com (Postfix) with ESMTP id 117F127CC7A; Fri, 27 Nov 2020 12:11:35 -0500 (EST) Received: from mail.efficios.com ([127.0.0.1]) by localhost (mail03.efficios.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id x9JyqKDhH6VJ; Fri, 27 Nov 2020 12:11:34 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mail.efficios.com (Postfix) with ESMTP id C5A3A27CA7D; Fri, 27 Nov 2020 12:11:34 -0500 (EST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.efficios.com C5A3A27CA7D X-Virus-Scanned: amavisd-new at efficios.com Received: from mail.efficios.com ([127.0.0.1]) by localhost (mail03.efficios.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id I763VJz7A2YX; Fri, 27 Nov 2020 12:11:34 -0500 (EST) Received: from mail03.efficios.com (mail03.efficios.com [167.114.26.124]) by mail.efficios.com (Postfix) with ESMTP id BBDF427CA7B; Fri, 27 Nov 2020 12:11:34 -0500 (EST) Date: Fri, 27 Nov 2020 12:11:34 -0500 (EST) To: Jonathan Rajotte-Julien Cc: =?utf-8?B?54aK5q+T5Y2O?= , lttng-dev Message-ID: <556721467.65922.1606497094670.JavaMail.zimbra@efficios.com> In-Reply-To: <828264314.65604.1606493063962.JavaMail.zimbra@efficios.com> References: <40a80cf.2b99.176086d97b1.Coremail.xiongyuhua@zju.edu.cn> <20201127140548.GB3947@joraj-alpa> <25e54ac4.4261.1760a553e88.Coremail.xiongyuhua@zju.edu.cn> <828264314.65604.1606493063962.JavaMail.zimbra@efficios.com> MIME-Version: 1.0 X-Originating-IP: [167.114.26.124] X-Mailer: Zimbra 8.8.15_GA_3975 (ZimbraWebClient - FF83 (Linux)/8.8.15_GA_3975) Thread-Topic: Some confusion about cpu usage of the lttng-consumerd process Thread-Index: yp5QYAUpKnzdSMpykx0c2/cDegrSAvEkMCgI Subject: Re: [lttng-dev] Some confusion about cpu usage of the lttng-consumerd process X-BeenThere: lttng-dev@lists.lttng.org X-Mailman-Version: 2.1.31 Precedence: list List-Id: LTTng development list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mathieu Desnoyers via lttng-dev Reply-To: Mathieu Desnoyers Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: lttng-dev-bounces@lists.lttng.org Sender: "lttng-dev" ----- On Nov 27, 2020, at 11:04 AM, lttng-dev lttng-dev@lists.lttng.org wrote: > Well we also want to know why! You will understand that albeit we develop lttng > we do not always have a quick and easy answer to all problems. Performance > related problem are always tricky. > And we also have to keep in mind that we do not necessarily optimize for low-cpu > usage on the lttng-consumerd side. That being said, we did optimize for low-cpu usage of lttng-consumerd for use-cases streaming to disk or to the network. However, the "live" mode was originally created for use-cases where only a few events per second would be emitted, and no such requirements were placed on performance. We can see today that its use has grown much beyond the few events per seconds, but then in those use-cases the live mode may not be the appropriate tool for the job then. We have introduced the "session rotation" feature as a more efficient alternative to the live mode. > We have to take a look at what "work" scale with the number of CPU on the > lttng-consumerd side. One such thing is the live timer which is fired on an > interval (default is 1s (1000000us)). > You could test this hypothesis by streaming the trace instead of using the live > feature. > lttng create --set-url .... Yes, I agree with Jonathan's recommendation: you should compare this cpu usage with that of the streaming mode of lttng by *not* using the "--live" option when creating the trace session. It will at least help identify whether consumerd also exhibits this cpu usage increase with number of cores in streaming mode, or if it is an expected additional overhead of periodically flushing more cpu buffers (because there are more cores) caused by the live timer. Thanks, Mathieu -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev