From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id f7L0JWzRmWBkLgAAWB0awg (envelope-from ) for ; Mon, 10 May 2021 20:35:56 -0400 Received: by simark.ca (Postfix, from userid 112) id 93F8C1F11C; Mon, 10 May 2021 20:35:56 -0400 (EDT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HTML_MESSAGE,MAILING_LIST_MULTI,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.2 Received: from lists.lttng.org (lists.lttng.org [167.114.26.123]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id 046441E783 for ; Mon, 10 May 2021 20:35:55 -0400 (EDT) Received: from lists-lttng01.efficios.com (localhost [IPv6:::1]) by lists.lttng.org (Postfix) with ESMTP id 4FfJqt37wPz1lwm; Mon, 10 May 2021 20:35:54 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lists.lttng.org; s=default; t=1620693354; bh=/mAiof/xbzIz11By9HlEIPSrBHC85axvwS6EL7/k4rg=; h=References:In-Reply-To:Date:To:Cc:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=kfSfTCiWx4E3/UR0x+LG0Yh8qI+wGMb0VWlJRlmm+0qcYyVLN5/h8DvqReHaZXvYO jcOT9ehnhQZOcknHdI2goQYbrIvMuBxb6WqqNB+okUDgjoE1vxQ0ZrDfH95Yua0uP/ FimlHIurFRyzFoAG8z8ZrTbwIjOAyNzY6N1xfTwkpn5bv7QOAaz6LHPXwk61VqxtK0 wmWcQXm9mSe44NIEjCBP6cVXteaolbzSQUV4BX6fV5+/zSrLqCFIdw5ZyCRvRGp4yo P8ol5ZN/qxpUu1AxXvJA/MdRKpZXwpB4xsRmlNNjueQHZ3ynFuICJS0+w0mhEad3Dn FmlGtzp/bb7Xg== Received: from mail-ot1-x335.google.com (mail-ot1-x335.google.com [IPv6:2607:f8b0:4864:20::335]) by lists.lttng.org (Postfix) with ESMTPS id 4FfJqr72Jpz1lTV for ; Mon, 10 May 2021 20:35:52 -0400 (EDT) Received: by mail-ot1-x335.google.com with SMTP id g15-20020a9d128f0000b02902a7d7a7bb6eso16080693otg.9 for ; Mon, 10 May 2021 17:35:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=FeQTQM19w6yFICfYEZf984DSjs5yCgadCxvhMnngCIc=; b=I8WDu3l9yBhFQEPXo1qvWjElxlZpfm5rMhbTG4Eo+MREbd8ogsMB384IaBRaZ31LA6 rwFbrNJxu9FpMnfP866d4mi9LzYqb1Ik0jL+UzkCYaG5ayfTQIcFxMhlt9vg/80Pv4SQ z6PBmUi7E3tV6m9ta07JGYNLDkamCxfmUxdZ+ynmuqf2lct2qPQJgOxd52RGZa05cZmL r/tLtVIxm9qy/Vqfl+O7vAykEvjvftkzv4ansc+0NBeXPX0xY9YCCPN9JLYKgO88CgHJ 8nvsdL+w1FOL2TW9xaqMQ3wXF5434+NZ+c624BhIFwaW3XpoZoguvPnn5KQTQ/TeQLwb ZERQ== X-Gm-Message-State: AOAM5330/fIjrXDGhcR9u0dAuL3e/l/TtyvdLsVRJ55brSDJreuKvlVa NhdqYB3x0NazB8wblzmxEUxzzUgrMQZ6PmMQ05E= X-Google-Smtp-Source: ABdhPJyVPgpuFi82hU8mdzOOiW5zShUHcyXY79a1CkGH12VI7HMFTaa3qq6FqEPMfyeNJj2XZbd6FRrR5PuF3aLj9l8= X-Received: by 2002:a9d:1d45:: with SMTP id m63mr2859434otm.302.1620693352136; Mon, 10 May 2021 17:35:52 -0700 (PDT) MIME-Version: 1.0 References: <20210507180532.GA1227214@joraj-alpa> In-Reply-To: <20210507180532.GA1227214@joraj-alpa> Date: Mon, 10 May 2021 17:35:41 -0700 Message-ID: To: Jonathan Rajotte-Julien Cc: lttng-dev@lists.lttng.org Subject: Re: [lttng-dev] lttng live event loss with babeltrace2 X-BeenThere: lttng-dev@lists.lttng.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: LTTng development list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Eqbal via lttng-dev Reply-To: Eqbal Content-Type: multipart/mixed; boundary="===============5401010722200824559==" Errors-To: lttng-dev-bounces@lists.lttng.org Sender: "lttng-dev" --===============5401010722200824559== Content-Type: multipart/alternative; boundary="0000000000008b42a405c203132d" --0000000000008b42a405c203132d Content-Type: text/plain; charset="UTF-8" Hi, I ran my test with the new changes and it seems to improve the lossiness but not eliminate it. For example it seems in the hello world example that if the application exits too quickly after emitting the tracepoint, we might still lose some events in the consumer even though they are present in the relayd output folder. I don't quite understand why that is. Would love some explanations on the various timing windows where we have a potential to lose the events on the consumer side. Thanks, Eqbal On Fri, May 7, 2021 at 11:05 AM Jonathan Rajotte-Julien < jonathan.rajotte-julien@efficios.com> wrote: > Hi, > > See > https://github.com/lttng/lttng-tools/commit/c876d657fb08a91ca7c907b92f1b7604aee664ee > . > > We would appreciate your feedback on this if possible based on your use > case. > > Cheers > > On Mon, May 03, 2021 at 05:05:10PM -0700, Eqbal via lttng-dev wrote: > > Hi, > > > > I have a lttng live session trace consumer application using > libbabeltrace2 > > where I create a graph to consume lttng live session traces and output to > > another sink. I am running the graph in a loop at some polling interval > as > > long as I get BT_GRAPH_RUN_STATUS_AGAIN status. What I am noticing is > that > > if my polling interval is large enough I tend to lose either all or some > of > > the events. I experimented with various polling intervals and it seems if > > the polling interval is less than *DELAYUS *from "lttng-create > > --live=DELAYUS" option then I am able to get all the events, otherwise I > > tend to lose events. > > > > Here are the steps I follow: > > 1. start session daemon and relay daemon > > 2. create a live session (with default delay of 1s), enable events and > start > > 3. Start my application (hello world example from lttng docs) > > 4. Start the consumer application built using libbabeltrace that connects > > to the live session > > > > I noticed that the events are actually persisted in the ~/lttng-traces by > > the relay daemon, but it does not reach babeltrace consumer application. > I > > have noticed the same behavior with babeltrace2 cli. > > > > I would like to understand what is the reason for such behavior and if > > playing with the polling interval in relation to the DELAYUS value is the > > right thing to do. > > > > Thanks, > > Eqbal > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev@lists.lttng.org > > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > -- > Jonathan Rajotte-Julien > EfficiOS > --0000000000008b42a405c203132d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

I ran my test with the n= ew changes and it seems to improve the lossiness but not eliminate it. For = example it seems in the hello world example that if the application exits t= oo quickly after emitting the tracepoint, we might still lose some events i= n the consumer even though they are present in the relayd output folder. I = don't quite understand why that is. Would love some explanations on the= various timing windows where we have a potential to lose the events on the= consumer side.

Thanks,
Eqbal
<= /div>
O= n Fri, May 7, 2021 at 11:05 AM Jonathan Rajotte-Julien <jonathan.rajotte-julien@efficios.co= m> wrote:
Hi,

See https://gith= ub.com/lttng/lttng-tools/commit/c876d657fb08a91ca7c907b92f1b7604aee664ee .

We would appreciate your feedback on this if possible based on your use cas= e.

Cheers

On Mon, May 03, 2021 at 05:05:10PM -0700, Eqbal via lttng-dev wrote:
> Hi,
>
> I have a lttng live session trace consumer application using libbabelt= race2
> where I create a graph to consume lttng live session traces and output= to
> another sink. I am running the graph in a loop at some polling interva= l as
> long as I get BT_GRAPH_RUN_STATUS_AGAIN status. What I am noticing is = that
> if my polling interval is large enough I tend to lose either all or so= me of
> the events. I experimented with various polling intervals and it seems= if
> the polling interval is less than *DELAYUS *from "lttng-create > --live=3DDELAYUS" option then I am able to get all the events, ot= herwise I
> tend to lose events.
>
> Here are the steps I follow:
> 1. start session daemon and relay daemon
> 2. create a live session (with default delay of 1s), enable events and= start
> 3. Start my application (hello world example from lttng docs)
> 4. Start the consumer application built using libbabeltrace that conne= cts
> to the live session
>
> I noticed that the events are actually persisted in the ~/lttng-traces= by
> the relay daemon, but it does not reach babeltrace consumer applicatio= n. I
> have noticed the same behavior with babeltrace2 cli.
>
> I would like to understand what is the reason for such behavior and if=
> playing with the polling interval in relation to the DELAYUS value is = the
> right thing to do.
>
> Thanks,
> Eqbal

> _______________________________________________
> lttng-dev mailing list
> lttng-d= ev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailm= an/listinfo/lttng-dev


--
Jonathan Rajotte-Julien
EfficiOS
--0000000000008b42a405c203132d-- --===============5401010722200824559== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev --===============5401010722200824559==--