[lttng-dev] About Lttng trace mode

[lttng-dev] About Lttng trace mode

[Thibault, Daniel]

Date: Mon, 3 Feb 2014 16:03:14 +0000 (UTC)
From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

>----- Original Message -----
>> From: "Daniel Thibault" <Daniel.Thibault@drdc-rddc.gc.ca>
>> Sent: Thursday, November 14, 2013 12:01:25 PM
>> 
>>  [...]
>>    or by the consumer.  In discard mode, while a sub-buffer is locked for
>>    consumer access, should the writers go around the set of sub-buffers and
>>  [...]
>
>The paragraph above is true, except for the part "while a sub-buffer is locked for consumer access". In discard mode, as soon as the writer is filling up all sub-buffers, and thus the >produced sub-buffers count reaches the consumed sub-buffer count minus the number of sub-buffers in the buffer (this is a buffer full situation), events are discarded. This is done >independently of whether or not the consumer is currently locking a sub-buffer for reading.

   Good, subtle point.  The sub-buffer is marked for consumption ("read-ready"), just not actively locked ("being read").

Daniel U. Thibault
Protection des syst?mes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber s?curit? pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la d?fense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Qu?bec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada
<http://www.valcartier.drdc-rddc.gc.ca/>

[lttng-dev] About Lttng trace mode

[Mathieu Desnoyers]

----- Original Message -----
> From: "Daniel Thibault" <Daniel.Thibault@drdc-rddc.gc.ca>
> To: lttng-dev at lists.lttng.org
> Cc: "XIANG Hao" <Hao.Xiang at alcatel-sbell.com.cn>
> Sent: Thursday, November 14, 2013 12:01:25 PM
> Subject: Re: [lttng-dev] About Lttng trace mode
> 
> -----Message d'origine-----
> Date: Thu, 14 Nov 2013 13:24:16 +0000
> From: XIANG Hao <Hao.Xiang@alcatel-sbell.com.cn>
> 
> >   Could Lttng support transmit trace to host PC in a "live mode" or "fly
> >   mode" now? So as to reduce the buffer size requirement on the target
> >   board.
> >
> >   BTW, I read some material mentioned:
> >   The discard mode is used by default. In this mode, new trace data
> >   arriving to a full buffer are discarded. In overwrite mode, the oldest
> >   trace data in the buffer are discarded.
> >   This mode is also known as flight-recorder mode. Note however, that this
> >   refers to the internal buffers, not to the trace log storage. The
> >   discarding of events is marked in
> >   the trace log and can be noticed when the trace is analyzed. In this way
> >   the user can interactively adjust his trace settings.
> >
> >   What is the difference between "internal buffers" and "trace log
> >   storage"? Could the trace log storage be located in the host PC?
> >   For example: the target board transmit the trace to Host PC via TCP/UDP,
> >   and only a very little internal flight-recorder buffer is required in
> >   the target device. Is it feasible?
> 
>    Live mode is nearly ready; expect it with version 2.4.
> 
>    The difference between internal buffers and trace log storage goes like
>    this: the event providers write into a memory buffer (the internal
>    buffers), and at the same time the consumer daemon(s) withdraw event
>    records from the buffer and commit them to permanent storage (trace log
>    storage).  The buffer is broken down into a number of sub-buffers, and
>    each sub-buffer can only be accessed either by the writers (collectively)
>    or by the consumer.  In discard mode, while a sub-buffer is locked for
>    consumer access, should the writers go around the set of sub-buffers and
>    come up behind the consumer, their request to write into the consumer's
>    sub-buffer will be denied, and they will then discard the event they
>    wanted to record.  The "not to the trace log storage" bit is there to
>    clarify that no event record may ever be lost from the trace files once
>    it has been written into them: they can neither be discarded nor
>    overwritten.  (Actually, using the enable-channel tracefile-size and
>    tracefile-count options, you *can* set up an on-disk flight recorder, but
>    that is a different scenario)

The paragraph above is true, except for the part "while a sub-buffer is locked
for consumer access". In discard mode, as soon as the writer is filling up all
sub-buffers, and thus the produced sub-buffers count reaches the consumed
sub-buffer count minus the number of sub-buffers in the buffer (this is a buffer
full situation), events are discarded. This is done independently of whether or
not the consumer is currently locking a sub-buffer for reading.

Thanks,

Mathieu

> 
>    LTTng can already transmit the trace events to a host machine.  Going back
>    to the example of the writers writing into a memory buffer while the
>    consumer withdraws events from it, the consumer can be instructed to
>    transmit the records to a TCP address instead of writing them to local
>    file storage.  At that address, a relay daemon receives the records and
>    writes them to file storage.  Example:
> 
> (on the host machine:)
> $ lttng-relayd &
> (on the traced machine:)
> $ lttng create mysession --set-url net://host
> (enable events and channels, etc.)
> $ lttng start
> (.)
> $ lttng destroy
> 
> Daniel U. Thibault
> Protection des syst?mes et contremesures (PSC) | Systems Protection &
> Countermeasures (SPC)
> Cyber s?curit? pour les missions essentielles (CME) | Mission Critical Cyber
> Security (MCCS)
> R & D pour la d?fense Canada - Valcartier (RDDC Valcartier) | Defence R&D
> Canada - Valcartier (DRDC Valcartier)
> 2459 route de la Bravoure
> Qu?bec QC  G3J 1X5
> CANADA
> Vox : (418) 844-4000 x4245
> Fax : (418) 844-4538
> NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
> Gouvernement du Canada | Government of Canada
> <http://www.valcartier.drdc-rddc.gc.ca/>
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> 

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[lttng-dev] About Lttng trace mode

[XIANG Hao]

Hi, Daniel

	Very clear, thank you very much for the very valuable information.

BR&Thanks
XiangHao



-----Original Message-----
From: Thibault, Daniel [mailto:Daniel.Thibault@drdc-rddc.gc.ca] 
Sent: 2013?11?15? 1:01
To: lttng-dev at lists.lttng.org
Cc: XIANG Hao
Subject: RE: [lttng-dev] About Lttng trace mode

-----Message d'origine-----
Date: Thu, 14 Nov 2013 13:24:16 +0000
From: XIANG Hao <Hao.Xiang@alcatel-sbell.com.cn>

>   Could Lttng support transmit trace to host PC in a "live mode" or "fly mode" now? So as to reduce the buffer size requirement on the target board.
>
>   BTW, I read some material mentioned:
>   The discard mode is used by default. In this mode, new trace data arriving to a full buffer are discarded. In overwrite mode, the oldest trace data in the buffer are discarded.
>   This mode is also known as flight-recorder mode. Note however, that this refers to the internal buffers, not to the trace log storage. The discarding of events is marked in
>   the trace log and can be noticed when the trace is analyzed. In this way the user can interactively adjust his trace settings.
>
>   What is the difference between "internal buffers" and "trace log storage"? Could the trace log storage be located in the host PC?
>   For example: the target board transmit the trace to Host PC via TCP/UDP, and only a very little internal flight-recorder buffer is required in the target device. Is it feasible?

   Live mode is nearly ready; expect it with version 2.4.

   The difference between internal buffers and trace log storage goes like this: the event providers write into a memory buffer (the internal buffers), and at the same time the consumer daemon(s) withdraw event records from the buffer and commit them to permanent storage (trace log storage).  The buffer is broken down into a number of sub-buffers, and each sub-buffer can only be accessed either by the writers (collectively) or by the consumer.  In discard mode, while a sub-buffer is locked for consumer access, should the writers go around the set of sub-buffers and come up behind the consumer, their request to write into the consumer's sub-buffer will be denied, and they will then discard the event they wanted to record.  The "not to the trace log storage" bit is there to clarify that no event record may ever be lost from the trace files once it has been written into them: they can neither be discarded nor overwritten.  (Actually, using the enable-channel tracefile-size and tracefile-count options, you *can* set up an on-disk flight recorder, but that is a different scenario)

   LTTng can already transmit the trace events to a host machine.  Going back to the example of the writers writing into a memory buffer while the consumer withdraws events from it, the consumer can be instructed to transmit the records to a TCP address instead of writing them to local file storage.  At that address, a relay daemon receives the records and writes them to file storage.  Example:

(on the host machine:)
$ lttng-relayd &
(on the traced machine:)
$ lttng create mysession --set-url net://host (enable events and channels, etc.) $ lttng start
(.)
$ lttng destroy

Daniel U. Thibault
Protection des syst?mes et contremesures (PSC) | Systems Protection & Countermeasures (SPC) Cyber s?curit? pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS) R & D pour la d?fense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Qu?bec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada <http://www.valcartier.drdc-rddc.gc.ca/>

[lttng-dev] About Lttng trace mode

[Thibault, Daniel]

-----Message d'origine-----
Date: Thu, 14 Nov 2013 13:24:16 +0000
From: XIANG Hao <Hao.Xiang@alcatel-sbell.com.cn>

>   Could Lttng support transmit trace to host PC in a "live mode" or "fly mode" now? So as to reduce the buffer size requirement on the target board.
>
>   BTW, I read some material mentioned:
>   The discard mode is used by default. In this mode, new trace data arriving to a full buffer are discarded. In overwrite mode, the oldest trace data in the buffer are discarded.
>   This mode is also known as flight-recorder mode. Note however, that this refers to the internal buffers, not to the trace log storage. The discarding of events is marked in
>   the trace log and can be noticed when the trace is analyzed. In this way the user can interactively adjust his trace settings.
>
>   What is the difference between "internal buffers" and "trace log storage"? Could the trace log storage be located in the host PC?
>   For example: the target board transmit the trace to Host PC via TCP/UDP, and only a very little internal flight-recorder buffer is required in the target device. Is it feasible?

   Live mode is nearly ready; expect it with version 2.4.

   The difference between internal buffers and trace log storage goes like this: the event providers write into a memory buffer (the internal buffers), and at the same time the consumer daemon(s) withdraw event records from the buffer and commit them to permanent storage (trace log storage).  The buffer is broken down into a number of sub-buffers, and each sub-buffer can only be accessed either by the writers (collectively) or by the consumer.  In discard mode, while a sub-buffer is locked for consumer access, should the writers go around the set of sub-buffers and come up behind the consumer, their request to write into the consumer's sub-buffer will be denied, and they will then discard the event they wanted to record.  The "not to the trace log storage" bit is there to clarify that no event record may ever be lost from the trace files once it has been written into them: they can neither be discarded nor overwritten.  (Actually, using the enable-channel tracefile-size and tracefile-count options, you *can* set up an on-disk flight recorder, but that is a different scenario)

   LTTng can already transmit the trace events to a host machine.  Going back to the example of the writers writing into a memory buffer while the consumer withdraws events from it, the consumer can be instructed to transmit the records to a TCP address instead of writing them to local file storage.  At that address, a relay daemon receives the records and writes them to file storage.  Example:

(on the host machine:)
$ lttng-relayd &
(on the traced machine:)
$ lttng create mysession --set-url net://host
(enable events and channels, etc.)
$ lttng start
(.)
$ lttng destroy

Daniel U. Thibault
Protection des syst?mes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber s?curit? pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la d?fense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Qu?bec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada
<http://www.valcartier.drdc-rddc.gc.ca/>

[lttng-dev] About Lttng trace mode

[XIANG Hao]

Hi, Dear Lttng Expert

         I am a beginner of the great tool: Lttng. We haven't so much target board memory for it. So I have some questions as below, could you please help me, thanks!

         Could Lttng support transmit trace to host PC in a "live mode" or "fly mode" now? So as to reduce the buffer size requirement on the target board.

         BTW, I read some material mentioned:
The discard mode is used by default. In this mode, new trace data arriving to a full buffer are discarded. In overwrite mode, the oldest trace data in the buffer are discarded. This mode is also known as flight-recorder mode. Note however, that this refers to the internal buffers, not to the trace log storage. The discarding of events is marked in the trace log and can be noticed when the trace is analyzed. In this way the user can interactively adjust his trace settings.
         What is the difference between "internal buffers" and "trace log storage"? Could the trace log storage be located in the host PC? For example: the target board transmit the trace to Host PC via TCP/UDP, and only a very little internal flight-recorder buffer is required in the target device. Is it feasible?

BR&Thanks
XiangHao

-------------------------------------------------------------------------------------------
ALCATEL-LUCENT SHANGHAI BELL
No.255 Guiqiao Road, Pudong, Shanghai, China
ZIP: 201206

OnNET: 2783.7051
Phone: +86.21.38563888.7051
Fax  : +86.21.38563400

Hao.Xiang at alcatel-sbell.com.cn<mailto:Hao.Xiang at alcatel-sbell.com.cn>
-------------------------------------------------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20131114/e62ac274/attachment.html>