[ltt-dev] lost events due to non consumed full buffers

[ltt-dev] lost events due to non consumed full buffers

[Akyurek, Ali (EXT)]

Hi all,

Below is a code snippet from ltt-relay.c of LTTng 0.9.10 running on the 2.6.22 kernel.
I got approx. 70000 lost events during tracing.
I've seen that the problem is non consumed full buffers, as i show with an arrow below.
This problem occurs when my user space application initializes itself.

What do you think about the problem of not consuming? What may cause this?

Thanks.

-----------------------------------------------------------------------
			reserve_commit_diff = SUBBUF_OFFSET(
				buf->chan->subbuf_size - local_read(
				&ltt_buf->commit_count[
						SUBBUF_INDEX(offset_begin,
						buf->chan)]), buf->chan);
			if (reserve_commit_diff == 0) {
				/* Next buffer not corrupted. */
				if (!ltt_channel->overwrite &&
					(SUBBUF_TRUNC(offset_begin, buf->chan)
					- SUBBUF_TRUNC(
					atomic_long_read(&ltt_buf->consumed),
					buf->chan))
					>= rchan->alloc_size) {
					/*
					 * We do not overwrite non consumed
					 * buffers and we are full : event
					 * is lost.
					 */
					/* printk("non_con_buf\n"); */
		==========>		local_inc(&ltt_buf->events_lost);
					return NULL;
				} else {
					/*
					 * next buffer not corrupted, we are
					 * either in overwrite mode or the
					 * buffer is not full. It's safe to
					 * write in this new subbuffer.
					 */
				}
			}
---------------------------------------------------------------------
 
Mit freundlichem Gruss / Best Regards
Ali G?nhan Aky?rek 
evosoft

[ltt-dev] lost events due to non consumed full buffers

[Mathieu Desnoyers]

* Akyurek, Ali (EXT) (ali.akyurek.ext at siemens.com) wrote:
> 
> Hi all,
> 
> Below is a code snippet from ltt-relay.c of LTTng 0.9.10 running on the 2.6.22 kernel.
> I got approx. 70000 lost events during tracing.
> I've seen that the problem is non consumed full buffers, as i show with an arrow below.

The problem is more exactly :

your consumer (lttd) is not fast enough to get the data from the writer
(the kernel).

Check that lttd is executing while you take your trace in hybrid and
normal tracing modes.

Other solutions are to use the "flight recorder" mode, the "hybrid"
mode, or to make your buffers larger. You might also want to export your
data through a NFS mount rather than writing it to disk if the disk is
the bottleneck.

Mathieu


> This problem occurs when my user space application initializes itself.
> 
> What do you think about the problem of not consuming? What may cause this?
> 
> Thanks.
> 
> -----------------------------------------------------------------------
> 			reserve_commit_diff = SUBBUF_OFFSET(
> 				buf->chan->subbuf_size - local_read(
> 				&ltt_buf->commit_count[
> 						SUBBUF_INDEX(offset_begin,
> 						buf->chan)]), buf->chan);
> 			if (reserve_commit_diff == 0) {
> 				/* Next buffer not corrupted. */
> 				if (!ltt_channel->overwrite &&
> 					(SUBBUF_TRUNC(offset_begin, buf->chan)
> 					- SUBBUF_TRUNC(
> 					atomic_long_read(&ltt_buf->consumed),
> 					buf->chan))
> 					>= rchan->alloc_size) {
> 					/*
> 					 * We do not overwrite non consumed
> 					 * buffers and we are full : event
> 					 * is lost.
> 					 */
> 					/* printk("non_con_buf\n"); */
> 		==========>		local_inc(&ltt_buf->events_lost);
> 					return NULL;
> 				} else {
> 					/*
> 					 * next buffer not corrupted, we are
> 					 * either in overwrite mode or the
> 					 * buffer is not full. It's safe to
> 					 * write in this new subbuffer.
> 					 */
> 				}
> 			}
> ---------------------------------------------------------------------
>  
> Mit freundlichem Gruss / Best Regards
> Ali G?nhan Aky?rek 
> evosoft
> 
> _______________________________________________
> ltt-dev mailing list
> ltt-dev at lists.casi.polymtl.ca
> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68