[ltt-dev] [RFC patch 00/41] LTTng 0.105 core for Linux 2.6.27-rc9

[ltt-dev] [RFC patch 00/41] LTTng 0.105 core for Linux 2.6.27-rc9

[Mathieu Desnoyers]

Hi,

I spent the last 4-5 months working with the Fujitsu team at implementing the
tracer elements identified as goals at Kernel Summit 2008 and at the following
Plumber Conference. My idea was to incremententally adapt the LTTng tracer,
currently used in the industry and well tested, to those requirements.

I spent the last days rearranging/folding/inspecting the LTTng patchset
to prepare it for an LKML post. The version 0.105 in the LTTng git tree
corresponds to the patchset I am posting here. The said patchset will
only include the core features of LTTng, excluding the timestamping
infrastructure (trace clock) and excluding the instrumentation.

The corresponding git tree contains also the trace clock patches and the lttng
instrumentation. The trace clock is required to use the tracer, but it can be
used without the instrumentation : there is already a kprobes and userspace
event support included in this patchset.

This tracer exports binary data through buffers using splice(). The resulting
binary files can be parsed from userspace because the format string metadata is
exported in the files. The event set can be enhanced by adding tracepoints to
the kernel code and by creating probe modules, which connects callbacks to the
tracepoints and contain the format string metainformation. Those callbacks are
responsible for writing the data in the trace buffers. This separation between
the trace buffer format string and the tracepoints is done on purpose so the
core kernel instrumentation (tracepoints) is not exported to userspace, which
will make maintainance much easier.

The tree including the trace clock patches is available at :

git://git.kernel.org/pub/scm/linux/kernel/git/compudj/linux-2.6-lttng.git
branch : 2.6.29-rc7-lttng-0.105

Project website : http://www.lttng.org/

Information about how to install and use the tracer is available at :

http://ltt.polymtl.ca/svn/trunk/lttv/LTTngManual.html

The size of the LTTng core patchset is 41 patches. The diffstat details
as follow :

 include/linux/ltt-core.h                                  |   35 
 include/linux/ltt-relay.h                                 |  161 +
 include/linux/ltt-tracer.h                                |   43 
 include/linux/marker.h                                    |  121 
 kernel/marker.c                                           |  353 ++
 kernel/module.c                                           |   31 
 linux-2.6-lttng/Documentation/markers.txt                 |   17 
 linux-2.6-lttng/MAINTAINERS                               |    7 
 linux-2.6-lttng/Makefile                                  |    2 
 linux-2.6-lttng/arch/powerpc/kernel/traps.c               |    5 
 linux-2.6-lttng/arch/powerpc/platforms/cell/spufs/spufs.h |    6 
 linux-2.6-lttng/arch/sparc/Makefile                       |    2 
 linux-2.6-lttng/arch/x86/kernel/dumpstack.c               |    5 
 linux-2.6-lttng/arch/x86/mm/fault.c                       |    1 
 linux-2.6-lttng/fs/ext4/fsync.c                           |    8 
 linux-2.6-lttng/fs/ext4/ialloc.c                          |   17 
 linux-2.6-lttng/fs/ext4/inode.c                           |   79 
 linux-2.6-lttng/fs/ext4/mballoc.c                         |   71 
 linux-2.6-lttng/fs/ext4/mballoc.h                         |    2 
 linux-2.6-lttng/fs/ext4/super.c                           |    6 
 linux-2.6-lttng/fs/jbd2/checkpoint.c                      |    7 
 linux-2.6-lttng/fs/jbd2/commit.c                          |   12 
 linux-2.6-lttng/fs/pipe.c                                 |    5 
 linux-2.6-lttng/fs/select.c                               |   41 
 linux-2.6-lttng/fs/seq_file.c                             |   45 
 linux-2.6-lttng/fs/splice.c                               |    1 
 linux-2.6-lttng/include/linux/immediate.h                 |   94 
 linux-2.6-lttng/include/linux/kvm_host.h                  |   12 
 linux-2.6-lttng/include/linux/ltt-channels.h              |   94 
 linux-2.6-lttng/include/linux/ltt-core.h                  |   47 
 linux-2.6-lttng/include/linux/ltt-relay.h                 |  186 +
 linux-2.6-lttng/include/linux/ltt-tracer.h                |  731 ++++++
 linux-2.6-lttng/include/linux/ltt-type-serializer.h       |  107 
 linux-2.6-lttng/include/linux/marker.h                    |   16 
 linux-2.6-lttng/include/linux/module.h                    |    6 
 linux-2.6-lttng/include/linux/poll.h                      |    2 
 linux-2.6-lttng/include/linux/seq_file.h                  |   20 
 linux-2.6-lttng/include/trace/ext4.h                      |  129 +
 linux-2.6-lttng/include/trace/jbd2.h                      |   19 
 linux-2.6-lttng/init/Kconfig                              |    2 
 linux-2.6-lttng/kernel/kallsyms.c                         |    1 
 linux-2.6-lttng/kernel/marker.c                           |   12 
 linux-2.6-lttng/kernel/module.c                           |   32 
 linux-2.6-lttng/ltt/Kconfig                               |  130 +
 linux-2.6-lttng/ltt/Makefile                              |   15 
 linux-2.6-lttng/ltt/ltt-channels.c                        |  338 ++
 linux-2.6-lttng/ltt/ltt-core.c                            |  101 
 linux-2.6-lttng/ltt/ltt-filter.c                          |   66 
 linux-2.6-lttng/ltt/ltt-kprobes.c                         |  479 +++
 linux-2.6-lttng/ltt/ltt-marker-control.c                  |  265 ++
 linux-2.6-lttng/ltt/ltt-relay-alloc.c                     |  715 +++++
 linux-2.6-lttng/ltt/ltt-relay-locked.c                    | 1704 ++++++++++++++
 linux-2.6-lttng/ltt/ltt-serialize.c                       |  685 +++++
 linux-2.6-lttng/ltt/ltt-trace-control.c                   | 1061 ++++++++
 linux-2.6-lttng/ltt/ltt-tracer.c                          | 1210 +++++++++
 linux-2.6-lttng/ltt/ltt-type-serializer.c                 |   96 
 linux-2.6-lttng/ltt/ltt-userspace-event.c                 |  131 +
 linux-2.6-lttng/samples/markers/Makefile                  |    2 
 linux-2.6-lttng/samples/markers/marker-example.c          |    4 
 linux-2.6-lttng/samples/markers/probe-example.c           |   10 
 linux-2.6-lttng/samples/markers/test-multi.c              |  120 
 linux-2.6-lttng/virt/kvm/kvm_trace.c                      |   12 
 ltt/Kconfig                                               |   24 
 ltt/Makefile                                              |    2 
 ltt/ltt-relay-alloc.c                                     |   80 
 65 files changed, 9445 insertions(+), 398 deletions(-)


Comments are welcome.

Mathieu


-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[ltt-dev] [RFC patch 01/41] LTTng - core header

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-core-header.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/4e11a2c3/attachment.txt>

[ltt-dev] [RFC patch 01/41] LTTng - core header

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-core-header.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/4e11a2c3/attachment-0001.txt>

[ltt-dev] [RFC patch 01/41] LTTng - core header

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-core-header.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/4e11a2c3/attachment.asc>

[ltt-dev] [RFC patch 01/41] LTTng - core header

[Steven Rostedt]

On Thu, 5 Mar 2009, Mathieu Desnoyers wrote:

> Contains the structures required by the builtin part of the LTTng tracer.


Also note, it is better to push the core changes needed first. This way if 
something comes up, you can make your changes to your code before it gets 
into the kernel. This has been the approach I try to use. I send out patch 
series that modify the core kernel first, see what the feed back is, get 
them accepted, before continuing with further patches.

Thanks,

-- Steve


> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca>
> ---
>  include/linux/ltt-core.h |   47 +++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 47 insertions(+)
> 
> Index: linux-2.6-lttng/include/linux/ltt-core.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/linux/ltt-core.h	2009-03-04 13:37:26.000000000 -0500
> @@ -0,0 +1,47 @@
> +/*
> + * Copyright (C) 2005,2006 Mathieu Desnoyers (mathieu.desnoyers at polymtl.ca)
> + *
> + * This contains the core definitions for the Linux Trace Toolkit.
> + */
> +
> +#ifndef LTT_CORE_H
> +#define LTT_CORE_H
> +
> +#include <linux/list.h>
> +#include <linux/percpu.h>
> +
> +/* ltt's root dir in debugfs */
> +#define LTT_ROOT        "ltt"
> +
> +/*
> + * All modifications of ltt_traces must be done by ltt-tracer.c, while holding
> + * the semaphore. Only reading of this information can be done elsewhere, with
> + * the RCU mechanism : the preemption must be disabled while reading the
> + * list.
> + */
> +struct ltt_traces {
> +	struct list_head setup_head;	/* Pre-allocated traces list */
> +	struct list_head head;		/* Allocated Traces list */
> +	unsigned int num_active_traces;	/* Number of active traces */
> +} ____cacheline_aligned;
> +
> +extern struct ltt_traces ltt_traces;
> +
> +/*
> + * get dentry of ltt's root dir
> + */
> +struct dentry *get_ltt_root(void);
> +
> +void put_ltt_root(void);
> +
> +/* Keep track of trap nesting inside LTT */
> +DECLARE_PER_CPU(unsigned int, ltt_nesting);
> +
> +typedef int (*ltt_run_filter_functor)(void *trace, uint16_t eID);
> +
> +extern ltt_run_filter_functor ltt_run_filter;
> +
> +extern void ltt_filter_register(ltt_run_filter_functor func);
> +extern void ltt_filter_unregister(void);
> +
> +#endif /* LTT_CORE_H */
> 
> -- 
> Mathieu Desnoyers
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> 

[ltt-dev] [RFC patch 02/41] LTTng - core data structures

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-core.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/f99458d6/attachment.asc>

[ltt-dev] [RFC patch 02/41] LTTng - core data structures

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-core.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/f99458d6/attachment-0001.asc>

[ltt-dev] [RFC patch 02/41] LTTng - core data structures

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-core.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/f99458d6/attachment.txt>

[ltt-dev] [RFC patch 02/41] LTTng - core data structures

[Steven Rostedt]

On Thu, 5 Mar 2009, Mathieu Desnoyers wrote:

> Home of the traces data structures. Needs to be built into the kernel.
> 
> LTT heartbeat is a module specialized into firing periodical interrupts to
> record events in traces (so cycle counter rollover can be detected) and to
> update the 64 bits "synthetic TSC" (extended from the CPU 32 bits TSC on MIPS).
> Also needs to be built into the kernel.

Why is patch 1 and 2 separate. They look like they should be a single 
patch. Patch 1 is pretty useless by itself. This patch depends on patch 1.

-- Steve



> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca>
> ---
>  MAINTAINERS              |    7 +++
>  include/linux/ltt-core.h |   10 ++++
>  ltt/ltt-core.c           |  101 +++++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 118 insertions(+)
> 
> Index: linux-2.6-lttng/MAINTAINERS
> ===================================================================
> --- linux-2.6-lttng.orig/MAINTAINERS	2009-03-04 13:24:38.000000000 -0500
> +++ linux-2.6-lttng/MAINTAINERS	2009-03-04 13:24:59.000000000 -0500
> @@ -2766,6 +2766,13 @@ P:	Eric Piel
>  M:	eric.piel at tremplin-utc.net
>  S:	Maintained
>  
> +LINUX TRACE TOOLKIT NEXT GENERATION
> +P:	Mathieu Desnoyers
> +M:	mathieu.desnoyers at polymtl.ca
> +L:	ltt-dev at lttng.org
> +W:	http://ltt.polymtl.ca
> +S:	Maintained
> +
>  LM83 HARDWARE MONITOR DRIVER
>  P:	Jean Delvare
>  M:	khali at linux-fr.org
> Index: linux-2.6-lttng/ltt/ltt-core.c
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/ltt/ltt-core.c	2009-03-04 13:36:17.000000000 -0500
> @@ -0,0 +1,101 @@
> +/*
> + * LTT core in-kernel infrastructure.
> + *
> + * Copyright 2006 - Mathieu Desnoyers mathieu.desnoyers at polymtl.ca
> + *
> + * Distributed under the GPL license
> + */
> +
> +#include <linux/ltt-core.h>
> +#include <linux/percpu.h>
> +#include <linux/module.h>
> +#include <linux/debugfs.h>
> +#include <linux/kref.h>
> +
> +/* Traces structures */
> +struct ltt_traces ltt_traces = {
> +	.setup_head = LIST_HEAD_INIT(ltt_traces.setup_head),
> +	.head = LIST_HEAD_INIT(ltt_traces.head),
> +};
> +EXPORT_SYMBOL(ltt_traces);
> +
> +/* Traces list writer locking */
> +static DEFINE_MUTEX(ltt_traces_mutex);
> +
> +/* root dentry mutex */
> +static DEFINE_MUTEX(ltt_root_mutex);
> +/* dentry of ltt's root dir */
> +static struct dentry *ltt_root_dentry;
> +static struct kref ltt_root_kref = {
> +	.refcount = ATOMIC_INIT(0),
> +};
> +
> +static void ltt_root_release(struct kref *ref)
> +{
> +	debugfs_remove(ltt_root_dentry);
> +	ltt_root_dentry = NULL;
> +}
> +
> +void put_ltt_root(void)
> +{
> +	mutex_lock(&ltt_root_mutex);
> +	if (ltt_root_dentry)
> +		kref_put(&ltt_root_kref, ltt_root_release);
> +	mutex_unlock(&ltt_root_mutex);
> +}
> +EXPORT_SYMBOL_GPL(put_ltt_root);
> +
> +struct dentry *get_ltt_root(void)
> +{
> +	mutex_lock(&ltt_root_mutex);
> +	if (!ltt_root_dentry) {
> +		ltt_root_dentry = debugfs_create_dir(LTT_ROOT, NULL);
> +		if (!ltt_root_dentry) {
> +			printk(KERN_ERR "LTT : create ltt root dir failed\n");
> +			goto out;
> +		}
> +		kref_init(&ltt_root_kref);
> +		goto out;
> +	}
> +	kref_get(&ltt_root_kref);
> +out:
> +	mutex_unlock(&ltt_root_mutex);
> +	return ltt_root_dentry;
> +}
> +EXPORT_SYMBOL_GPL(get_ltt_root);
> +
> +void ltt_lock_traces(void)
> +{
> +	mutex_lock(&ltt_traces_mutex);
> +}
> +EXPORT_SYMBOL_GPL(ltt_lock_traces);
> +
> +void ltt_unlock_traces(void)
> +{
> +	mutex_unlock(&ltt_traces_mutex);
> +}
> +EXPORT_SYMBOL_GPL(ltt_unlock_traces);
> +
> +DEFINE_PER_CPU(unsigned int, ltt_nesting);
> +EXPORT_PER_CPU_SYMBOL(ltt_nesting);
> +
> +int ltt_run_filter_default(void *trace, uint16_t eID)
> +{
> +	return 1;
> +}
> +
> +/* This function pointer is protected by a trace activation check */
> +ltt_run_filter_functor ltt_run_filter = ltt_run_filter_default;
> +EXPORT_SYMBOL_GPL(ltt_run_filter);
> +
> +void ltt_filter_register(ltt_run_filter_functor func)
> +{
> +	ltt_run_filter = func;
> +}
> +EXPORT_SYMBOL_GPL(ltt_filter_register);
> +
> +void ltt_filter_unregister(void)
> +{
> +	ltt_run_filter = ltt_run_filter_default;
> +}
> +EXPORT_SYMBOL_GPL(ltt_filter_unregister);
> 
> -- 
> Mathieu Desnoyers
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> 

[ltt-dev] [RFC patch 03/41] LTTng core x86

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-core-x86.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/ca50701d/attachment.asc>

[ltt-dev] [RFC patch 03/41] LTTng core x86

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-core-x86.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/ca50701d/attachment-0001.asc>

[ltt-dev] [RFC patch 03/41] LTTng core x86

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-core-x86.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/ca50701d/attachment.txt>

[ltt-dev] [RFC patch 04/41] LTTng core powerpc

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-core-powerpc.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/7ed44b0c/attachment.txt>

[ltt-dev] [RFC patch 04/41] LTTng core powerpc

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-core-powerpc.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/7ed44b0c/attachment-0001.txt>

[ltt-dev] [RFC patch 04/41] LTTng core powerpc

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-core-powerpc.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/7ed44b0c/attachment.asc>

[ltt-dev] [RFC patch 05/41] LTTng relay buffer allocation, read, write

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-relay-alloc.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/daf7f4b0/attachment.txt>

[ltt-dev] [RFC patch 05/41] LTTng relay buffer allocation, read, write

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-relay-alloc.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/daf7f4b0/attachment.asc>

[ltt-dev] [RFC patch 05/41] LTTng relay buffer allocation, read, write

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-relay-alloc.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/daf7f4b0/attachment-0001.txt>

[ltt-dev] [RFC patch 05/41] LTTng relay buffer allocation, read, write

[Steven Rostedt]

On Thu, 5 Mar 2009, Mathieu Desnoyers wrote:

> As I told Martin, I was thinking about taking an axe and moving stuff around in
> relay. Which I just did.
> 
> This patch reimplements relay with a linked list of pages. Provides read/write
> wrappers which should be used to read or write from the buffers. It's the core
> of a layered approach to the design requirements expressed by Martin and
> discussed earlier.
> 
> It does not provide _any_ sort of locking on buffer data. Locking should be done
> by the caller. Given that we might think of very lightweight locking schemes, it
> makes sense to me that the underlying buffering infrastructure supports event
> records larger than 1 page.

You bring up two points.

1) lockless

2) larger than 1 page of data

Soon the ring buffer will be lockless on the write side. On the read side 
we have locking. This could be changed to allow for a new API with 
specific requirements that does not need reader side locking. If the ring 
buffer is strictly produce/consumer without overwrite, then it would make 
sense to have a lockless system on both reader and writer.

I have even expressed interest in implementing this. But right now my 
focus has been on getting other aspects working. Ftrace runs in overwrite 
mode so it must have the locking.


> 
> A cache saving 4 pointers is used to keep track of current page used for the
> buffer for write, current page read and two contiguous subbuffer header pointer
> lookup. The offset of each page within the buffer is saved in a structure
> containing the offset, linked list and page frame pointer to permit cache lookup
> without extra locking.

I'm also all for optimizations. Right now the focus has been on making 
sure the ring buffer can do all that is requested of it. I wanted it to be 
able to useful for all users, not focused on a select few. This actually 
includes ftrace. That is, I did not develop the ring buffer to have only 
ftrace as its only user.

> 
> The offset and linked list are not placed in the page frame itself to allow
> using the pages directly for disk I/O, network I/O or to mmap it to userspace
> for live processing.
> 
> Write and header address lookup tested through LTTng. This patch contains
> self-test code which detects if a client is actually trying to use the
> read/write/get header address API to do random buffer offset access. If such
> behavior is detected, a warning message is issued and the random access is done
> as requested.

Hmm, if a warning message is done, it seems that the tracer should shut 
down. Either support the operation or do not support it. Do not give the 
user a "Oh, you really should not do that, but I'll let you anyway". 
Otherwise you will be pressured to make it a true feature.

> 
> TODO : Currently, no splice file operations are implemented. Should come soon.
> The idea is to splice the buffers directly into files or to the network.
> We have to make sure the page frame fields used are not used by disk I/O or
> network.
> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca>
> CC: Jens Axboe <jens.axboe at oracle.com>
> CC: Martin Bligh <mbligh at google.com>
> CC: Peter Zijlstra <a.p.zijlstra at chello.nl>
> CC: Tom Zanussi <zanussi at comcast.net>
> CC: prasad at linux.vnet.ibm.com
> CC: Linus Torvalds <torvalds at linux-foundation.org>
> CC: Thomas Gleixner <tglx at linutronix.de>
> CC: Steven Rostedt <rostedt at goodmis.org>
> CC: od at suse.com
> CC: "Frank Ch. Eigler" <fche at redhat.com>
> CC: Andrew Morton <akpm at linux-foundation.org>
> CC: hch at lst.de
> CC: David Wilder <dwilder at us.ibm.com>
> ---
>  include/linux/ltt-relay.h |  182 +++++++++++
>  ltt/ltt-relay-alloc.c     |  705 ++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 887 insertions(+)
> 
> Index: linux-2.6-lttng/ltt/ltt-relay-alloc.c
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/ltt/ltt-relay-alloc.c	2009-03-05 15:05:56.000000000 -0500
> @@ -0,0 +1,705 @@
> +/*
> + * Public API and common code for kernel->userspace relay file support.
> + *
> + * Copyright (C) 2002-2005 - Tom Zanussi (zanussi at us.ibm.com), IBM Corp
> + * Copyright (C) 1999-2005 - Karim Yaghmour (karim at opersys.com)
> + * Copyright (C) 2008 - Mathieu Desnoyers (mathieu.desnoyers at polymtl.ca)
> + *
> + * Moved to kernel/relay.c by Paul Mundt, 2006.
> + * November 2006 - CPU hotplug support by Mathieu Desnoyers
> + * 	(mathieu.desnoyers at polymtl.ca)
> + *
> + * This file is released under the GPL.
> + */
> +#include <linux/errno.h>
> +#include <linux/stddef.h>
> +#include <linux/slab.h>
> +#include <linux/module.h>
> +#include <linux/string.h>
> +#include <linux/ltt-relay.h>
> +#include <linux/vmalloc.h>
> +#include <linux/mm.h>
> +#include <linux/cpu.h>
> +#include <linux/splice.h>
> +#include <linux/bitops.h>
> +
> +/* list of open channels, for cpu hotplug */
> +static DEFINE_MUTEX(relay_channels_mutex);
> +static LIST_HEAD(relay_channels);
> +
> +/**
> + *	relay_alloc_buf - allocate a channel buffer
> + *	@buf: the buffer struct
> + *	@size: total size of the buffer
> + */
> +static int relay_alloc_buf(struct rchan_buf *buf, size_t *size)
> +{
> +	unsigned int i, n_pages;
> +	struct buf_page *buf_page, *n;
> +
> +	*size = PAGE_ALIGN(*size);
> +	n_pages = *size >> PAGE_SHIFT;
> +
> +	INIT_LIST_HEAD(&buf->pages);
> +
> +	for (i = 0; i < n_pages; i++) {
> +		buf_page = kmalloc_node(sizeof(*buf_page), GFP_KERNEL,
> +			cpu_to_node(buf->cpu));
> +		if (unlikely(!buf_page))
> +			goto depopulate;
> +		buf_page->page = alloc_pages_node(cpu_to_node(buf->cpu),
> +			GFP_KERNEL | __GFP_ZERO, 0);

So these buffers allocate single pages?

> +		if (unlikely(!buf_page->page)) {
> +			kfree(buf_page);
> +			goto depopulate;
> +		}
> +		list_add_tail(&buf_page->list, &buf->pages);
> +		buf_page->offset = (size_t)i << PAGE_SHIFT;
> +		set_page_private(buf_page->page, (unsigned long)buf_page);
> +		if (i == 0) {
> +			buf->wpage = buf_page;
> +			buf->hpage[0] = buf_page;
> +			buf->hpage[1] = buf_page;
> +			buf->rpage = buf_page;
> +		}
> +	}
> +	buf->page_count = n_pages;
> +	return 0;
> +
> +depopulate:
> +	list_for_each_entry_safe(buf_page, n, &buf->pages, list) {
> +		list_del_init(&buf_page->list);
> +		__free_page(buf_page->page);
> +		kfree(buf_page);
> +	}
> +	return -ENOMEM;
> +}
> +
> +/**
> + *	relay_create_buf - allocate and initialize a channel buffer
> + *	@chan: the relay channel
> + *	@cpu: cpu the buffer belongs to
> + *
> + *	Returns channel buffer if successful, %NULL otherwise.
> + */
> +static struct rchan_buf *relay_create_buf(struct rchan *chan, int cpu)
> +{
> +	int ret;
> +	struct rchan_buf *buf = kzalloc(sizeof(struct rchan_buf), GFP_KERNEL);
> +	if (!buf)
> +		return NULL;
> +
> +	buf->cpu = cpu;
> +	ret = relay_alloc_buf(buf, &chan->alloc_size);
> +	if (ret)
> +		goto free_buf;
> +
> +	buf->chan = chan;
> +	kref_get(&buf->chan->kref);
> +	return buf;
> +
> +free_buf:
> +	kfree(buf);
> +	return NULL;
> +}
> +
> +/**
> + *	relay_destroy_channel - free the channel struct
> + *	@kref: target kernel reference that contains the relay channel
> + *
> + *	Should only be called from kref_put().
> + */
> +static void relay_destroy_channel(struct kref *kref)
> +{
> +	struct rchan *chan = container_of(kref, struct rchan, kref);
> +	kfree(chan);
> +}
> +
> +void ltt_relay_get_chan(struct rchan *chan)
> +{
> +	kref_get(&chan->kref);
> +}
> +EXPORT_SYMBOL_GPL(ltt_relay_get_chan);
> +
> +void ltt_relay_put_chan(struct rchan *chan)
> +{
> +	kref_put(&chan->kref, relay_destroy_channel);
> +}
> +EXPORT_SYMBOL_GPL(ltt_relay_put_chan);
> +
> +/**
> + *	relay_destroy_buf - destroy an rchan_buf struct and associated buffer
> + *	@buf: the buffer struct
> + */
> +static void relay_destroy_buf(struct rchan_buf *buf)
> +{
> +	struct rchan *chan = buf->chan;
> +	struct buf_page *buf_page, *n;
> +
> +	list_for_each_entry_safe(buf_page, n, &buf->pages, list) {
> +		list_del_init(&buf_page->list);
> +		__free_page(buf_page->page);
> +		kfree(buf_page);
> +	}
> +	chan->buf[buf->cpu] = NULL;
> +	kfree(buf);
> +	kref_put(&chan->kref, relay_destroy_channel);
> +}
> +
> +/**
> + *	relay_remove_buf - remove a channel buffer
> + *	@kref: target kernel reference that contains the relay buffer
> + *
> + *	Removes the file from the fileystem, which also frees the
> + *	rchan_buf_struct and the channel buffer.  Should only be called from
> + *	kref_put().
> + */
> +static void relay_remove_buf(struct kref *kref)
> +{
> +	struct rchan_buf *buf = container_of(kref, struct rchan_buf, kref);
> +	buf->chan->cb->remove_buf_file(buf->dentry);
> +	relay_destroy_buf(buf);
> +}
> +
> +void ltt_relay_get_chan_buf(struct rchan_buf *buf)
> +{
> +	kref_get(&buf->kref);
> +}
> +EXPORT_SYMBOL_GPL(ltt_relay_get_chan_buf);
> +
> +void ltt_relay_put_chan_buf(struct rchan_buf *buf)
> +{
> +	kref_put(&buf->kref, relay_remove_buf);
> +}
> +EXPORT_SYMBOL_GPL(ltt_relay_put_chan_buf);
> +
> +/*
> + * High-level relay kernel API and associated functions.
> + */
> +
> +/*
> + * rchan_callback implementations defining default channel behavior.  Used
> + * in place of corresponding NULL values in client callback struct.
> + */
> +
> +/*
> + * create_buf_file_create() default callback.  Does nothing.
> + */
> +static struct dentry *create_buf_file_default_callback(const char *filename,
> +						       struct dentry *parent,
> +						       int mode,
> +						       struct rchan_buf *buf)
> +{
> +	return NULL;
> +}
> +
> +/*
> + * remove_buf_file() default callback.  Does nothing.
> + */
> +static int remove_buf_file_default_callback(struct dentry *dentry)
> +{
> +	return -EINVAL;
> +}
> +
> +/* relay channel default callbacks */
> +static struct rchan_callbacks default_channel_callbacks = {
> +	.create_buf_file = create_buf_file_default_callback,
> +	.remove_buf_file = remove_buf_file_default_callback,
> +};
> +
> +/**
> + *	__relay_reset - reset a channel buffer
> + *	@buf: the channel buffer
> + *	@init: 1 if this is a first-time initialization
> + *
> + *	See relay_reset() for description of effect.
> + */
> +static void __relay_reset(struct rchan_buf *buf, unsigned int init)
> +{
> +	if (init)
> +		kref_init(&buf->kref);
> +}
> +
> +/*
> + *	relay_open_buf - create a new relay channel buffer
> + *
> + *	used by relay_open() and CPU hotplug.
> + */
> +static struct rchan_buf *relay_open_buf(struct rchan *chan, unsigned int cpu)
> +{
> +	struct rchan_buf *buf = NULL;
> +	struct dentry *dentry;
> +	char *tmpname;
> +
> +	tmpname = kzalloc(NAME_MAX + 1, GFP_KERNEL);
> +	if (!tmpname)
> +		goto end;
> +	snprintf(tmpname, NAME_MAX, "%s%d", chan->base_filename, cpu);
> +
> +	buf = relay_create_buf(chan, cpu);
> +	if (!buf)
> +		goto free_name;
> +
> +	__relay_reset(buf, 1);
> +
> +	/* Create file in fs */
> +	dentry = chan->cb->create_buf_file(tmpname, chan->parent, S_IRUSR,
> +					   buf);
> +	if (!dentry)
> +		goto free_buf;
> +
> +	buf->dentry = dentry;
> +
> +	goto free_name;
> +
> +free_buf:
> +	relay_destroy_buf(buf);
> +	buf = NULL;
> +free_name:
> +	kfree(tmpname);
> +end:
> +	return buf;
> +}
> +
> +/**
> + *	relay_close_buf - close a channel buffer
> + *	@buf: channel buffer
> + *
> + *	Restores the default callbacks.
> + *	The channel buffer and channel buffer data structure are then freed
> + *	automatically when the last reference is given up.
> + */
> +static void relay_close_buf(struct rchan_buf *buf)
> +{
> +	kref_put(&buf->kref, relay_remove_buf);
> +}
> +
> +static void setup_callbacks(struct rchan *chan,
> +				   struct rchan_callbacks *cb)
> +{
> +	if (!cb) {
> +		chan->cb = &default_channel_callbacks;
> +		return;
> +	}
> +
> +	if (!cb->create_buf_file)
> +		cb->create_buf_file = create_buf_file_default_callback;
> +	if (!cb->remove_buf_file)
> +		cb->remove_buf_file = remove_buf_file_default_callback;
> +	chan->cb = cb;
> +}
> +
> +/**
> + * 	relay_hotcpu_callback - CPU hotplug callback
> + * 	@nb: notifier block
> + * 	@action: hotplug action to take
> + * 	@hcpu: CPU number
> + *
> + * 	Returns the success/failure of the operation. (%NOTIFY_OK, %NOTIFY_BAD)
> + */
> +static int __cpuinit relay_hotcpu_callback(struct notifier_block *nb,
> +				unsigned long action,
> +				void *hcpu)
> +{
> +	unsigned int hotcpu = (unsigned long)hcpu;
> +	struct rchan *chan;
> +
> +	switch (action) {
> +	case CPU_UP_PREPARE:
> +	case CPU_UP_PREPARE_FROZEN:
> +		mutex_lock(&relay_channels_mutex);
> +		list_for_each_entry(chan, &relay_channels, list) {
> +			if (chan->buf[hotcpu])
> +				continue;
> +			chan->buf[hotcpu] = relay_open_buf(chan, hotcpu);
> +			if (!chan->buf[hotcpu]) {
> +				printk(KERN_ERR
> +					"relay_hotcpu_callback: cpu %d buffer "
> +					"creation failed\n", hotcpu);
> +				mutex_unlock(&relay_channels_mutex);
> +				return NOTIFY_BAD;
> +			}
> +		}
> +		mutex_unlock(&relay_channels_mutex);
> +		break;
> +	case CPU_DEAD:
> +	case CPU_DEAD_FROZEN:
> +		/* No need to flush the cpu : will be flushed upon
> +		 * final relay_flush() call. */
> +		break;
> +	}
> +	return NOTIFY_OK;
> +}
> +
> +/**
> + *	ltt_relay_open - create a new relay channel
> + *	@base_filename: base name of files to create
> + *	@parent: dentry of parent directory, %NULL for root directory
> + *	@subbuf_size: size of sub-buffers
> + *	@n_subbufs: number of sub-buffers
> + *	@cb: client callback functions
> + *	@private_data: user-defined data
> + *
> + *	Returns channel pointer if successful, %NULL otherwise.
> + *
> + *	Creates a channel buffer for each cpu using the sizes and
> + *	attributes specified.  The created channel buffer files
> + *	will be named base_filename0...base_filenameN-1.  File
> + *	permissions will be %S_IRUSR.
> + */
> +struct rchan *ltt_relay_open(const char *base_filename,
> +			 struct dentry *parent,
> +			 size_t subbuf_size,
> +			 size_t n_subbufs,
> +			 struct rchan_callbacks *cb,
> +			 void *private_data)
> +{
> +	unsigned int i;
> +	struct rchan *chan;
> +	if (!base_filename)
> +		return NULL;
> +
> +	if (!(subbuf_size && n_subbufs))
> +		return NULL;
> +
> +	chan = kzalloc(sizeof(struct rchan), GFP_KERNEL);
> +	if (!chan)
> +		return NULL;
> +
> +	chan->version = LTT_RELAY_CHANNEL_VERSION;
> +	chan->n_subbufs = n_subbufs;
> +	chan->subbuf_size = subbuf_size;

You declare the sub buf size here, but I do not see how it gets allocated.

-- Steve

> +	chan->subbuf_size_order = get_count_order(subbuf_size);
> +	chan->alloc_size = FIX_SIZE(subbuf_size * n_subbufs);
> +	chan->parent = parent;
> +	chan->private_data = private_data;
> +	strlcpy(chan->base_filename, base_filename, NAME_MAX);
> +	setup_callbacks(chan, cb);
> +	kref_init(&chan->kref);
> +
> +	mutex_lock(&relay_channels_mutex);
> +	for_each_online_cpu(i) {
> +		chan->buf[i] = relay_open_buf(chan, i);
> +		if (!chan->buf[i])
> +			goto free_bufs;
> +	}
> +	list_add(&chan->list, &relay_channels);
> +	mutex_unlock(&relay_channels_mutex);
> +
> +	return chan;
> +
> +free_bufs:
> +	for_each_possible_cpu(i) {
> +		if (!chan->buf[i])
> +			break;
> +		relay_close_buf(chan->buf[i]);
> +	}
> +
> +	kref_put(&chan->kref, relay_destroy_channel);
> +	mutex_unlock(&relay_channels_mutex);
> +	return NULL;
> +}
> +EXPORT_SYMBOL_GPL(ltt_relay_open);
> +
> +/**
> + *	ltt_relay_close - close the channel
> + *	@chan: the channel
> + *
> + *	Closes all channel buffers and frees the channel.
> + */
> +void ltt_relay_close(struct rchan *chan)
> +{
> +	unsigned int i;
> +
> +	if (!chan)
> +		return;
> +
> +	mutex_lock(&relay_channels_mutex);
> +	for_each_possible_cpu(i)
> +		if (chan->buf[i])
> +			relay_close_buf(chan->buf[i]);
> +
> +	list_del(&chan->list);
> +	kref_put(&chan->kref, relay_destroy_channel);
> +	mutex_unlock(&relay_channels_mutex);
> +}
> +EXPORT_SYMBOL_GPL(ltt_relay_close);
> +
> +/*
> + * Start iteration at the previous element. Skip the real list head.
> + */
> +static struct buf_page *ltt_relay_find_prev_page(struct rchan_buf *buf,
> +	struct buf_page *page, size_t offset, ssize_t diff_offset)
> +{
> +	struct buf_page *iter;
> +	size_t orig_iter_off;
> +	unsigned int i = 0;
> +
> +	orig_iter_off = page->offset;
> +	list_for_each_entry_reverse(iter, &page->list, list) {
> +		/*
> +		 * Skip the real list head.
> +		 */
> +		if (&iter->list == &buf->pages)
> +			continue;
> +		i++;
> +		if (offset >= iter->offset
> +			&& offset < iter->offset + PAGE_SIZE) {
> +#ifdef CONFIG_LTT_RELAY_CHECK_RANDOM_ACCESS
> +			if (i > 1) {
> +				printk(KERN_WARNING
> +					"Backward random access detected in "
> +					"ltt_relay. Iterations %u, "
> +					"offset %zu, orig iter->off %zu, "
> +					"iter->off %zu diff_offset %zd.\n", i,
> +					offset, orig_iter_off, iter->offset,
> +					diff_offset);
> +				WARN_ON(1);
> +			}
> +#endif
> +			return iter;
> +		}
> +	}
> +	return NULL;
> +}
> +
> +/*
> + * Start iteration at the next element. Skip the real list head.
> + */
> +static struct buf_page *ltt_relay_find_next_page(struct rchan_buf *buf,
> +	struct buf_page *page, size_t offset, ssize_t diff_offset)
> +{
> +	struct buf_page *iter;
> +	unsigned int i = 0;
> +	size_t orig_iter_off;
> +
> +	orig_iter_off = page->offset;
> +	list_for_each_entry(iter, &page->list, list) {
> +		/*
> +		 * Skip the real list head.
> +		 */
> +		if (&iter->list == &buf->pages)
> +			continue;
> +		i++;
> +		if (offset >= iter->offset
> +			&& offset < iter->offset + PAGE_SIZE) {
> +#ifdef CONFIG_LTT_RELAY_CHECK_RANDOM_ACCESS
> +			if (i > 1) {
> +				printk(KERN_WARNING
> +					"Forward random access detected in "
> +					"ltt_relay. Iterations %u, "
> +					"offset %zu, orig iter->off %zu, "
> +					"iter->off %zu diff_offset %zd.\n", i,
> +					offset, orig_iter_off, iter->offset,
> +					diff_offset);
> +				WARN_ON(1);
> +			}
> +#endif
> +			return iter;
> +		}
> +	}
> +	return NULL;
> +}
> +
> +/*
> + * Find the page containing "offset". Cache it if it is after the currently
> + * cached page.
> + */
> +static struct buf_page *ltt_relay_cache_page(struct rchan_buf *buf,
> +		struct buf_page **page_cache,
> +		struct buf_page *page, size_t offset)
> +{
> +	ssize_t diff_offset;
> +	ssize_t half_buf_size = buf->chan->alloc_size >> 1;
> +
> +	/*
> +	 * Make sure this is the page we want to write into. The current
> +	 * page is changed concurrently by other writers. [wrh]page are
> +	 * used as a cache remembering the last page written
> +	 * to/read/looked up for header address. No synchronization;
> +	 * could have to find the previous page is a nested write
> +	 * occured. Finding the right page is done by comparing the
> +	 * dest_offset with the buf_page offsets.
> +	 * When at the exact opposite of the buffer, bias towards forward search
> +	 * because it will be cached.
> +	 */
> +
> +	diff_offset = (ssize_t)offset - (ssize_t)page->offset;
> +	if (diff_offset <= -(ssize_t)half_buf_size)
> +		diff_offset += buf->chan->alloc_size;
> +	else if (diff_offset > half_buf_size)
> +		diff_offset -= buf->chan->alloc_size;
> +
> +	if (unlikely(diff_offset >= (ssize_t)PAGE_SIZE)) {
> +		page = ltt_relay_find_next_page(buf, page, offset, diff_offset);
> +		WARN_ON(!page);
> +		*page_cache = page;
> +	} else if (unlikely(diff_offset < 0)) {
> +		page = ltt_relay_find_prev_page(buf, page, offset, diff_offset);
> +		WARN_ON(!page);
> +	}
> +	return page;
> +}
> +
> +/**
> + * ltt_relay_write - write data to a ltt_relay buffer.
> + * @buf : buffer
> + * @offset : offset within the buffer
> + * @src : source address
> + * @len : length to write
> + */
> +int ltt_relay_write(struct rchan_buf *buf, size_t offset,
> +	const void *src, size_t len)
> +{
> +	struct buf_page *page;
> +	ssize_t pagecpy, orig_len;
> +
> +	orig_len = len;
> +	offset &= buf->chan->alloc_size - 1;
> +	page = buf->wpage;
> +	if (unlikely(!len))
> +		return 0;
> +	for (;;) {
> +		page = ltt_relay_cache_page(buf, &buf->wpage, page, offset);
> +		pagecpy = min_t(size_t, len, PAGE_SIZE - (offset & ~PAGE_MASK));
> +		memcpy(page_address(page->page)
> +			+ (offset & ~PAGE_MASK), src, pagecpy);
> +		len -= pagecpy;
> +		if (likely(!len))
> +			break;
> +		src += pagecpy;
> +		offset += pagecpy;
> +		/*
> +		 * Underlying layer should never ask for writes across
> +		 * subbuffers.
> +		 */
> +		WARN_ON(offset >= buf->chan->alloc_size);
> +	}
> +	return orig_len;
> +}
> +EXPORT_SYMBOL_GPL(ltt_relay_write);
> +
> +/**
> + * ltt_relay_read - read data from ltt_relay_buffer.
> + * @buf : buffer
> + * @offset : offset within the buffer
> + * @dest : destination address
> + * @len : length to write
> + */
> +int ltt_relay_read(struct rchan_buf *buf, size_t offset,
> +	void *dest, size_t len)
> +{
> +	struct buf_page *page;
> +	ssize_t pagecpy, orig_len;
> +
> +	orig_len = len;
> +	offset &= buf->chan->alloc_size - 1;
> +	page = buf->rpage;
> +	if (unlikely(!len))
> +		return 0;
> +	for (;;) {
> +		page = ltt_relay_cache_page(buf, &buf->rpage, page, offset);
> +		pagecpy = min_t(size_t, len, PAGE_SIZE - (offset & ~PAGE_MASK));
> +		memcpy(dest, page_address(page->page) + (offset & ~PAGE_MASK),
> +			pagecpy);
> +		len -= pagecpy;
> +		if (likely(!len))
> +			break;
> +		dest += pagecpy;
> +		offset += pagecpy;
> +		/*
> +		 * Underlying layer should never ask for reads across
> +		 * subbuffers.
> +		 */
> +		WARN_ON(offset >= buf->chan->alloc_size);
> +	}
> +	return orig_len;
> +}
> +EXPORT_SYMBOL_GPL(ltt_relay_read);
> +
> +/**
> + * ltt_relay_read_get_page - Get a whole page to read from
> + * @buf : buffer
> + * @offset : offset within the buffer
> + */
> +struct buf_page *ltt_relay_read_get_page(struct rchan_buf *buf, size_t offset)
> +{
> +	struct buf_page *page;
> +
> +	offset &= buf->chan->alloc_size - 1;
> +	page = buf->rpage;
> +	page = ltt_relay_cache_page(buf, &buf->rpage, page, offset);
> +	return page;
> +}
> +EXPORT_SYMBOL_GPL(ltt_relay_read_get_page);
> +
> +/**
> + * ltt_relay_offset_address - get address of a location within the buffer
> + * @buf : buffer
> + * @offset : offset within the buffer.
> + *
> + * Return the address where a given offset is located.
> + * Should be used to get the current subbuffer header pointer. Given we know
> + * it's never on a page boundary, it's safe to write directly to this address,
> + * as long as the write is never bigger than a page size.
> + */
> +void *ltt_relay_offset_address(struct rchan_buf *buf, size_t offset)
> +{
> +	struct buf_page *page;
> +	unsigned int odd;
> +
> +	offset &= buf->chan->alloc_size - 1;
> +	odd = !!(offset & buf->chan->subbuf_size);
> +	page = buf->hpage[odd];
> +	if (offset < page->offset || offset >= page->offset + PAGE_SIZE)
> +		buf->hpage[odd] = page = buf->wpage;
> +	page = ltt_relay_cache_page(buf, &buf->hpage[odd], page, offset);
> +	return page_address(page->page) + (offset & ~PAGE_MASK);
> +}
> +EXPORT_SYMBOL_GPL(ltt_relay_offset_address);
> +
> +/**
> + *	relay_file_open - open file op for relay files
> + *	@inode: the inode
> + *	@filp: the file
> + *
> + *	Increments the channel buffer refcount.
> + */
> +static int relay_file_open(struct inode *inode, struct file *filp)
> +{
> +	struct rchan_buf *buf = inode->i_private;
> +	kref_get(&buf->kref);
> +	filp->private_data = buf;
> +
> +	return nonseekable_open(inode, filp);
> +}
> +
> +/**
> + *	relay_file_release - release file op for relay files
> + *	@inode: the inode
> + *	@filp: the file
> + *
> + *	Decrements the channel refcount, as the filesystem is
> + *	no longer using it.
> + */
> +static int relay_file_release(struct inode *inode, struct file *filp)
> +{
> +	struct rchan_buf *buf = filp->private_data;
> +	kref_put(&buf->kref, relay_remove_buf);
> +
> +	return 0;
> +}
> +
> +const struct file_operations ltt_relay_file_operations = {
> +	.open		= relay_file_open,
> +	.release	= relay_file_release,
> +};
> +EXPORT_SYMBOL_GPL(ltt_relay_file_operations);
> +
> +static __init int relay_init(void)
> +{
> +	hotcpu_notifier(relay_hotcpu_callback, 5);
> +	return 0;
> +}
> +
> +module_init(relay_init);
> Index: linux-2.6-lttng/include/linux/ltt-relay.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/linux/ltt-relay.h	2009-03-05 15:05:56.000000000 -0500
> @@ -0,0 +1,182 @@
> +/*
> + * linux/include/linux/ltt-relay.h
> + *
> + * Copyright (C) 2002, 2003 - Tom Zanussi (zanussi at us.ibm.com), IBM Corp
> + * Copyright (C) 1999, 2000, 2001, 2002 - Karim Yaghmour (karim at opersys.com)
> + * Copyright (C) 2008 - Mathieu Desnoyers (mathieu.desnoyers at polymtl.ca)
> + *
> + * CONFIG_RELAY definitions and declarations
> + */
> +
> +#ifndef _LINUX_LTT_RELAY_H
> +#define _LINUX_LTT_RELAY_H
> +
> +#include <linux/types.h>
> +#include <linux/sched.h>
> +#include <linux/timer.h>
> +#include <linux/wait.h>
> +#include <linux/list.h>
> +#include <linux/fs.h>
> +#include <linux/poll.h>
> +#include <linux/kref.h>
> +#include <linux/mm.h>
> +
> +/* Needs a _much_ better name... */
> +#define FIX_SIZE(x) ((((x) - 1) & PAGE_MASK) + PAGE_SIZE)
> +
> +/*
> + * Tracks changes to rchan/rchan_buf structs
> + */
> +#define LTT_RELAY_CHANNEL_VERSION		8
> +
> +struct rchan_buf;
> +
> +struct buf_page {
> +	struct page *page;
> +	size_t offset;		/* page offset in the buffer */
> +	struct list_head list;	/* buffer linked list */
> +};
> +
> +/*
> + * Per-cpu relay channel buffer
> + */
> +struct rchan_buf {
> +	void *chan_private;		/* private data for this buf */
> +	struct rchan *chan;		/* associated channel */
> +	struct dentry *dentry;		/* channel file dentry */
> +	struct kref kref;		/* channel buffer refcount */
> +	struct list_head pages;		/* list of buffer pages */
> +	struct buf_page *wpage;		/* current write page (cache) */
> +	struct buf_page *hpage[2];	/* current subbuf header page (cache) */
> +	struct buf_page *rpage;		/* current subbuf read page (cache) */
> +	unsigned int page_count;	/* number of current buffer pages */
> +	unsigned int cpu;		/* this buf's cpu */
> +} ____cacheline_aligned;
> +
> +/*
> + * Relay channel data structure
> + */
> +struct rchan {
> +	u32 version;			/* the version of this struct */
> +	size_t subbuf_size;		/* sub-buffer size */
> +	size_t n_subbufs;		/* number of sub-buffers per buffer */
> +	size_t alloc_size;		/* total buffer size allocated */
> +	struct rchan_callbacks *cb;	/* client callbacks */
> +	struct kref kref;		/* channel refcount */
> +	void *private_data;		/* for user-defined data */
> +	struct rchan_buf *buf[NR_CPUS]; /* per-cpu channel buffers */
> +	struct list_head list;		/* for channel list */
> +	struct dentry *parent;		/* parent dentry passed to open */
> +	int subbuf_size_order;		/* order of sub-buffer size */
> +	char base_filename[NAME_MAX];	/* saved base filename */
> +};
> +
> +/*
> + * Relay channel client callbacks
> + */
> +struct rchan_callbacks {
> +	/*
> +	 * subbuf_start - called on buffer-switch to a new sub-buffer
> +	 * @buf: the channel buffer containing the new sub-buffer
> +	 * @subbuf: the start of the new sub-buffer
> +	 * @prev_subbuf: the start of the previous sub-buffer
> +	 * @prev_padding: unused space at the end of previous sub-buffer
> +	 *
> +	 * The client should return 1 to continue logging, 0 to stop
> +	 * logging.
> +	 *
> +	 * NOTE: subbuf_start will also be invoked when the buffer is
> +	 *       created, so that the first sub-buffer can be initialized
> +	 *       if necessary.  In this case, prev_subbuf will be NULL.
> +	 *
> +	 * NOTE: the client can reserve bytes at the beginning of the new
> +	 *       sub-buffer by calling subbuf_start_reserve() in this callback.
> +	 */
> +	int (*subbuf_start) (struct rchan_buf *buf,
> +			     void *subbuf,
> +			     void *prev_subbuf,
> +			     size_t prev_padding);
> +
> +	/*
> +	 * create_buf_file - create file to represent a relay channel buffer
> +	 * @filename: the name of the file to create
> +	 * @parent: the parent of the file to create
> +	 * @mode: the mode of the file to create
> +	 * @buf: the channel buffer
> +	 *
> +	 * Called during relay_open(), once for each per-cpu buffer,
> +	 * to allow the client to create a file to be used to
> +	 * represent the corresponding channel buffer.  If the file is
> +	 * created outside of relay, the parent must also exist in
> +	 * that filesystem.
> +	 *
> +	 * The callback should return the dentry of the file created
> +	 * to represent the relay buffer.
> +	 *
> +	 * Setting the is_global outparam to a non-zero value will
> +	 * cause relay_open() to create a single global buffer rather
> +	 * than the default set of per-cpu buffers.
> +	 *
> +	 * See Documentation/filesystems/relayfs.txt for more info.
> +	 */
> +	struct dentry *(*create_buf_file)(const char *filename,
> +					  struct dentry *parent,
> +					  int mode,
> +					  struct rchan_buf *buf);
> +
> +	/*
> +	 * remove_buf_file - remove file representing a relay channel buffer
> +	 * @dentry: the dentry of the file to remove
> +	 *
> +	 * Called during relay_close(), once for each per-cpu buffer,
> +	 * to allow the client to remove a file used to represent a
> +	 * channel buffer.
> +	 *
> +	 * The callback should return 0 if successful, negative if not.
> +	 */
> +	int (*remove_buf_file)(struct dentry *dentry);
> +};
> +
> +extern int ltt_relay_write(struct rchan_buf *buf, size_t offset,
> +	const void *src, size_t len);
> +
> +extern int ltt_relay_read(struct rchan_buf *buf, size_t offset,
> +	void *dest, size_t len);
> +
> +extern struct buf_page *ltt_relay_read_get_page(struct rchan_buf *buf,
> +	size_t offset);
> +
> +/*
> + * Return the address where a given offset is located.
> + * Should be used to get the current subbuffer header pointer. Given we know
> + * it's never on a page boundary, it's safe to write directly to this address,
> + * as long as the write is never bigger than a page size.
> + */
> +extern void *ltt_relay_offset_address(struct rchan_buf *buf,
> +	size_t offset);
> +
> +/*
> + * CONFIG_LTT_RELAY kernel API, ltt/ltt-relay-alloc.c
> + */
> +
> +struct rchan *ltt_relay_open(const char *base_filename,
> +			 struct dentry *parent,
> +			 size_t subbuf_size,
> +			 size_t n_subbufs,
> +			 struct rchan_callbacks *cb,
> +			 void *private_data);
> +extern void ltt_relay_close(struct rchan *chan);
> +
> +void ltt_relay_get_chan(struct rchan *chan);
> +void ltt_relay_put_chan(struct rchan *chan);
> +
> +void ltt_relay_get_chan_buf(struct rchan_buf *buf);
> +void ltt_relay_put_chan_buf(struct rchan_buf *buf);
> +
> +/*
> + * exported ltt_relay file operations, ltt/ltt-relay-alloc.c
> + */
> +extern const struct file_operations ltt_relay_file_operations;
> +
> +#endif /* _LINUX_LTT_RELAY_H */
> +
> 
> -- 
> Mathieu Desnoyers
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> 

[ltt-dev] [RFC patch 05/41] LTTng relay buffer allocation, read, write

[Mathieu Desnoyers]

* Steven Rostedt (rostedt at goodmis.org) wrote:
> 
> On Thu, 5 Mar 2009, Mathieu Desnoyers wrote:
> 
> > As I told Martin, I was thinking about taking an axe and moving stuff around in
> > relay. Which I just did.
> > 
> > This patch reimplements relay with a linked list of pages. Provides read/write
> > wrappers which should be used to read or write from the buffers. It's the core
> > of a layered approach to the design requirements expressed by Martin and
> > discussed earlier.
> > 
> > It does not provide _any_ sort of locking on buffer data. Locking should be done
> > by the caller. Given that we might think of very lightweight locking schemes, it
> > makes sense to me that the underlying buffering infrastructure supports event
> > records larger than 1 page.
> 
> You bring up two points.
> 
> 1) lockless
> 
> 2) larger than 1 page of data
> 
> Soon the ring buffer will be lockless on the write side. On the read side 
> we have locking. This could be changed to allow for a new API with 
> specific requirements that does not need reader side locking. If the ring 
> buffer is strictly produce/consumer without overwrite, then it would make 
> sense to have a lockless system on both reader and writer.
> 
> I have even expressed interest in implementing this. But right now my 
> focus has been on getting other aspects working. Ftrace runs in overwrite 
> mode so it must have the locking.
> 

Note that this "buffer allocation" is not lockless in the sense that it
deals with concurrent writers in a lockless manner. It just _does not
provide_ any protection for the write side nor the read-side. The layer
over this provides this. In this patchset, it's called
"ltt-relay-locked" for the irq off/spinlock version of the concurrency
management. I did not post the lockless version in this post, it is
further down in my patchset and will probably require a bit more
discussion.

I really like to have the allocation and "locking management" layers
separated, because we can then easily mix and match each of those.

For allocation :
- Page-based backend
- Static array-based backend
- Video memory based backend (useful for crash trace extraction, because
  this storage survives a hot reboot)

For locking :
- lockless
- irq off/spinlock

> 
> > 
> > A cache saving 4 pointers is used to keep track of current page used for the
> > buffer for write, current page read and two contiguous subbuffer header pointer
> > lookup. The offset of each page within the buffer is saved in a structure
> > containing the offset, linked list and page frame pointer to permit cache lookup
> > without extra locking.
> 
> I'm also all for optimizations. Right now the focus has been on making 
> sure the ring buffer can do all that is requested of it. I wanted it to be 
> able to useful for all users, not focused on a select few. This actually 
> includes ftrace. That is, I did not develop the ring buffer to have only 
> ftrace as its only user.
> 

I think we both agree that it's good to make the buffering
infrastructure usable by all.

> > 
> > The offset and linked list are not placed in the page frame itself to allow
> > using the pages directly for disk I/O, network I/O or to mmap it to userspace
> > for live processing.
> > 
> > Write and header address lookup tested through LTTng. This patch contains
> > self-test code which detects if a client is actually trying to use the
> > read/write/get header address API to do random buffer offset access. If such
> > behavior is detected, a warning message is issued and the random access is done
> > as requested.
> 
> Hmm, if a warning message is done, it seems that the tracer should shut 
> down. Either support the operation or do not support it. Do not give the 
> user a "Oh, you really should not do that, but I'll let you anyway". 
> Otherwise you will be pressured to make it a true feature.
> 

It is _possible_ that such scenario occurs, and it is supported. It's
just very unlikely. With the lockless tracer, if the tracing code is
interrupted for a long time and still holds a reference to a previous
page while the current write offset went too far away, then it's OK to
do a backward-multiple-pages-walk. But it's very, very unlikely. And
this debugging option helps pinpointing ill uses of the buffers in the
"likely" path very quickly. So I would never consider it a "failure" per
se. It's one of those useful tracer-debug-only options.

> > 
> > TODO : Currently, no splice file operations are implemented. Should come soon.
> > The idea is to splice the buffers directly into files or to the network.
> > We have to make sure the page frame fields used are not used by disk I/O or
> > network.
> > 
> > Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca>
> > CC: Jens Axboe <jens.axboe at oracle.com>
> > CC: Martin Bligh <mbligh at google.com>
> > CC: Peter Zijlstra <a.p.zijlstra at chello.nl>
> > CC: Tom Zanussi <zanussi at comcast.net>
> > CC: prasad at linux.vnet.ibm.com
> > CC: Linus Torvalds <torvalds at linux-foundation.org>
> > CC: Thomas Gleixner <tglx at linutronix.de>
> > CC: Steven Rostedt <rostedt at goodmis.org>
> > CC: od at suse.com
> > CC: "Frank Ch. Eigler" <fche at redhat.com>
> > CC: Andrew Morton <akpm at linux-foundation.org>
> > CC: hch at lst.de
> > CC: David Wilder <dwilder at us.ibm.com>
> > ---
> >  include/linux/ltt-relay.h |  182 +++++++++++
> >  ltt/ltt-relay-alloc.c     |  705 ++++++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 887 insertions(+)
> > 
> > Index: linux-2.6-lttng/ltt/ltt-relay-alloc.c
> > ===================================================================
> > --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> > +++ linux-2.6-lttng/ltt/ltt-relay-alloc.c	2009-03-05 15:05:56.000000000 -0500
> > @@ -0,0 +1,705 @@
> > +/*
> > + * Public API and common code for kernel->userspace relay file support.
> > + *
> > + * Copyright (C) 2002-2005 - Tom Zanussi (zanussi at us.ibm.com), IBM Corp
> > + * Copyright (C) 1999-2005 - Karim Yaghmour (karim at opersys.com)
> > + * Copyright (C) 2008 - Mathieu Desnoyers (mathieu.desnoyers at polymtl.ca)
> > + *
> > + * Moved to kernel/relay.c by Paul Mundt, 2006.
> > + * November 2006 - CPU hotplug support by Mathieu Desnoyers
> > + * 	(mathieu.desnoyers at polymtl.ca)
> > + *
> > + * This file is released under the GPL.
> > + */
> > +#include <linux/errno.h>
> > +#include <linux/stddef.h>
> > +#include <linux/slab.h>
> > +#include <linux/module.h>
> > +#include <linux/string.h>
> > +#include <linux/ltt-relay.h>
> > +#include <linux/vmalloc.h>
> > +#include <linux/mm.h>
> > +#include <linux/cpu.h>
> > +#include <linux/splice.h>
> > +#include <linux/bitops.h>
> > +
> > +/* list of open channels, for cpu hotplug */
> > +static DEFINE_MUTEX(relay_channels_mutex);
> > +static LIST_HEAD(relay_channels);
> > +
> > +/**
> > + *	relay_alloc_buf - allocate a channel buffer
> > + *	@buf: the buffer struct
> > + *	@size: total size of the buffer
> > + */
> > +static int relay_alloc_buf(struct rchan_buf *buf, size_t *size)
> > +{
> > +	unsigned int i, n_pages;
> > +	struct buf_page *buf_page, *n;
> > +
> > +	*size = PAGE_ALIGN(*size);
> > +	n_pages = *size >> PAGE_SHIFT;
> > +
> > +	INIT_LIST_HEAD(&buf->pages);
> > +
> > +	for (i = 0; i < n_pages; i++) {
> > +		buf_page = kmalloc_node(sizeof(*buf_page), GFP_KERNEL,
> > +			cpu_to_node(buf->cpu));
> > +		if (unlikely(!buf_page))
> > +			goto depopulate;
> > +		buf_page->page = alloc_pages_node(cpu_to_node(buf->cpu),
> > +			GFP_KERNEL | __GFP_ZERO, 0);
> 
> So these buffers allocate single pages?
> 

Each buffer allocate n_pages. Each of these pages are allocated with
alloc_pages_node(), and linked together in a linked list.

I use a linked list of pages rather than a page pointer array to make
sure I never depend on vmalloc if the page pointer array grows too
large. vmalloc'd data can trigger page faults, which I don't want. Note
that an alternate option to this would be to use vmalloc_sync_all()
after the pages have been allocated, but I prefer not to use vmalloc at
all unless there is a clear advantage.

Note that the tracer calls vmalloc_sync_all() for each tracer module
registered. Actually, the probe modules should also call
vmalloc_sync_all() before their callback gets registered. This would
ensure the module code and data would never trigger a page fault.

I actually wonder why we don't add a vmalloc_sync_all() call in
module.c ? this is a slow path anyway...

> > +		if (unlikely(!buf_page->page)) {
> > +			kfree(buf_page);
> > +			goto depopulate;
> > +		}
> > +		list_add_tail(&buf_page->list, &buf->pages);
> > +		buf_page->offset = (size_t)i << PAGE_SHIFT;
> > +		set_page_private(buf_page->page, (unsigned long)buf_page);
> > +		if (i == 0) {
> > +			buf->wpage = buf_page;
> > +			buf->hpage[0] = buf_page;
> > +			buf->hpage[1] = buf_page;
> > +			buf->rpage = buf_page;
> > +		}
> > +	}
> > +	buf->page_count = n_pages;
> > +	return 0;
> > +
> > +depopulate:
> > +	list_for_each_entry_safe(buf_page, n, &buf->pages, list) {
> > +		list_del_init(&buf_page->list);
> > +		__free_page(buf_page->page);
> > +		kfree(buf_page);
> > +	}
> > +	return -ENOMEM;
> > +}
> > +
> > +/**
> > + *	relay_create_buf - allocate and initialize a channel buffer
> > + *	@chan: the relay channel
> > + *	@cpu: cpu the buffer belongs to
> > + *
> > + *	Returns channel buffer if successful, %NULL otherwise.
> > + */
> > +static struct rchan_buf *relay_create_buf(struct rchan *chan, int cpu)
> > +{
> > +	int ret;
> > +	struct rchan_buf *buf = kzalloc(sizeof(struct rchan_buf), GFP_KERNEL);
> > +	if (!buf)
> > +		return NULL;
> > +
> > +	buf->cpu = cpu;
> > +	ret = relay_alloc_buf(buf, &chan->alloc_size);
> > +	if (ret)
> > +		goto free_buf;
> > +
> > +	buf->chan = chan;
> > +	kref_get(&buf->chan->kref);
> > +	return buf;
> > +
> > +free_buf:
> > +	kfree(buf);
> > +	return NULL;
> > +}
> > +
> > +/**
> > + *	relay_destroy_channel - free the channel struct
> > + *	@kref: target kernel reference that contains the relay channel
> > + *
> > + *	Should only be called from kref_put().
> > + */
> > +static void relay_destroy_channel(struct kref *kref)
> > +{
> > +	struct rchan *chan = container_of(kref, struct rchan, kref);
> > +	kfree(chan);
> > +}
> > +
> > +void ltt_relay_get_chan(struct rchan *chan)
> > +{
> > +	kref_get(&chan->kref);
> > +}
> > +EXPORT_SYMBOL_GPL(ltt_relay_get_chan);
> > +
> > +void ltt_relay_put_chan(struct rchan *chan)
> > +{
> > +	kref_put(&chan->kref, relay_destroy_channel);
> > +}
> > +EXPORT_SYMBOL_GPL(ltt_relay_put_chan);
> > +
> > +/**
> > + *	relay_destroy_buf - destroy an rchan_buf struct and associated buffer
> > + *	@buf: the buffer struct
> > + */
> > +static void relay_destroy_buf(struct rchan_buf *buf)
> > +{
> > +	struct rchan *chan = buf->chan;
> > +	struct buf_page *buf_page, *n;
> > +
> > +	list_for_each_entry_safe(buf_page, n, &buf->pages, list) {
> > +		list_del_init(&buf_page->list);
> > +		__free_page(buf_page->page);
> > +		kfree(buf_page);
> > +	}
> > +	chan->buf[buf->cpu] = NULL;
> > +	kfree(buf);
> > +	kref_put(&chan->kref, relay_destroy_channel);
> > +}
> > +
> > +/**
> > + *	relay_remove_buf - remove a channel buffer
> > + *	@kref: target kernel reference that contains the relay buffer
> > + *
> > + *	Removes the file from the fileystem, which also frees the
> > + *	rchan_buf_struct and the channel buffer.  Should only be called from
> > + *	kref_put().
> > + */
> > +static void relay_remove_buf(struct kref *kref)
> > +{
> > +	struct rchan_buf *buf = container_of(kref, struct rchan_buf, kref);
> > +	buf->chan->cb->remove_buf_file(buf->dentry);
> > +	relay_destroy_buf(buf);
> > +}
> > +
> > +void ltt_relay_get_chan_buf(struct rchan_buf *buf)
> > +{
> > +	kref_get(&buf->kref);
> > +}
> > +EXPORT_SYMBOL_GPL(ltt_relay_get_chan_buf);
> > +
> > +void ltt_relay_put_chan_buf(struct rchan_buf *buf)
> > +{
> > +	kref_put(&buf->kref, relay_remove_buf);
> > +}
> > +EXPORT_SYMBOL_GPL(ltt_relay_put_chan_buf);
> > +
> > +/*
> > + * High-level relay kernel API and associated functions.
> > + */
> > +
> > +/*
> > + * rchan_callback implementations defining default channel behavior.  Used
> > + * in place of corresponding NULL values in client callback struct.
> > + */
> > +
> > +/*
> > + * create_buf_file_create() default callback.  Does nothing.
> > + */
> > +static struct dentry *create_buf_file_default_callback(const char *filename,
> > +						       struct dentry *parent,
> > +						       int mode,
> > +						       struct rchan_buf *buf)
> > +{
> > +	return NULL;
> > +}
> > +
> > +/*
> > + * remove_buf_file() default callback.  Does nothing.
> > + */
> > +static int remove_buf_file_default_callback(struct dentry *dentry)
> > +{
> > +	return -EINVAL;
> > +}
> > +
> > +/* relay channel default callbacks */
> > +static struct rchan_callbacks default_channel_callbacks = {
> > +	.create_buf_file = create_buf_file_default_callback,
> > +	.remove_buf_file = remove_buf_file_default_callback,
> > +};
> > +
> > +/**
> > + *	__relay_reset - reset a channel buffer
> > + *	@buf: the channel buffer
> > + *	@init: 1 if this is a first-time initialization
> > + *
> > + *	See relay_reset() for description of effect.
> > + */
> > +static void __relay_reset(struct rchan_buf *buf, unsigned int init)
> > +{
> > +	if (init)
> > +		kref_init(&buf->kref);
> > +}
> > +
> > +/*
> > + *	relay_open_buf - create a new relay channel buffer
> > + *
> > + *	used by relay_open() and CPU hotplug.
> > + */
> > +static struct rchan_buf *relay_open_buf(struct rchan *chan, unsigned int cpu)
> > +{
> > +	struct rchan_buf *buf = NULL;
> > +	struct dentry *dentry;
> > +	char *tmpname;
> > +
> > +	tmpname = kzalloc(NAME_MAX + 1, GFP_KERNEL);
> > +	if (!tmpname)
> > +		goto end;
> > +	snprintf(tmpname, NAME_MAX, "%s%d", chan->base_filename, cpu);
> > +
> > +	buf = relay_create_buf(chan, cpu);
> > +	if (!buf)
> > +		goto free_name;
> > +
> > +	__relay_reset(buf, 1);
> > +
> > +	/* Create file in fs */
> > +	dentry = chan->cb->create_buf_file(tmpname, chan->parent, S_IRUSR,
> > +					   buf);
> > +	if (!dentry)
> > +		goto free_buf;
> > +
> > +	buf->dentry = dentry;
> > +
> > +	goto free_name;
> > +
> > +free_buf:
> > +	relay_destroy_buf(buf);
> > +	buf = NULL;
> > +free_name:
> > +	kfree(tmpname);
> > +end:
> > +	return buf;
> > +}
> > +
> > +/**
> > + *	relay_close_buf - close a channel buffer
> > + *	@buf: channel buffer
> > + *
> > + *	Restores the default callbacks.
> > + *	The channel buffer and channel buffer data structure are then freed
> > + *	automatically when the last reference is given up.
> > + */
> > +static void relay_close_buf(struct rchan_buf *buf)
> > +{
> > +	kref_put(&buf->kref, relay_remove_buf);
> > +}
> > +
> > +static void setup_callbacks(struct rchan *chan,
> > +				   struct rchan_callbacks *cb)
> > +{
> > +	if (!cb) {
> > +		chan->cb = &default_channel_callbacks;
> > +		return;
> > +	}
> > +
> > +	if (!cb->create_buf_file)
> > +		cb->create_buf_file = create_buf_file_default_callback;
> > +	if (!cb->remove_buf_file)
> > +		cb->remove_buf_file = remove_buf_file_default_callback;
> > +	chan->cb = cb;
> > +}
> > +
> > +/**
> > + * 	relay_hotcpu_callback - CPU hotplug callback
> > + * 	@nb: notifier block
> > + * 	@action: hotplug action to take
> > + * 	@hcpu: CPU number
> > + *
> > + * 	Returns the success/failure of the operation. (%NOTIFY_OK, %NOTIFY_BAD)
> > + */
> > +static int __cpuinit relay_hotcpu_callback(struct notifier_block *nb,
> > +				unsigned long action,
> > +				void *hcpu)
> > +{
> > +	unsigned int hotcpu = (unsigned long)hcpu;
> > +	struct rchan *chan;
> > +
> > +	switch (action) {
> > +	case CPU_UP_PREPARE:
> > +	case CPU_UP_PREPARE_FROZEN:
> > +		mutex_lock(&relay_channels_mutex);
> > +		list_for_each_entry(chan, &relay_channels, list) {
> > +			if (chan->buf[hotcpu])
> > +				continue;
> > +			chan->buf[hotcpu] = relay_open_buf(chan, hotcpu);
> > +			if (!chan->buf[hotcpu]) {
> > +				printk(KERN_ERR
> > +					"relay_hotcpu_callback: cpu %d buffer "
> > +					"creation failed\n", hotcpu);
> > +				mutex_unlock(&relay_channels_mutex);
> > +				return NOTIFY_BAD;
> > +			}
> > +		}
> > +		mutex_unlock(&relay_channels_mutex);
> > +		break;
> > +	case CPU_DEAD:
> > +	case CPU_DEAD_FROZEN:
> > +		/* No need to flush the cpu : will be flushed upon
> > +		 * final relay_flush() call. */
> > +		break;
> > +	}
> > +	return NOTIFY_OK;
> > +}
> > +
> > +/**
> > + *	ltt_relay_open - create a new relay channel
> > + *	@base_filename: base name of files to create
> > + *	@parent: dentry of parent directory, %NULL for root directory
> > + *	@subbuf_size: size of sub-buffers
> > + *	@n_subbufs: number of sub-buffers
> > + *	@cb: client callback functions
> > + *	@private_data: user-defined data
> > + *
> > + *	Returns channel pointer if successful, %NULL otherwise.
> > + *
> > + *	Creates a channel buffer for each cpu using the sizes and
> > + *	attributes specified.  The created channel buffer files
> > + *	will be named base_filename0...base_filenameN-1.  File
> > + *	permissions will be %S_IRUSR.
> > + */
> > +struct rchan *ltt_relay_open(const char *base_filename,
> > +			 struct dentry *parent,
> > +			 size_t subbuf_size,
> > +			 size_t n_subbufs,
> > +			 struct rchan_callbacks *cb,
> > +			 void *private_data)
> > +{
> > +	unsigned int i;
> > +	struct rchan *chan;
> > +	if (!base_filename)
> > +		return NULL;
> > +
> > +	if (!(subbuf_size && n_subbufs))
> > +		return NULL;
> > +
> > +	chan = kzalloc(sizeof(struct rchan), GFP_KERNEL);
> > +	if (!chan)
> > +		return NULL;
> > +
> > +	chan->version = LTT_RELAY_CHANNEL_VERSION;
> > +	chan->n_subbufs = n_subbufs;
> > +	chan->subbuf_size = subbuf_size;
> 
> You declare the sub buf size here, but I do not see how it gets allocated.
> 

below :

chan->alloc_size = FIX_SIZE(subbuf_size * n_subbufs);

Then :

relay_open_buf()
  relay_create_buf()
    relay_alloc_buf(buf, &chan->alloc_size); 

And there we iterate on n_pages, this is PAGE_ALIGN(*size) >> PAGE_SHIFT

> > +static int relay_alloc_buf(struct rchan_buf *buf, size_t *size)
> > +{
> > +   unsigned int i, n_pages;
> > +   struct buf_page *buf_page, *n;
> > +
> > +   *size = PAGE_ALIGN(*size);
> > +   n_pages = *size >> PAGE_SHIFT;
> > +
> > +   INIT_LIST_HEAD(&buf->pages);
> > +
> > +   for (i = 0; i < n_pages; i++) {

Mathieu


> -- Steve
> 
> > +	chan->subbuf_size_order = get_count_order(subbuf_size);
> > +	chan->alloc_size = FIX_SIZE(subbuf_size * n_subbufs);
> > +	chan->parent = parent;
> > +	chan->private_data = private_data;
> > +	strlcpy(chan->base_filename, base_filename, NAME_MAX);
> > +	setup_callbacks(chan, cb);
> > +	kref_init(&chan->kref);
> > +
> > +	mutex_lock(&relay_channels_mutex);
> > +	for_each_online_cpu(i) {
> > +		chan->buf[i] = relay_open_buf(chan, i);
> > +		if (!chan->buf[i])
> > +			goto free_bufs;
> > +	}
> > +	list_add(&chan->list, &relay_channels);
> > +	mutex_unlock(&relay_channels_mutex);
> > +
> > +	return chan;
> > +
> > +free_bufs:
> > +	for_each_possible_cpu(i) {
> > +		if (!chan->buf[i])
> > +			break;
> > +		relay_close_buf(chan->buf[i]);
> > +	}
> > +
> > +	kref_put(&chan->kref, relay_destroy_channel);
> > +	mutex_unlock(&relay_channels_mutex);
> > +	return NULL;
> > +}
> > +EXPORT_SYMBOL_GPL(ltt_relay_open);
> > +
> > +/**
> > + *	ltt_relay_close - close the channel
> > + *	@chan: the channel
> > + *
> > + *	Closes all channel buffers and frees the channel.
> > + */
> > +void ltt_relay_close(struct rchan *chan)
> > +{
> > +	unsigned int i;
> > +
> > +	if (!chan)
> > +		return;
> > +
> > +	mutex_lock(&relay_channels_mutex);
> > +	for_each_possible_cpu(i)
> > +		if (chan->buf[i])
> > +			relay_close_buf(chan->buf[i]);
> > +
> > +	list_del(&chan->list);
> > +	kref_put(&chan->kref, relay_destroy_channel);
> > +	mutex_unlock(&relay_channels_mutex);
> > +}
> > +EXPORT_SYMBOL_GPL(ltt_relay_close);
> > +
> > +/*
> > + * Start iteration at the previous element. Skip the real list head.
> > + */
> > +static struct buf_page *ltt_relay_find_prev_page(struct rchan_buf *buf,
> > +	struct buf_page *page, size_t offset, ssize_t diff_offset)
> > +{
> > +	struct buf_page *iter;
> > +	size_t orig_iter_off;
> > +	unsigned int i = 0;
> > +
> > +	orig_iter_off = page->offset;
> > +	list_for_each_entry_reverse(iter, &page->list, list) {
> > +		/*
> > +		 * Skip the real list head.
> > +		 */
> > +		if (&iter->list == &buf->pages)
> > +			continue;
> > +		i++;
> > +		if (offset >= iter->offset
> > +			&& offset < iter->offset + PAGE_SIZE) {
> > +#ifdef CONFIG_LTT_RELAY_CHECK_RANDOM_ACCESS
> > +			if (i > 1) {
> > +				printk(KERN_WARNING
> > +					"Backward random access detected in "
> > +					"ltt_relay. Iterations %u, "
> > +					"offset %zu, orig iter->off %zu, "
> > +					"iter->off %zu diff_offset %zd.\n", i,
> > +					offset, orig_iter_off, iter->offset,
> > +					diff_offset);
> > +				WARN_ON(1);
> > +			}
> > +#endif
> > +			return iter;
> > +		}
> > +	}
> > +	return NULL;
> > +}
> > +
> > +/*
> > + * Start iteration at the next element. Skip the real list head.
> > + */
> > +static struct buf_page *ltt_relay_find_next_page(struct rchan_buf *buf,
> > +	struct buf_page *page, size_t offset, ssize_t diff_offset)
> > +{
> > +	struct buf_page *iter;
> > +	unsigned int i = 0;
> > +	size_t orig_iter_off;
> > +
> > +	orig_iter_off = page->offset;
> > +	list_for_each_entry(iter, &page->list, list) {
> > +		/*
> > +		 * Skip the real list head.
> > +		 */
> > +		if (&iter->list == &buf->pages)
> > +			continue;
> > +		i++;
> > +		if (offset >= iter->offset
> > +			&& offset < iter->offset + PAGE_SIZE) {
> > +#ifdef CONFIG_LTT_RELAY_CHECK_RANDOM_ACCESS
> > +			if (i > 1) {
> > +				printk(KERN_WARNING
> > +					"Forward random access detected in "
> > +					"ltt_relay. Iterations %u, "
> > +					"offset %zu, orig iter->off %zu, "
> > +					"iter->off %zu diff_offset %zd.\n", i,
> > +					offset, orig_iter_off, iter->offset,
> > +					diff_offset);
> > +				WARN_ON(1);
> > +			}
> > +#endif
> > +			return iter;
> > +		}
> > +	}
> > +	return NULL;
> > +}
> > +
> > +/*
> > + * Find the page containing "offset". Cache it if it is after the currently
> > + * cached page.
> > + */
> > +static struct buf_page *ltt_relay_cache_page(struct rchan_buf *buf,
> > +		struct buf_page **page_cache,
> > +		struct buf_page *page, size_t offset)
> > +{
> > +	ssize_t diff_offset;
> > +	ssize_t half_buf_size = buf->chan->alloc_size >> 1;
> > +
> > +	/*
> > +	 * Make sure this is the page we want to write into. The current
> > +	 * page is changed concurrently by other writers. [wrh]page are
> > +	 * used as a cache remembering the last page written
> > +	 * to/read/looked up for header address. No synchronization;
> > +	 * could have to find the previous page is a nested write
> > +	 * occured. Finding the right page is done by comparing the
> > +	 * dest_offset with the buf_page offsets.
> > +	 * When at the exact opposite of the buffer, bias towards forward search
> > +	 * because it will be cached.
> > +	 */
> > +
> > +	diff_offset = (ssize_t)offset - (ssize_t)page->offset;
> > +	if (diff_offset <= -(ssize_t)half_buf_size)
> > +		diff_offset += buf->chan->alloc_size;
> > +	else if (diff_offset > half_buf_size)
> > +		diff_offset -= buf->chan->alloc_size;
> > +
> > +	if (unlikely(diff_offset >= (ssize_t)PAGE_SIZE)) {
> > +		page = ltt_relay_find_next_page(buf, page, offset, diff_offset);
> > +		WARN_ON(!page);
> > +		*page_cache = page;
> > +	} else if (unlikely(diff_offset < 0)) {
> > +		page = ltt_relay_find_prev_page(buf, page, offset, diff_offset);
> > +		WARN_ON(!page);
> > +	}
> > +	return page;
> > +}
> > +
> > +/**
> > + * ltt_relay_write - write data to a ltt_relay buffer.
> > + * @buf : buffer
> > + * @offset : offset within the buffer
> > + * @src : source address
> > + * @len : length to write
> > + */
> > +int ltt_relay_write(struct rchan_buf *buf, size_t offset,
> > +	const void *src, size_t len)
> > +{
> > +	struct buf_page *page;
> > +	ssize_t pagecpy, orig_len;
> > +
> > +	orig_len = len;
> > +	offset &= buf->chan->alloc_size - 1;
> > +	page = buf->wpage;
> > +	if (unlikely(!len))
> > +		return 0;
> > +	for (;;) {
> > +		page = ltt_relay_cache_page(buf, &buf->wpage, page, offset);
> > +		pagecpy = min_t(size_t, len, PAGE_SIZE - (offset & ~PAGE_MASK));
> > +		memcpy(page_address(page->page)
> > +			+ (offset & ~PAGE_MASK), src, pagecpy);
> > +		len -= pagecpy;
> > +		if (likely(!len))
> > +			break;
> > +		src += pagecpy;
> > +		offset += pagecpy;
> > +		/*
> > +		 * Underlying layer should never ask for writes across
> > +		 * subbuffers.
> > +		 */
> > +		WARN_ON(offset >= buf->chan->alloc_size);
> > +	}
> > +	return orig_len;
> > +}
> > +EXPORT_SYMBOL_GPL(ltt_relay_write);
> > +
> > +/**
> > + * ltt_relay_read - read data from ltt_relay_buffer.
> > + * @buf : buffer
> > + * @offset : offset within the buffer
> > + * @dest : destination address
> > + * @len : length to write
> > + */
> > +int ltt_relay_read(struct rchan_buf *buf, size_t offset,
> > +	void *dest, size_t len)
> > +{
> > +	struct buf_page *page;
> > +	ssize_t pagecpy, orig_len;
> > +
> > +	orig_len = len;
> > +	offset &= buf->chan->alloc_size - 1;
> > +	page = buf->rpage;
> > +	if (unlikely(!len))
> > +		return 0;
> > +	for (;;) {
> > +		page = ltt_relay_cache_page(buf, &buf->rpage, page, offset);
> > +		pagecpy = min_t(size_t, len, PAGE_SIZE - (offset & ~PAGE_MASK));
> > +		memcpy(dest, page_address(page->page) + (offset & ~PAGE_MASK),
> > +			pagecpy);
> > +		len -= pagecpy;
> > +		if (likely(!len))
> > +			break;
> > +		dest += pagecpy;
> > +		offset += pagecpy;
> > +		/*
> > +		 * Underlying layer should never ask for reads across
> > +		 * subbuffers.
> > +		 */
> > +		WARN_ON(offset >= buf->chan->alloc_size);
> > +	}
> > +	return orig_len;
> > +}
> > +EXPORT_SYMBOL_GPL(ltt_relay_read);
> > +
> > +/**
> > + * ltt_relay_read_get_page - Get a whole page to read from
> > + * @buf : buffer
> > + * @offset : offset within the buffer
> > + */
> > +struct buf_page *ltt_relay_read_get_page(struct rchan_buf *buf, size_t offset)
> > +{
> > +	struct buf_page *page;
> > +
> > +	offset &= buf->chan->alloc_size - 1;
> > +	page = buf->rpage;
> > +	page = ltt_relay_cache_page(buf, &buf->rpage, page, offset);
> > +	return page;
> > +}
> > +EXPORT_SYMBOL_GPL(ltt_relay_read_get_page);
> > +
> > +/**
> > + * ltt_relay_offset_address - get address of a location within the buffer
> > + * @buf : buffer
> > + * @offset : offset within the buffer.
> > + *
> > + * Return the address where a given offset is located.
> > + * Should be used to get the current subbuffer header pointer. Given we know
> > + * it's never on a page boundary, it's safe to write directly to this address,
> > + * as long as the write is never bigger than a page size.
> > + */
> > +void *ltt_relay_offset_address(struct rchan_buf *buf, size_t offset)
> > +{
> > +	struct buf_page *page;
> > +	unsigned int odd;
> > +
> > +	offset &= buf->chan->alloc_size - 1;
> > +	odd = !!(offset & buf->chan->subbuf_size);
> > +	page = buf->hpage[odd];
> > +	if (offset < page->offset || offset >= page->offset + PAGE_SIZE)
> > +		buf->hpage[odd] = page = buf->wpage;
> > +	page = ltt_relay_cache_page(buf, &buf->hpage[odd], page, offset);
> > +	return page_address(page->page) + (offset & ~PAGE_MASK);
> > +}
> > +EXPORT_SYMBOL_GPL(ltt_relay_offset_address);
> > +
> > +/**
> > + *	relay_file_open - open file op for relay files
> > + *	@inode: the inode
> > + *	@filp: the file
> > + *
> > + *	Increments the channel buffer refcount.
> > + */
> > +static int relay_file_open(struct inode *inode, struct file *filp)
> > +{
> > +	struct rchan_buf *buf = inode->i_private;
> > +	kref_get(&buf->kref);
> > +	filp->private_data = buf;
> > +
> > +	return nonseekable_open(inode, filp);
> > +}
> > +
> > +/**
> > + *	relay_file_release - release file op for relay files
> > + *	@inode: the inode
> > + *	@filp: the file
> > + *
> > + *	Decrements the channel refcount, as the filesystem is
> > + *	no longer using it.
> > + */
> > +static int relay_file_release(struct inode *inode, struct file *filp)
> > +{
> > +	struct rchan_buf *buf = filp->private_data;
> > +	kref_put(&buf->kref, relay_remove_buf);
> > +
> > +	return 0;
> > +}
> > +
> > +const struct file_operations ltt_relay_file_operations = {
> > +	.open		= relay_file_open,
> > +	.release	= relay_file_release,
> > +};
> > +EXPORT_SYMBOL_GPL(ltt_relay_file_operations);
> > +
> > +static __init int relay_init(void)
> > +{
> > +	hotcpu_notifier(relay_hotcpu_callback, 5);
> > +	return 0;
> > +}
> > +
> > +module_init(relay_init);
> > Index: linux-2.6-lttng/include/linux/ltt-relay.h
> > ===================================================================
> > --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> > +++ linux-2.6-lttng/include/linux/ltt-relay.h	2009-03-05 15:05:56.000000000 -0500
> > @@ -0,0 +1,182 @@
> > +/*
> > + * linux/include/linux/ltt-relay.h
> > + *
> > + * Copyright (C) 2002, 2003 - Tom Zanussi (zanussi at us.ibm.com), IBM Corp
> > + * Copyright (C) 1999, 2000, 2001, 2002 - Karim Yaghmour (karim at opersys.com)
> > + * Copyright (C) 2008 - Mathieu Desnoyers (mathieu.desnoyers at polymtl.ca)
> > + *
> > + * CONFIG_RELAY definitions and declarations
> > + */
> > +
> > +#ifndef _LINUX_LTT_RELAY_H
> > +#define _LINUX_LTT_RELAY_H
> > +
> > +#include <linux/types.h>
> > +#include <linux/sched.h>
> > +#include <linux/timer.h>
> > +#include <linux/wait.h>
> > +#include <linux/list.h>
> > +#include <linux/fs.h>
> > +#include <linux/poll.h>
> > +#include <linux/kref.h>
> > +#include <linux/mm.h>
> > +
> > +/* Needs a _much_ better name... */
> > +#define FIX_SIZE(x) ((((x) - 1) & PAGE_MASK) + PAGE_SIZE)
> > +
> > +/*
> > + * Tracks changes to rchan/rchan_buf structs
> > + */
> > +#define LTT_RELAY_CHANNEL_VERSION		8
> > +
> > +struct rchan_buf;
> > +
> > +struct buf_page {
> > +	struct page *page;
> > +	size_t offset;		/* page offset in the buffer */
> > +	struct list_head list;	/* buffer linked list */
> > +};
> > +
> > +/*
> > + * Per-cpu relay channel buffer
> > + */
> > +struct rchan_buf {
> > +	void *chan_private;		/* private data for this buf */
> > +	struct rchan *chan;		/* associated channel */
> > +	struct dentry *dentry;		/* channel file dentry */
> > +	struct kref kref;		/* channel buffer refcount */
> > +	struct list_head pages;		/* list of buffer pages */
> > +	struct buf_page *wpage;		/* current write page (cache) */
> > +	struct buf_page *hpage[2];	/* current subbuf header page (cache) */
> > +	struct buf_page *rpage;		/* current subbuf read page (cache) */
> > +	unsigned int page_count;	/* number of current buffer pages */
> > +	unsigned int cpu;		/* this buf's cpu */
> > +} ____cacheline_aligned;
> > +
> > +/*
> > + * Relay channel data structure
> > + */
> > +struct rchan {
> > +	u32 version;			/* the version of this struct */
> > +	size_t subbuf_size;		/* sub-buffer size */
> > +	size_t n_subbufs;		/* number of sub-buffers per buffer */
> > +	size_t alloc_size;		/* total buffer size allocated */
> > +	struct rchan_callbacks *cb;	/* client callbacks */
> > +	struct kref kref;		/* channel refcount */
> > +	void *private_data;		/* for user-defined data */
> > +	struct rchan_buf *buf[NR_CPUS]; /* per-cpu channel buffers */
> > +	struct list_head list;		/* for channel list */
> > +	struct dentry *parent;		/* parent dentry passed to open */
> > +	int subbuf_size_order;		/* order of sub-buffer size */
> > +	char base_filename[NAME_MAX];	/* saved base filename */
> > +};
> > +
> > +/*
> > + * Relay channel client callbacks
> > + */
> > +struct rchan_callbacks {
> > +	/*
> > +	 * subbuf_start - called on buffer-switch to a new sub-buffer
> > +	 * @buf: the channel buffer containing the new sub-buffer
> > +	 * @subbuf: the start of the new sub-buffer
> > +	 * @prev_subbuf: the start of the previous sub-buffer
> > +	 * @prev_padding: unused space at the end of previous sub-buffer
> > +	 *
> > +	 * The client should return 1 to continue logging, 0 to stop
> > +	 * logging.
> > +	 *
> > +	 * NOTE: subbuf_start will also be invoked when the buffer is
> > +	 *       created, so that the first sub-buffer can be initialized
> > +	 *       if necessary.  In this case, prev_subbuf will be NULL.
> > +	 *
> > +	 * NOTE: the client can reserve bytes at the beginning of the new
> > +	 *       sub-buffer by calling subbuf_start_reserve() in this callback.
> > +	 */
> > +	int (*subbuf_start) (struct rchan_buf *buf,
> > +			     void *subbuf,
> > +			     void *prev_subbuf,
> > +			     size_t prev_padding);
> > +
> > +	/*
> > +	 * create_buf_file - create file to represent a relay channel buffer
> > +	 * @filename: the name of the file to create
> > +	 * @parent: the parent of the file to create
> > +	 * @mode: the mode of the file to create
> > +	 * @buf: the channel buffer
> > +	 *
> > +	 * Called during relay_open(), once for each per-cpu buffer,
> > +	 * to allow the client to create a file to be used to
> > +	 * represent the corresponding channel buffer.  If the file is
> > +	 * created outside of relay, the parent must also exist in
> > +	 * that filesystem.
> > +	 *
> > +	 * The callback should return the dentry of the file created
> > +	 * to represent the relay buffer.
> > +	 *
> > +	 * Setting the is_global outparam to a non-zero value will
> > +	 * cause relay_open() to create a single global buffer rather
> > +	 * than the default set of per-cpu buffers.
> > +	 *
> > +	 * See Documentation/filesystems/relayfs.txt for more info.
> > +	 */
> > +	struct dentry *(*create_buf_file)(const char *filename,
> > +					  struct dentry *parent,
> > +					  int mode,
> > +					  struct rchan_buf *buf);
> > +
> > +	/*
> > +	 * remove_buf_file - remove file representing a relay channel buffer
> > +	 * @dentry: the dentry of the file to remove
> > +	 *
> > +	 * Called during relay_close(), once for each per-cpu buffer,
> > +	 * to allow the client to remove a file used to represent a
> > +	 * channel buffer.
> > +	 *
> > +	 * The callback should return 0 if successful, negative if not.
> > +	 */
> > +	int (*remove_buf_file)(struct dentry *dentry);
> > +};
> > +
> > +extern int ltt_relay_write(struct rchan_buf *buf, size_t offset,
> > +	const void *src, size_t len);
> > +
> > +extern int ltt_relay_read(struct rchan_buf *buf, size_t offset,
> > +	void *dest, size_t len);
> > +
> > +extern struct buf_page *ltt_relay_read_get_page(struct rchan_buf *buf,
> > +	size_t offset);
> > +
> > +/*
> > + * Return the address where a given offset is located.
> > + * Should be used to get the current subbuffer header pointer. Given we know
> > + * it's never on a page boundary, it's safe to write directly to this address,
> > + * as long as the write is never bigger than a page size.
> > + */
> > +extern void *ltt_relay_offset_address(struct rchan_buf *buf,
> > +	size_t offset);
> > +
> > +/*
> > + * CONFIG_LTT_RELAY kernel API, ltt/ltt-relay-alloc.c
> > + */
> > +
> > +struct rchan *ltt_relay_open(const char *base_filename,
> > +			 struct dentry *parent,
> > +			 size_t subbuf_size,
> > +			 size_t n_subbufs,
> > +			 struct rchan_callbacks *cb,
> > +			 void *private_data);
> > +extern void ltt_relay_close(struct rchan *chan);
> > +
> > +void ltt_relay_get_chan(struct rchan *chan);
> > +void ltt_relay_put_chan(struct rchan *chan);
> > +
> > +void ltt_relay_get_chan_buf(struct rchan_buf *buf);
> > +void ltt_relay_put_chan_buf(struct rchan_buf *buf);
> > +
> > +/*
> > + * exported ltt_relay file operations, ltt/ltt-relay-alloc.c
> > + */
> > +extern const struct file_operations ltt_relay_file_operations;
> > +
> > +#endif /* _LINUX_LTT_RELAY_H */
> > +
> > 
> > -- 
> > Mathieu Desnoyers
> > OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> > 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[ltt-dev] [RFC patch 06/41] LTTng optimize write to page function

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-optimize-write-to-page-function.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/e1ac5b24/attachment.asc>

[ltt-dev] [RFC patch 06/41] LTTng optimize write to page function

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-optimize-write-to-page-function.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/e1ac5b24/attachment-0001.asc>

[ltt-dev] [RFC patch 06/41] LTTng optimize write to page function

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-optimize-write-to-page-function.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/e1ac5b24/attachment.txt>

[ltt-dev] [RFC patch 07/41] LTTng dynamic channels

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-dynamic-channels.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/57ae5d7c/attachment-0001.asc>

[ltt-dev] [RFC patch 07/41] LTTng dynamic channels

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-dynamic-channels.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/57ae5d7c/attachment.asc>

[ltt-dev] [RFC patch 07/41] LTTng dynamic channels

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-dynamic-channels.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/57ae5d7c/attachment.txt>

[ltt-dev] [RFC patch 08/41] LTTng - tracer header

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-tracer-header.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/94d12417/attachment.txt>

[ltt-dev] [RFC patch 08/41] LTTng - tracer header

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-tracer-header.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/94d12417/attachment-0001.asc>

[ltt-dev] [RFC patch 08/41] LTTng - tracer header

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-tracer-header.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/94d12417/attachment.asc>

[ltt-dev] [RFC patch 09/41] LTTng optimize write to page function deal with unaligned access

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-optimize-write-to-page-function-deal-with-unaligned.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/4b2b7917/attachment.txt>

[ltt-dev] [RFC patch 09/41] LTTng optimize write to page function deal with unaligned access

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-optimize-write-to-page-function-deal-with-unaligned.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/4b2b7917/attachment.asc>

[ltt-dev] [RFC patch 09/41] LTTng optimize write to page function deal with unaligned access

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-optimize-write-to-page-function-deal-with-unaligned.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/4b2b7917/attachment-0001.txt>

[ltt-dev] [RFC patch 10/41] lttng-optimize-write-to-page-function-remove-some-memcpy-calls

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-optimize-write-to-page-function-remove-some-memcpy-calls.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/64dde8f4/attachment.asc>

[ltt-dev] [RFC patch 10/41] lttng-optimize-write-to-page-function-remove-some-memcpy-calls

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-optimize-write-to-page-function-remove-some-memcpy-calls.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/64dde8f4/attachment-0001.asc>

[ltt-dev] [RFC patch 10/41] lttng-optimize-write-to-page-function-remove-some-memcpy-calls

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-optimize-write-to-page-function-remove-some-memcpy-calls.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/64dde8f4/attachment.txt>

[ltt-dev] [RFC patch 11/41] ltt-relay: cache pages address

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-optimize-write-to-page-function-cache-page-address.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/eea5edb8/attachment-0001.asc>

[ltt-dev] [RFC patch 11/41] ltt-relay: cache pages address

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-optimize-write-to-page-function-cache-page-address.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/eea5edb8/attachment.asc>

[ltt-dev] [RFC patch 11/41] ltt-relay: cache pages address

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-optimize-write-to-page-function-cache-page-address.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/eea5edb8/attachment.txt>

[ltt-dev] [RFC patch 12/41] x86 : export vmalloc_sync_all()

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: export-vmalloc-sync-all-symbol.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/76074d2e/attachment.asc>

[ltt-dev] [RFC patch 12/41] x86 : export vmalloc_sync_all()

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: export-vmalloc-sync-all-symbol.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/76074d2e/attachment.txt>

[ltt-dev] [RFC patch 12/41] x86 : export vmalloc_sync_all()

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: export-vmalloc-sync-all-symbol.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/76074d2e/attachment-0001.asc>

[ltt-dev] [RFC patch 13/41] LTTng - tracer code

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-tracer.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/4ea4bfd5/attachment-0001.asc>

[ltt-dev] [RFC patch 13/41] LTTng - tracer code

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-tracer.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/4ea4bfd5/attachment.asc>

[ltt-dev] [RFC patch 13/41] LTTng - tracer code

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-tracer.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/4ea4bfd5/attachment.txt>

[ltt-dev] [RFC patch 14/41] Splice and pipe : export pipe buf operations for GPL modules

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: splice-support-modules.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/61e3df9b/attachment-0001.txt>

[ltt-dev] [RFC patch 14/41] Splice and pipe : export pipe buf operations for GPL modules

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: splice-support-modules.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/61e3df9b/attachment.txt>

[ltt-dev] [RFC patch 14/41] Splice and pipe : export pipe buf operations for GPL modules

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: splice-support-modules.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/61e3df9b/attachment.asc>

[ltt-dev] [RFC patch 15/41] Poll : add poll_wait_set_exclusive

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: poll-wait-exclusive.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/b928e014/attachment.txt>

[ltt-dev] [RFC patch 15/41] Poll : add poll_wait_set_exclusive

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: poll-wait-exclusive.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/b928e014/attachment-0001.txt>

[ltt-dev] [RFC patch 15/41] Poll : add poll_wait_set_exclusive

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: poll-wait-exclusive.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/b928e014/attachment.asc>

[ltt-dev] [RFC patch 16/41] LTTng Transport Locked

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-transport-locked.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/921f4ecf/attachment.asc>

[ltt-dev] [RFC patch 16/41] LTTng Transport Locked

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-transport-locked.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/921f4ecf/attachment.txt>

[ltt-dev] [RFC patch 16/41] LTTng Transport Locked

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-transport-locked.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/921f4ecf/attachment-0001.asc>

[ltt-dev] [RFC patch 17/41] LTTng - serialization

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-serialize.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/6bc3312e/attachment.asc>

[ltt-dev] [RFC patch 17/41] LTTng - serialization

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-serialize.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/6bc3312e/attachment-0001.txt>

[ltt-dev] [RFC patch 17/41] LTTng - serialization

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-serialize.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/6bc3312e/attachment.txt>

[ltt-dev] [RFC patch 18/41] Seq_file add support for sorted list

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: seq_file_sorted.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/f36601f2/attachment-0001.txt>

[ltt-dev] [RFC patch 18/41] Seq_file add support for sorted list

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: seq_file_sorted.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/f36601f2/attachment.txt>

[ltt-dev] [RFC patch 18/41] Seq_file add support for sorted list

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: seq_file_sorted.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/f36601f2/attachment.asc>

[ltt-dev] [RFC patch 19/41] Sort module list by pointer address to get coherent sleepable seq_file iterators

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: module.c-sort-module-list.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/e0313512/attachment.asc>

[ltt-dev] [RFC patch 19/41] Sort module list by pointer address to get coherent sleepable seq_file iterators

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: module.c-sort-module-list.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/e0313512/attachment-0001.asc>

[ltt-dev] [RFC patch 19/41] Sort module list by pointer address to get coherent sleepable seq_file iterators

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: module.c-sort-module-list.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/e0313512/attachment.txt>

[ltt-dev] [RFC patch 20/41] Linux Kernel Markers - Iterator

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: linux-kernel-markers-iterator.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/16b20619/attachment-0001.asc>

[ltt-dev] [RFC patch 20/41] Linux Kernel Markers - Iterator

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: linux-kernel-markers-iterator.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/16b20619/attachment.txt>

[ltt-dev] [RFC patch 20/41] Linux Kernel Markers - Iterator

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: linux-kernel-markers-iterator.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/16b20619/attachment.asc>

[ltt-dev] [RFC patch 21/41] LTTng probes specialized tracepoints

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-type-serializer.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/a7e773bc/attachment-0001.asc>

[ltt-dev] [RFC patch 21/41] LTTng probes specialized tracepoints

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-type-serializer.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/a7e773bc/attachment.asc>

[ltt-dev] [RFC patch 21/41] LTTng probes specialized tracepoints

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-type-serializer.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/a7e773bc/attachment.txt>

[ltt-dev] [RFC patch 22/41] LTTng marker control

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-marker-control.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/f7c93149/attachment-0001.txt>

[ltt-dev] [RFC patch 22/41] LTTng marker control

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-marker-control.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/f7c93149/attachment.txt>

[ltt-dev] [RFC patch 22/41] LTTng marker control

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-marker-control.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/f7c93149/attachment.asc>

[ltt-dev] [RFC patch 23/41] Immediate Values Stub header

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: immediate-value-stub-header.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/c237b0f8/attachment-0001.asc>

[ltt-dev] [RFC patch 23/41] Immediate Values Stub header

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: immediate-value-stub-header.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/c237b0f8/attachment.asc>

[ltt-dev] [RFC patch 23/41] Immediate Values Stub header

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: immediate-value-stub-header.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/c237b0f8/attachment.txt>

[ltt-dev] [RFC patch 24/41] Linux Kernel Markers - Use Immediate Values

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: linux-kernel-markers-immediate-values.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/190c137e/attachment-0001.txt>

[ltt-dev] [RFC patch 24/41] Linux Kernel Markers - Use Immediate Values

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: linux-kernel-markers-immediate-values.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/190c137e/attachment.asc>

[ltt-dev] [RFC patch 24/41] Linux Kernel Markers - Use Immediate Values

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: linux-kernel-markers-immediate-values.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/190c137e/attachment.txt>

[ltt-dev] [RFC patch 25/41] Markers Support for Proprierary Modules

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-support-for-proprietary-modules.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/25a6c3f7/attachment.txt>

[ltt-dev] [RFC patch 25/41] Markers Support for Proprierary Modules

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-support-for-proprietary-modules.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/25a6c3f7/attachment-0001.asc>

[ltt-dev] [RFC patch 25/41] Markers Support for Proprierary Modules

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-support-for-proprietary-modules.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/25a6c3f7/attachment.asc>

[ltt-dev] [RFC patch 26/41] Marers remove old comment

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-remove-old-comment.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/bf3a8f56/attachment.txt>

[ltt-dev] [RFC patch 26/41] Marers remove old comment

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-remove-old-comment.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/bf3a8f56/attachment-0001.txt>

[ltt-dev] [RFC patch 26/41] Marers remove old comment

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-remove-old-comment.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/bf3a8f56/attachment.asc>

[ltt-dev] [RFC patch 27/41] Markers use dynamic channels

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-use-dynamic-channels.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/1a8baba2/attachment.asc>

[ltt-dev] [RFC patch 27/41] Markers use dynamic channels

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-use-dynamic-channels.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/1a8baba2/attachment-0001.asc>

[ltt-dev] [RFC patch 27/41] Markers use dynamic channels

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-use-dynamic-channels.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/1a8baba2/attachment.txt>

[ltt-dev] [RFC patch 28/41] LTT trace control

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: ltt-trace-control.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/70d390d7/attachment.txt>

[ltt-dev] [RFC patch 28/41] LTT trace control

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: ltt-trace-control.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/70d390d7/attachment.asc>

[ltt-dev] [RFC patch 28/41] LTT trace control

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: ltt-trace-control.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/70d390d7/attachment-0001.txt>

[ltt-dev] [RFC patch 29/41] LTTng menus

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-menus.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/f1d0d016/attachment-0001.txt>

[ltt-dev] [RFC patch 29/41] LTTng menus

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-menus.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/f1d0d016/attachment.asc>

[ltt-dev] [RFC patch 29/41] LTTng menus

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-menus.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/f1d0d016/attachment.txt>

[ltt-dev] [RFC patch 29/41] LTTng menus

[Randy Dunlap]

Mathieu Desnoyers wrote:

<<attachment :(>>


+menuconfig LTT
+	bool "Linux Trace Toolkit Next Generation (LTTng)"
+	depends on EXPERIMENTAL
+	select MARKERS
+	select TRACEPOINTS
+	default y

Not default 'y', please.

-- 
~Randy

[ltt-dev] [RFC patch 29/41] LTTng menus

[Mathieu Desnoyers]

* Randy Dunlap (randy.dunlap at oracle.com) wrote:
> Mathieu Desnoyers wrote:
> 
> <<attachment :(>>
> 
> 
> +menuconfig LTT
> +	bool "Linux Trace Toolkit Next Generation (LTTng)"
> +	depends on EXPERIMENTAL
> +	select MARKERS
> +	select TRACEPOINTS
> +	default y
> 
> Not default 'y', please.
> 

OK, so default n it is. But I plan to leave the main menu "sub-features" as
default y, given that people get the standard features when they choose
to enable the tracer. Hopefully this is ok ?

Mathieu

> -- 
> ~Randy

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[ltt-dev] [RFC patch 29/41] LTTng menus

[Randy Dunlap]

Mathieu Desnoyers wrote:
> * Randy Dunlap (randy.dunlap at oracle.com) wrote:
>> Mathieu Desnoyers wrote:
>>
>> <<attachment :(>>
>>
>>
>> +menuconfig LTT
>> +	bool "Linux Trace Toolkit Next Generation (LTTng)"
>> +	depends on EXPERIMENTAL
>> +	select MARKERS
>> +	select TRACEPOINTS
>> +	default y
>>
>> Not default 'y', please.
>>
> 
> OK, so default n it is. But I plan to leave the main menu "sub-features" as
> default y, given that people get the standard features when they choose
> to enable the tracer. Hopefully this is ok ?

Sure, as long as it just enables viewing the menu and not adding
code to a growing kernel.

-- 
~Randy

[ltt-dev] [RFC patch 29/41] LTTng menus

[Mathieu Desnoyers]

* Randy Dunlap (randy.dunlap at oracle.com) wrote:
> Mathieu Desnoyers wrote:
> > * Randy Dunlap (randy.dunlap at oracle.com) wrote:
> >> Mathieu Desnoyers wrote:
> >>
> >> <<attachment :(>>
> >>
> >>
> >> +menuconfig LTT
> >> +	bool "Linux Trace Toolkit Next Generation (LTTng)"
> >> +	depends on EXPERIMENTAL
> >> +	select MARKERS
> >> +	select TRACEPOINTS
> >> +	default y
> >>
> >> Not default 'y', please.
> >>
> > 
> > OK, so default n it is. But I plan to leave the main menu "sub-features" as
> > default y, given that people get the standard features when they choose
> > to enable the tracer. Hopefully this is ok ?
> 
> Sure, as long as it just enables viewing the menu and not adding
> code to a growing kernel.
> 

I want to be sure to understand your point. Would be following be OK ?

Menu [ ] Linux Trace Toolkit Next Generation (LTTng)  --->  (default n)

Within this menu, the following options enable various tracer modules,
some of which are typically needed, except in some very specific tracer
use :

< >   Linux Trace Toolkit Lock-Protected Data Relay (default n)
               (default y is planned to be used for the lockless data
               relay module, which is not posted as part of this patchset)
[ ]   Debug check for random access in ltt relay buffers (default n)
<*>   Linux Trace Toolkit Serializer (default y)
-*-   Linux Trace Toolkit Custom Serializer (default y)
-*-   Linux Trace Toolkit Trace Controller (default m)
<*>   Linux Trace Toolkit Tracer (default y)
[ ]   Align Linux Trace Toolkit Traces (default n, selected of
        !HAVE_EFFICIENT_UNALIGNED_ACCESS)
[ ]   Add event size field to LTT events for tracer debugging (default n)
<M>   Support logging events from userspace (default m)
[*]   Support trace extraction from crash dump (default y)
[*]   Linux Trace Toolkit Kprobes Support (default y)

I understand from your answer above that just enabling the "LTTng"
submenu should not activate any of these items, am I correct ?

Mathieu

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[ltt-dev] [RFC patch 29/41] LTTng menus

[Randy Dunlap]

Mathieu Desnoyers wrote:
> * Randy Dunlap (randy.dunlap at oracle.com) wrote:
>> Mathieu Desnoyers wrote:
>>> * Randy Dunlap (randy.dunlap at oracle.com) wrote:
>>>> Mathieu Desnoyers wrote:
>>>>
>>>> <<attachment :(>>
>>>>
>>>>
>>>> +menuconfig LTT
>>>> +	bool "Linux Trace Toolkit Next Generation (LTTng)"
>>>> +	depends on EXPERIMENTAL
>>>> +	select MARKERS
>>>> +	select TRACEPOINTS
>>>> +	default y
>>>>
>>>> Not default 'y', please.
>>>>
>>> OK, so default n it is. But I plan to leave the main menu "sub-features" as
>>> default y, given that people get the standard features when they choose
>>> to enable the tracer. Hopefully this is ok ?
>> Sure, as long as it just enables viewing the menu and not adding
>> code to a growing kernel.
>>

I see what you mean now.  Thanks for the details.

> 
> I want to be sure to understand your point. Would be following be OK ?
> 
> Menu [ ] Linux Trace Toolkit Next Generation (LTTng)  --->  (default n)
> 
> Within this menu, the following options enable various tracer modules,
> some of which are typically needed, except in some very specific tracer
> use :
> 
> < >   Linux Trace Toolkit Lock-Protected Data Relay (default n)
>                (default y is planned to be used for the lockless data
>                relay module, which is not posted as part of this patchset)
> [ ]   Debug check for random access in ltt relay buffers (default n)
> <*>   Linux Trace Toolkit Serializer (default y)
> -*-   Linux Trace Toolkit Custom Serializer (default y)
> -*-   Linux Trace Toolkit Trace Controller (default m)
> <*>   Linux Trace Toolkit Tracer (default y)
> [ ]   Align Linux Trace Toolkit Traces (default n, selected of
>         !HAVE_EFFICIENT_UNALIGNED_ACCESS)
> [ ]   Add event size field to LTT events for tracer debugging (default n)
> <M>   Support logging events from userspace (default m)
> [*]   Support trace extraction from crash dump (default y)
> [*]   Linux Trace Toolkit Kprobes Support (default y)
> 
> I understand from your answer above that just enabling the "LTTng"
> submenu should not activate any of these items, am I correct ?

Ideally one wouldn't add bloat to the kernel, but if someone enables
the top-level menu item, I'm OK with enabling others under it.

(not that we all consider the same things to be bloat ;)

-- 
~Randy

[ltt-dev] [RFC patch 30/41] LTTng build

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-build-instrumentation-menu.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/72e2b25d/attachment.asc>

[ltt-dev] [RFC patch 30/41] LTTng build

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-build-instrumentation-menu.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/72e2b25d/attachment-0001.asc>

[ltt-dev] [RFC patch 30/41] LTTng build

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-build-instrumentation-menu.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/72e2b25d/attachment.txt>

[ltt-dev] [RFC patch 31/41] LTTng userspace event v2

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-userspace-write-event.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/b367d2af/attachment.asc>

[ltt-dev] [RFC patch 31/41] LTTng userspace event v2

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-userspace-write-event.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/b367d2af/attachment-0001.txt>

[ltt-dev] [RFC patch 31/41] LTTng userspace event v2

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-userspace-write-event.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/b367d2af/attachment.txt>

[ltt-dev] [RFC patch 32/41] LTTng filter

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-filter.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/780348c3/attachment.asc>

[ltt-dev] [RFC patch 32/41] LTTng filter

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-filter.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/780348c3/attachment.txt>

[ltt-dev] [RFC patch 32/41] LTTng filter

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-filter.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/780348c3/attachment-0001.txt>

[ltt-dev] [RFC patch 33/41] LTTng dynamic tracing support with kprobes

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-kprobes-support.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/9ef4f8af/attachment.asc>

[ltt-dev] [RFC patch 33/41] LTTng dynamic tracing support with kprobes

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-kprobes-support.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/9ef4f8af/attachment-0001.txt>

[ltt-dev] [RFC patch 33/41] LTTng dynamic tracing support with kprobes

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: lttng-kprobes-support.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/9ef4f8af/attachment.txt>

[ltt-dev] [RFC patch 34/41] Marker header API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: marker.h-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/2fc4cd88/attachment.asc>

[ltt-dev] [RFC patch 34/41] Marker header API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: marker.h-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/2fc4cd88/attachment-0001.asc>

[ltt-dev] [RFC patch 34/41] Marker header API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: marker.h-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/2fc4cd88/attachment.txt>

[ltt-dev] [RFC patch 35/41] Marker API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: marker.c-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/2d7aac0e/attachment.txt>

[ltt-dev] [RFC patch 35/41] Marker API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: marker.c-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/2d7aac0e/attachment.asc>

[ltt-dev] [RFC patch 35/41] Marker API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: marker.c-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/2d7aac0e/attachment-0001.asc>

[ltt-dev] [RFC patch 36/41] kvm markers API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: kvm-markers-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/b946d541/attachment.asc>

[ltt-dev] [RFC patch 36/41] kvm markers API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: kvm-markers-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/b946d541/attachment-0001.txt>

[ltt-dev] [RFC patch 36/41] kvm markers API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: kvm-markers-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/b946d541/attachment.txt>

[ltt-dev] [RFC patch 37/41] Markers : multi-probes test

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-multi-probes-test.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/738ae989/attachment-0001.txt>

[ltt-dev] [RFC patch 37/41] Markers : multi-probes test

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-multi-probes-test.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/738ae989/attachment.txt>

[ltt-dev] [RFC patch 37/41] Markers : multi-probes test

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-multi-probes-test.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/738ae989/attachment.asc>

[ltt-dev] [RFC patch 38/41] Markers examples API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-examples-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/163563b6/attachment-0001.asc>

[ltt-dev] [RFC patch 38/41] Markers examples API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-examples-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/163563b6/attachment.txt>

[ltt-dev] [RFC patch 38/41] Markers examples API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: markers-examples-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/163563b6/attachment.asc>

[ltt-dev] [RFC patch 39/41] SPUFS markers API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: spufs-markers-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/253381ff/attachment-0001.asc>

[ltt-dev] [RFC patch 39/41] SPUFS markers API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: spufs-markers-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/253381ff/attachment.asc>

[ltt-dev] [RFC patch 39/41] SPUFS markers API update

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: spufs-markers-api-update.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/253381ff/attachment.txt>

[ltt-dev] [RFC patch 40/41] EXT4: instrumentation with tracepoints

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: ext4-move-from-markers-to-tracepoints.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/1be91ddb/attachment.txt>

[ltt-dev] [RFC patch 40/41] EXT4: instrumentation with tracepoints

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: ext4-move-from-markers-to-tracepoints.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/1be91ddb/attachment-0001.txt>

[ltt-dev] [RFC patch 40/41] EXT4: instrumentation with tracepoints

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: ext4-move-from-markers-to-tracepoints.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/1be91ddb/attachment.asc>

[ltt-dev] [RFC patch 41/41] JBD2: use tracepoints for instrumentation

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: jbd2-instrumentation-move-to-tracepoints.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/f9e03d32/attachment.txt>

[ltt-dev] [RFC patch 41/41] JBD2: use tracepoints for instrumentation

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: jbd2-instrumentation-move-to-tracepoints.patch
URL: <http://lists.casi.polymtl.ca/pipermail/ltt-dev/attachments/20090305/f9e03d32/attachment.asc>

[ltt-dev] [RFC patch 41/41] JBD2: use tracepoints for instrumentation

[Mathieu Desnoyers]

An embedded and charset-unspecified text was scrubbed...
Name: jbd2-instrumentation-move-to-tracepoints.patch
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20090305/f9e03d32/attachment-0001.txt>

[ltt-dev] [RFC patch 00/41] LTTng 0.105 core for Linux 2.6.27-rc9

[Ingo Molnar]

* Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca> wrote:

> Hi,
> 
> I spent the last 4-5 months working with the Fujitsu team at 
> implementing the tracer elements identified as goals at Kernel 
> Summit 2008 and at the following Plumber Conference. My idea 
> was to incremententally adapt the LTTng tracer, currently used 
> in the industry and well tested, to those requirements.
> 
> I spent the last days rearranging/folding/inspecting the LTTng 
> patchset to prepare it for an LKML post. The version 0.105 in 
> the LTTng git tree corresponds to the patchset I am posting 
> here. The said patchset will only include the core features of 
> LTTng, excluding the timestamping infrastructure (trace clock) 
> and excluding the instrumentation.

I'd like to merge the good bits into the tracing tree. Looking 
at the patches you submitted there's a lot of avoidable overlap 
with existing tracing features either present upstream already 
or queued up for v2.6.30 - and we need to work more on 
eliminating that overlap.

I dont think there's much fundamental disagreement just 
different implementations - so we should evaluate each of those 
details one by one, iteratively.

The first step would be to split the patches up into three 
logical buckets:

 - Unique features not present in the tracing infracture, in the 
   event tracer or other tracing plugins - those should be 
   structured as feature additions.

 - Features that you consider superior to existing tracing
   features of the kernel. For those, please iterate the
   existing code with your enhancements - instead of a parallel 
   implementation.

 - Items which offer nothing new and are not superior to 
   existing features, those should be dropped probably. This too 
   is a case by case thing.

Would you be interested in working with us on that? I know that 
both Steve and me would be very much interested in that. If you 
have time/interest to work on that then we can go through each 
patch one by one and categorize them and map out the way to go.

Let me give you a few examples of existing areas of overlap:

> The corresponding git tree contains also the trace clock 
> patches and the lttng instrumentation. The trace clock is 
> required to use the tracer, but it can be used without the 
> instrumentation : there is already a kprobes and userspace 
> event support included in this patchset.

The latest tracing tree includes kernel/tracing/trace_clock.c 
which offers three trace clock variants, with different 
performance/precision tradeoffs:

 trace_clock_local()   [ for pure CPU-local tracers with no idle 
                         events. This is the fastest but least 
                         coherent tracing clock. ]

 trace_clock()         [ intermediate, scalable clock with
                         usable but imprecise global coherency. ]

 trace_clock_global()  [ globally serialized, coherent clock. 
                         It is the slowest but most accurate variant. ]

Tracing plugins can pick their choice. (This is relatively new 
code but you get the idea.)

> This tracer exports binary data through buffers using 
> splice(). The resulting binary files can be parsed from 
> userspace because the format string metadata is exported in 
> the files. The event set can be enhanced by adding tracepoints 
> to the kernel code and by creating probe modules, which 
> connects callbacks to the tracepoints and contain the format 
> string metainformation. Those callbacks are responsible for 
> writing the data in the trace buffers. This separation between 
> the trace buffer format string and the tracepoints is done on 
> purpose so the core kernel instrumentation (tracepoints) is 
> not exported to userspace, which will make maintainance much 
> easier.

A tracepoint format specification mechanism plus working (and 
fast!) zero-copy splice() support of the ring-buffer exists in 
the latest tracing tree already - as you are probably aware of 
because you commented on those patches a few days ago.

There are 3 good ways to go from here regarding the trace 
buffering and splice code:

  1- we end up switching to the lttng version in essence
  2- we end up keeping the tracing tree version
  3- we end up somewhere inbetween

Which point in the above spectrum we will settle down on depends 
on the technical details.

Note, whichever path we choose a gradual, iterative workflow is 
still needed, so that we improve the existing upstream code with
lttng enhancements gradually.

This approach works for all your other patches as well. A 
direct, constructive comparison and active work on unifying them 
is required.

Thanks,

	Ingo

[ltt-dev] [RFC patch 00/41] LTTng 0.105 core for Linux 2.6.27-rc9

[Mathieu Desnoyers]

* Ingo Molnar (mingo at elte.hu) wrote:
> 
> * Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca> wrote:
> 
> > Hi,
> > 
> > I spent the last 4-5 months working with the Fujitsu team at 
> > implementing the tracer elements identified as goals at Kernel 
> > Summit 2008 and at the following Plumber Conference. My idea 
> > was to incremententally adapt the LTTng tracer, currently used 
> > in the industry and well tested, to those requirements.
> > 
> > I spent the last days rearranging/folding/inspecting the LTTng 
> > patchset to prepare it for an LKML post. The version 0.105 in 
> > the LTTng git tree corresponds to the patchset I am posting 
> > here. The said patchset will only include the core features of 
> > LTTng, excluding the timestamping infrastructure (trace clock) 
> > and excluding the instrumentation.
> 
> I'd like to merge the good bits into the tracing tree. Looking 
> at the patches you submitted there's a lot of avoidable overlap 
> with existing tracing features either present upstream already 
> or queued up for v2.6.30 - and we need to work more on 
> eliminating that overlap.
> 
> I dont think there's much fundamental disagreement just 
> different implementations - so we should evaluate each of those 
> details one by one, iteratively.
> 
> The first step would be to split the patches up into three 
> logical buckets:
> 
>  - Unique features not present in the tracing infracture, in the 
>    event tracer or other tracing plugins - those should be 
>    structured as feature additions.
> 
>  - Features that you consider superior to existing tracing
>    features of the kernel. For those, please iterate the
>    existing code with your enhancements - instead of a parallel 
>    implementation.
> 
>  - Items which offer nothing new and are not superior to 
>    existing features, those should be dropped probably. This too 
>    is a case by case thing.
> 
> Would you be interested in working with us on that? I know that 
> both Steve and me would be very much interested in that. If you 
> have time/interest to work on that then we can go through each 
> patch one by one and categorize them and map out the way to go.
> 

Hi Ingo,

Yes, I think an incremental inclusion is the way to go in the current
context. If we do it correctly, the resulting discussion will end up
putting the best features of both tracers in the resulting one. The only
problem is that I have less time at the moment (I should be writing my
Ph.D. thesis full time), but I think it's very important at this stage
to interact with the kernel community so everyone can benefit of the
work done in the past years.

I guess that identifying the good parts in each tracer will be a first
step towards integration. If you want, I could start by replying to my
own patchset post and do a ftrace-lttng comparison on each important
item.

I don't know how much time I'll be able to put into refactoring all this
though : I just spent 4 years developing LTTng and making sure every
nuts and bolts fits together fine. Hopefully we'll be able to keep the
modifications as lightweight and as iterative as possible.

> Let me give you a few examples of existing areas of overlap:
> 
> > The corresponding git tree contains also the trace clock 
> > patches and the lttng instrumentation. The trace clock is 
> > required to use the tracer, but it can be used without the 
> > instrumentation : there is already a kprobes and userspace 
> > event support included in this patchset.
> 
> The latest tracing tree includes kernel/tracing/trace_clock.c 
> which offers three trace clock variants, with different 
> performance/precision tradeoffs:
> 
>  trace_clock_local()   [ for pure CPU-local tracers with no idle 
>                          events. This is the fastest but least 
>                          coherent tracing clock. ]
> 
>  trace_clock()         [ intermediate, scalable clock with
>                          usable but imprecise global coherency. ]
> 
>  trace_clock_global()  [ globally serialized, coherent clock. 
>                          It is the slowest but most accurate variant. ]
> 
> Tracing plugins can pick their choice. (This is relatively new 
> code but you get the idea.)
> 

Hehe this reminds me of the trace clock thread I started a few months
ago on LKML. So you guys took over that work ? Nice :) Is it based on
the trace-clock patches I proposed back then ? Ah, no. Well I guess
we'll have to discuss this too. I agree on the
trace_clock_local/trace_clock/trace_clock_global interface, it looks
nice. The underlying implementation will have to be discussed though.


> > This tracer exports binary data through buffers using 
> > splice(). The resulting binary files can be parsed from 
> > userspace because the format string metadata is exported in 
> > the files. The event set can be enhanced by adding tracepoints 
> > to the kernel code and by creating probe modules, which 
> > connects callbacks to the tracepoints and contain the format 
> > string metainformation. Those callbacks are responsible for 
> > writing the data in the trace buffers. This separation between 
> > the trace buffer format string and the tracepoints is done on 
> > purpose so the core kernel instrumentation (tracepoints) is 
> > not exported to userspace, which will make maintainance much 
> > easier.
> 
> A tracepoint format specification mechanism plus working (and 
> fast!) zero-copy splice() support of the ring-buffer exists in 
> the latest tracing tree already - as you are probably aware of 
> because you commented on those patches a few days ago.

Yep, I know. :)

> 
> There are 3 good ways to go from here regarding the trace 
> buffering and splice code:
> 
>   1- we end up switching to the lttng version in essence
>   2- we end up keeping the tracing tree version
>   3- we end up somewhere inbetween
> 
> Which point in the above spectrum we will settle down on depends 
> on the technical details.
> 
> Note, whichever path we choose a gradual, iterative workflow is 
> still needed, so that we improve the existing upstream code with
> lttng enhancements gradually.
> 
> This approach works for all your other patches as well. A 
> direct, constructive comparison and active work on unifying them 
> is required.
> 

Yes, let's try to do it. Maybe it's better to start a new thread with
less CCs for this type of work ?

Mathieu

> Thanks,
> 
> 	Ingo

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[ltt-dev] [RFC patch 00/41] LTTng 0.105 core for Linux 2.6.27-rc9

[Ingo Molnar]

* Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca> wrote:

> > Let me give you a few examples of existing areas of overlap:
> > 
> > > The corresponding git tree contains also the trace clock 
> > > patches and the lttng instrumentation. The trace clock is 
> > > required to use the tracer, but it can be used without the 
> > > instrumentation : there is already a kprobes and userspace 
> > > event support included in this patchset.
> > 
> > The latest tracing tree includes 
> > kernel/tracing/trace_clock.c which offers three trace clock 
> > variants, with different performance/precision tradeoffs:
> > 
> >  trace_clock_local()   [ for pure CPU-local tracers with no idle 
> >                          events. This is the fastest but least 
> >                          coherent tracing clock. ]
> > 
> >  trace_clock()         [ intermediate, scalable clock with
> >                          usable but imprecise global coherency. ]
> > 
> >  trace_clock_global()  [ globally serialized, coherent clock. 
> >                          It is the slowest but most accurate variant. ]
> > 
> > Tracing plugins can pick their choice. (This is relatively new 
> > code but you get the idea.)
> > 
> 
> Hehe this reminds me of the trace clock thread I started a few 
> months ago on LKML. So you guys took over that work ? Nice :) 
> Is it based on the trace-clock patches I proposed back then ? 
> Ah, no. Well I guess we'll have to discuss this too. I agree 
> on the trace_clock_local/trace_clock/trace_clock_global 
> interface, it looks nice. The underlying implementation will 
> have to be discussed though.

Beware: i found the assembly trace_clock() stuff you did back 
then rather ugly ;-) I dont think there's any easy solutions 
here, so i went for this palette of clocks.

> > This approach works for all your other patches as well. A 
> > direct, constructive comparison and active work on unifying 
> > them is required.
> 
> Yes, let's try to do it. Maybe it's better to start a new 
> thread with less CCs for this type of work ?

Yeah. More finegrained steps are really needed.

The least controversial bits would be the many tracepoints you 
identified in LTTng as interesting. Mind sending them separately 
so that we can make some progress?

In the latest tracing code all tracepoints will show up 
automatically under /debug/tracing/events/ and can be used by 
user-space tools.

	Ingo

[ltt-dev] [RFC patch 00/41] LTTng 0.105 core for Linux 2.6.27-rc9

[Mathieu Desnoyers]

* Ingo Molnar (mingo at elte.hu) wrote:
> 
> * Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca> wrote:
> 
> > > Let me give you a few examples of existing areas of overlap:
> > > 
> > > > The corresponding git tree contains also the trace clock 
> > > > patches and the lttng instrumentation. The trace clock is 
> > > > required to use the tracer, but it can be used without the 
> > > > instrumentation : there is already a kprobes and userspace 
> > > > event support included in this patchset.
> > > 
> > > The latest tracing tree includes 
> > > kernel/tracing/trace_clock.c which offers three trace clock 
> > > variants, with different performance/precision tradeoffs:
> > > 
> > >  trace_clock_local()   [ for pure CPU-local tracers with no idle 
> > >                          events. This is the fastest but least 
> > >                          coherent tracing clock. ]
> > > 
> > >  trace_clock()         [ intermediate, scalable clock with
> > >                          usable but imprecise global coherency. ]
> > > 
> > >  trace_clock_global()  [ globally serialized, coherent clock. 
> > >                          It is the slowest but most accurate variant. ]
> > > 
> > > Tracing plugins can pick their choice. (This is relatively new 
> > > code but you get the idea.)
> > > 
> > 
> > Hehe this reminds me of the trace clock thread I started a few 
> > months ago on LKML. So you guys took over that work ? Nice :) 
> > Is it based on the trace-clock patches I proposed back then ? 
> > Ah, no. Well I guess we'll have to discuss this too. I agree 
> > on the trace_clock_local/trace_clock/trace_clock_global 
> > interface, it looks nice. The underlying implementation will 
> > have to be discussed though.
> 
> Beware: i found the assembly trace_clock() stuff you did back 
> then rather ugly ;-) I dont think there's any easy solutions 
> here, so i went for this palette of clocks.
> 

Hi Ingo,

I agree for the palette of clocks to fit all needs. I wonder what
exactly you found ugly in the approach I took with my trace_clock()
implementation ? Maybe you could refresh my memory, I do not recall
writing any part of it in assembly.. ? But this is a whole different
topic. We can discuss this later.


> > > This approach works for all your other patches as well. A 
> > > direct, constructive comparison and active work on unifying 
> > > them is required.
> > 
> > Yes, let's try to do it. Maybe it's better to start a new 
> > thread with less CCs for this type of work ?
> 
> Yeah. More finegrained steps are really needed.
> 
> The least controversial bits would be the many tracepoints you 
> identified in LTTng as interesting. Mind sending them separately 
> so that we can make some progress?
> 

OK, I'll work on it. Note however that I flipped my patchset around in
the past months : thinking that the tracer acceptance would be easier
than tracepoints. And now we are back at square one. Is it just me or I
have the funny feeling of acting like a dog running in circles after his
tail ? :)


> In the latest tracing code all tracepoints will show up 
> automatically under /debug/tracing/events/ and can be used by 
> user-space tools.
> 

Hrm, the thing is : I strongly disagree with showing tracepoints to
userspace and with the fact that you embed the data serialization
"pseudo-callbacks" into the tracepoint headers. Here is why. Peter
Zijlstra convinced me that putting format strings directly in tracepoint
headers was a bad idea. First off, you end up requiring all tracers
which connect on the tracepoints to choose your event format description
if they ever want to benefit from it. It's a "all included" formula :
either the tracers use them or they cannot output "standard" trace
information.

Second point : the tracepoints are meant to be tied to the kernel
source. Putting those event descriptions in global headers seems like
the people responsible for writing the kernel code surrounding the
tracepoints will end up being responsible for updating those tracepoint
event format descriptions. I think this is an unacceptable maintainance
burden for the whole community. Only tracer-specific modules should
refuse to build whenever it does not match the inner kernel structures
anymore.

Third point : it's plainly ugly. If we look at your tracepoint example :


/*
 * Tracepoint for task switches, performed by the scheduler:
 *
 * (NOTE: the 'rq' argument is not used by generic trace events,
 *        but used by the latency tracer plugin. )
 */
TRACE_EVENT(sched_switch,

        TP_PROTO(struct rq *rq, struct task_struct *prev,
                 struct task_struct *next),

        TP_ARGS(rq, prev, next),

        TP_STRUCT__entry(
                __array(        char,   prev_comm,      TASK_COMM_LEN   )
                __field(        pid_t,  prev_pid                        )
                __field(        int,    prev_prio                       )
                __array(        char,   next_comm,      TASK_COMM_LEN   )
                __field(        pid_t,  next_pid                        )
                __field(        int,    next_prio                       )
        ),

        TP_printk("task %s:%d [%d] ==> %s:%d [%d]",
                __entry->prev_comm, __entry->prev_pid, __entry->prev_prio,
                __entry->next_comm, __entry->next_pid, __entry->next_prio),

        TP_fast_assign(
                memcpy(__entry->next_comm, next->comm, TASK_COMM_LEN);
                __entry->prev_pid       = prev->pid;
                __entry->prev_prio      = prev->prio;
                memcpy(__entry->prev_comm, prev->comm, TASK_COMM_LEN);
                __entry->next_pid       = next->pid;
                __entry->next_prio      = next->prio;
        )
);


I notice that you actually embed the "function" that converts between
the format string into a header macro declaration. Why don't we write
this in plain C ?

in include/trace/sched.h :

DECLARE_TRACE(sched_switch,
        TPPROTO(struct rq *rq, struct task_struct *prev,
                struct task_struct *next),
                TPARGS(rq, prev, next));

in ltt/probes/kernel-trace.c :


void probe_sched_switch(struct rq *rq, struct task_struct *prev,
                struct task_struct *next);

DEFINE_MARKER_TP(kernel, sched_schedule, sched_switch, probe_sched_switch,
        "prev_pid %d next_pid %d prev_state #2d%ld");

notrace void probe_sched_switch(struct rq *rq, struct task_struct *prev,
                struct task_struct *next)
{
        struct marker *marker;
        struct serialize_int_int_short data;

        data.f1 = prev->pid;
        data.f2 = next->pid;
        data.f3 = prev->state;

        marker = &GET_MARKER(kernel, sched_schedule);
        ltt_specialized_trace(marker, marker->single.probe_private,
                &data, serialize_sizeof(data), sizeof(int));
}

This way, if the content of task_struct ever changes, only the tracer
module will break, not code touching a global header.

Mathieu

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[ltt-dev] [RFC patch 00/41] LTTng 0.105 core for Linux 2.6.27-rc9

[Ingo Molnar]

* Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca> wrote:

> * Ingo Molnar (mingo at elte.hu) wrote:
> > 
> > * Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca> wrote:
> > 
> > > > Let me give you a few examples of existing areas of overlap:
> > > > 
> > > > > The corresponding git tree contains also the trace clock 
> > > > > patches and the lttng instrumentation. The trace clock is 
> > > > > required to use the tracer, but it can be used without the 
> > > > > instrumentation : there is already a kprobes and userspace 
> > > > > event support included in this patchset.
> > > > 
> > > > The latest tracing tree includes 
> > > > kernel/tracing/trace_clock.c which offers three trace clock 
> > > > variants, with different performance/precision tradeoffs:
> > > > 
> > > >  trace_clock_local()   [ for pure CPU-local tracers with no idle 
> > > >                          events. This is the fastest but least 
> > > >                          coherent tracing clock. ]
> > > > 
> > > >  trace_clock()         [ intermediate, scalable clock with
> > > >                          usable but imprecise global coherency. ]
> > > > 
> > > >  trace_clock_global()  [ globally serialized, coherent clock. 
> > > >                          It is the slowest but most accurate variant. ]
> > > > 
> > > > Tracing plugins can pick their choice. (This is relatively new 
> > > > code but you get the idea.)
> > > > 
> > > 
> > > Hehe this reminds me of the trace clock thread I started a few 
> > > months ago on LKML. So you guys took over that work ? Nice :) 
> > > Is it based on the trace-clock patches I proposed back then ? 
> > > Ah, no. Well I guess we'll have to discuss this too. I agree 
> > > on the trace_clock_local/trace_clock/trace_clock_global 
> > > interface, it looks nice. The underlying implementation will 
> > > have to be discussed though.
> > 
> > Beware: i found the assembly trace_clock() stuff you did back 
> > then rather ugly ;-) I dont think there's any easy solutions 
> > here, so i went for this palette of clocks.
> > 
> 
> Hi Ingo,
> 
> I agree for the palette of clocks to fit all needs. I wonder 
> what exactly you found ugly in the approach I took with my 
> trace_clock() implementation ? Maybe you could refresh my 
> memory, I do not recall writing any part of it in assembly.. ? 
> But this is a whole different topic. We can discuss this 
> later.

hm, it was months ago. Ok, it must have been this one:

 http://lkml.org/lkml/2008/11/7/21
 http://lkml.org/lkml/2008/11/7/23

indeed no assembly but almost ;-) What i found rather ugly were 
the cnt32_to_63() complications.

	Ingo

[ltt-dev] [RFC patch 00/41] LTTng 0.105 core for Linux 2.6.27-rc9

[Mathieu Desnoyers]

* Ingo Molnar (mingo at elte.hu) wrote:
> 
> * Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca> wrote:
> 
> > * Ingo Molnar (mingo at elte.hu) wrote:
> > > 
> > > * Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca> wrote:
> > > 
> > > > > Let me give you a few examples of existing areas of overlap:
> > > > > 
> > > > > > The corresponding git tree contains also the trace clock 
> > > > > > patches and the lttng instrumentation. The trace clock is 
> > > > > > required to use the tracer, but it can be used without the 
> > > > > > instrumentation : there is already a kprobes and userspace 
> > > > > > event support included in this patchset.
> > > > > 
> > > > > The latest tracing tree includes 
> > > > > kernel/tracing/trace_clock.c which offers three trace clock 
> > > > > variants, with different performance/precision tradeoffs:
> > > > > 
> > > > >  trace_clock_local()   [ for pure CPU-local tracers with no idle 
> > > > >                          events. This is the fastest but least 
> > > > >                          coherent tracing clock. ]
> > > > > 
> > > > >  trace_clock()         [ intermediate, scalable clock with
> > > > >                          usable but imprecise global coherency. ]
> > > > > 
> > > > >  trace_clock_global()  [ globally serialized, coherent clock. 
> > > > >                          It is the slowest but most accurate variant. ]
> > > > > 
> > > > > Tracing plugins can pick their choice. (This is relatively new 
> > > > > code but you get the idea.)
> > > > > 
> > > > 
> > > > Hehe this reminds me of the trace clock thread I started a few 
> > > > months ago on LKML. So you guys took over that work ? Nice :) 
> > > > Is it based on the trace-clock patches I proposed back then ? 
> > > > Ah, no. Well I guess we'll have to discuss this too. I agree 
> > > > on the trace_clock_local/trace_clock/trace_clock_global 
> > > > interface, it looks nice. The underlying implementation will 
> > > > have to be discussed though.
> > > 
> > > Beware: i found the assembly trace_clock() stuff you did back 
> > > then rather ugly ;-) I dont think there's any easy solutions 
> > > here, so i went for this palette of clocks.
> > > 
> > 
> > Hi Ingo,
> > 
> > I agree for the palette of clocks to fit all needs. I wonder 
> > what exactly you found ugly in the approach I took with my 
> > trace_clock() implementation ? Maybe you could refresh my 
> > memory, I do not recall writing any part of it in assembly.. ? 
> > But this is a whole different topic. We can discuss this 
> > later.
> 
> hm, it was months ago. Ok, it must have been this one:
> 
>  http://lkml.org/lkml/2008/11/7/21
>  http://lkml.org/lkml/2008/11/7/23
> 
> indeed no assembly but almost ;-) What i found rather ugly were 
> the cnt32_to_63() complications.
> 

The fact that I put a patch touching cnt32_to_63 back then was just a
way to point out how the current cnt32_to_63 implementation is broken
for SMP and should stay in UP-only architecture-specific code (that was
an answer to Peter Zijlstra's reuse concerns). Once I got agreement that
tracers should not be expected to use cnt32_to_63, I dropped any patch
touching this piece of infrastructure and stayed with my
trace-clock-32-to-64.c implementation, which is SMP-safe, scalable and
basically extends atomically (through a rcu-like algorithm) a N bit
clock to a full 64-bits clock. This is very, very useful for lots of
architectures. Is it that code you find ugly ?

Mathieu

> 	Ingo
> 
> _______________________________________________
> ltt-dev mailing list
> ltt-dev at lists.casi.polymtl.ca
> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[ltt-dev] [RFC patch 00/41] LTTng 0.105 core for Linux 2.6.27-rc9

[Steven Rostedt]

Hi Mathieu,

Thanks for posting this. But it might be better to post in much smaller 
chunks. Lets work out the little things first. Posting 41 patches is a bit 
overwhelming. Took me a few hours to look at them all, and when I got to 
the end, I forgot what was at the beginning.

There's also minor changes to core kernel infrastructure code. seq_file, 
exporting functions, and such. These really need to be packaged 
separately, and sent to the proper maintainers. Having them in a patch 
bomb does not get the proper focus that they need.


On Thu, 5 Mar 2009, Mathieu Desnoyers wrote:

> Hi,
> 
> I spent the last 4-5 months working with the Fujitsu team at implementing the
> tracer elements identified as goals at Kernel Summit 2008 and at the following
> Plumber Conference. My idea was to incremententally adapt the LTTng tracer,
> currently used in the industry and well tested, to those requirements.

We really need to work together on this too. The biggest requirement that 
came out of that conference was to have a "single unified buffering 
system". And this was discussed quite heavily on LKML afterwards. All 
development was done incrementally and publicly.

> 
> I spent the last days rearranging/folding/inspecting the LTTng patchset
> to prepare it for an LKML post. The version 0.105 in the LTTng git tree
> corresponds to the patchset I am posting here. The said patchset will
> only include the core features of LTTng, excluding the timestamping
> infrastructure (trace clock) and excluding the instrumentation.
> 
> The corresponding git tree contains also the trace clock patches and the lttng
> instrumentation. The trace clock is required to use the tracer, but it can be
> used without the instrumentation : there is already a kprobes and userspace
> event support included in this patchset.
> 
> This tracer exports binary data through buffers using splice(). The resulting
> binary files can be parsed from userspace because the format string metadata is
> exported in the files. The event set can be enhanced by adding tracepoints to
> the kernel code and by creating probe modules, which connects callbacks to the
> tracepoints and contain the format string metainformation. Those callbacks are
> responsible for writing the data in the trace buffers. This separation between
> the trace buffer format string and the tracepoints is done on purpose so the
> core kernel instrumentation (tracepoints) is not exported to userspace, which
> will make maintainance much easier.

I've discussed this in my previous email. There is still a separation with
the TRACE_EVENT_FORMAT and the maintainers code. The format sting is 
"hint" only and may change without notice. LTTng could use it or ignore 
it, it is up to the tracer to actually export that string. ftrace chose to 
export it because it was a simple way to extract that information. My 
utility will need to do a bit more work when the events get more complex, 
but the way it is set up, we can do that on a case by case basis.


> 
> The tree including the trace clock patches is available at :
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/compudj/linux-2.6-lttng.git
> branch : 2.6.29-rc7-lttng-0.105
> 
> Project website : http://www.lttng.org/
> 
> Information about how to install and use the tracer is available at :
> 
> http://ltt.polymtl.ca/svn/trunk/lttv/LTTngManual.html
> 
> The size of the LTTng core patchset is 41 patches. The diffstat details
> as follow :


Again, this is overwhelming. This needs to be broken up into a small 
subsets that can be examined piece by piece.

> 
>  include/linux/ltt-core.h                                  |   35 
>  include/linux/ltt-relay.h                                 |  161 +
>  include/linux/ltt-tracer.h                                |   43 
>  include/linux/marker.h                                    |  121 
>  kernel/marker.c                                           |  353 ++
>  kernel/module.c                                           |   31 
>  linux-2.6-lttng/Documentation/markers.txt                 |   17 
>  linux-2.6-lttng/MAINTAINERS                               |    7 
>  linux-2.6-lttng/Makefile                                  |    2 
>  linux-2.6-lttng/arch/powerpc/kernel/traps.c               |    5 
>  linux-2.6-lttng/arch/powerpc/platforms/cell/spufs/spufs.h |    6 
>  linux-2.6-lttng/arch/sparc/Makefile                       |    2 
>  linux-2.6-lttng/arch/x86/kernel/dumpstack.c               |    5 
>  linux-2.6-lttng/arch/x86/mm/fault.c                       |    1 
>  linux-2.6-lttng/fs/ext4/fsync.c                           |    8 
>  linux-2.6-lttng/fs/ext4/ialloc.c                          |   17 
>  linux-2.6-lttng/fs/ext4/inode.c                           |   79 
>  linux-2.6-lttng/fs/ext4/mballoc.c                         |   71 
>  linux-2.6-lttng/fs/ext4/mballoc.h                         |    2 
>  linux-2.6-lttng/fs/ext4/super.c                           |    6 
>  linux-2.6-lttng/fs/jbd2/checkpoint.c                      |    7 
>  linux-2.6-lttng/fs/jbd2/commit.c                          |   12 
>  linux-2.6-lttng/fs/pipe.c                                 |    5 
>  linux-2.6-lttng/fs/select.c                               |   41 
>  linux-2.6-lttng/fs/seq_file.c                             |   45 
>  linux-2.6-lttng/fs/splice.c                               |    1 

There is a lot of code above that needs to be in their own patch series.
Maintainers do not have the time to pick through 41 patches to find out 
which patch might deal with their code.

Thanks,

-- Steve


>  linux-2.6-lttng/include/linux/immediate.h                 |   94 
>  linux-2.6-lttng/include/linux/kvm_host.h                  |   12 
>  linux-2.6-lttng/include/linux/ltt-channels.h              |   94 
>  linux-2.6-lttng/include/linux/ltt-core.h                  |   47 
>  linux-2.6-lttng/include/linux/ltt-relay.h                 |  186 +
>  linux-2.6-lttng/include/linux/ltt-tracer.h                |  731 ++++++
>  linux-2.6-lttng/include/linux/ltt-type-serializer.h       |  107 
>  linux-2.6-lttng/include/linux/marker.h                    |   16 
>  linux-2.6-lttng/include/linux/module.h                    |    6 
>  linux-2.6-lttng/include/linux/poll.h                      |    2 
>  linux-2.6-lttng/include/linux/seq_file.h                  |   20 
>  linux-2.6-lttng/include/trace/ext4.h                      |  129 +
>  linux-2.6-lttng/include/trace/jbd2.h                      |   19 
>  linux-2.6-lttng/init/Kconfig                              |    2 
>  linux-2.6-lttng/kernel/kallsyms.c                         |    1 
>  linux-2.6-lttng/kernel/marker.c                           |   12 
>  linux-2.6-lttng/kernel/module.c                           |   32 
>  linux-2.6-lttng/ltt/Kconfig                               |  130 +
>  linux-2.6-lttng/ltt/Makefile                              |   15 
>  linux-2.6-lttng/ltt/ltt-channels.c                        |  338 ++
>  linux-2.6-lttng/ltt/ltt-core.c                            |  101 
>  linux-2.6-lttng/ltt/ltt-filter.c                          |   66 
>  linux-2.6-lttng/ltt/ltt-kprobes.c                         |  479 +++
>  linux-2.6-lttng/ltt/ltt-marker-control.c                  |  265 ++
>  linux-2.6-lttng/ltt/ltt-relay-alloc.c                     |  715 +++++
>  linux-2.6-lttng/ltt/ltt-relay-locked.c                    | 1704 ++++++++++++++
>  linux-2.6-lttng/ltt/ltt-serialize.c                       |  685 +++++
>  linux-2.6-lttng/ltt/ltt-trace-control.c                   | 1061 ++++++++
>  linux-2.6-lttng/ltt/ltt-tracer.c                          | 1210 +++++++++
>  linux-2.6-lttng/ltt/ltt-type-serializer.c                 |   96 
>  linux-2.6-lttng/ltt/ltt-userspace-event.c                 |  131 +
>  linux-2.6-lttng/samples/markers/Makefile                  |    2 
>  linux-2.6-lttng/samples/markers/marker-example.c          |    4 
>  linux-2.6-lttng/samples/markers/probe-example.c           |   10 
>  linux-2.6-lttng/samples/markers/test-multi.c              |  120 
>  linux-2.6-lttng/virt/kvm/kvm_trace.c                      |   12 
>  ltt/Kconfig                                               |   24 
>  ltt/Makefile                                              |    2 
>  ltt/ltt-relay-alloc.c                                     |   80 
>  65 files changed, 9445 insertions(+), 398 deletions(-)
> 
> 
> Comments are welcome.
> 
> Mathieu
> 
> 
> -- 
> Mathieu Desnoyers
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> 

[ltt-dev] [RFC patch 00/41] LTTng 0.105 core for Linux 2.6.27-rc9

[Frederic Weisbecker]

On Fri, Mar 06, 2009 at 01:34:43PM -0500, Steven Rostedt wrote:
> 
> Hi Mathieu,
> 
> Thanks for posting this. But it might be better to post in much smaller 
> chunks. Lets work out the little things first. Posting 41 patches is a bit 
> overwhelming. Took me a few hours to look at them all, and when I got to 
> the end, I forgot what was at the beginning.
> 
> There's also minor changes to core kernel infrastructure code. seq_file, 
> exporting functions, and such. These really need to be packaged 
> separately, and sent to the proper maintainers. Having them in a patch 
> bomb does not get the proper focus that they need.
> 


Yes, I must confess I tried to review some of them but I have been discouraged
by the high volume and the multiple subjects that come with.

Iterating with smaller topics at a time, more focused subjects would help us to bring
the attention it deserves.

Frederic.

 
> On Thu, 5 Mar 2009, Mathieu Desnoyers wrote:
> 
> > Hi,
> > 
> > I spent the last 4-5 months working with the Fujitsu team at implementing the
> > tracer elements identified as goals at Kernel Summit 2008 and at the following
> > Plumber Conference. My idea was to incremententally adapt the LTTng tracer,
> > currently used in the industry and well tested, to those requirements.
> 
> We really need to work together on this too. The biggest requirement that 
> came out of that conference was to have a "single unified buffering 
> system". And this was discussed quite heavily on LKML afterwards. All 
> development was done incrementally and publicly.
> > 
> > I spent the last days rearranging/folding/inspecting the LTTng patchset
> > to prepare it for an LKML post. The version 0.105 in the LTTng git tree
> > corresponds to the patchset I am posting here. The said patchset will
> > only include the core features of LTTng, excluding the timestamping
> > infrastructure (trace clock) and excluding the instrumentation.
> > 
> > The corresponding git tree contains also the trace clock patches and the lttng
> > instrumentation. The trace clock is required to use the tracer, but it can be
> > used without the instrumentation : there is already a kprobes and userspace
> > event support included in this patchset.
> > 
> > This tracer exports binary data through buffers using splice(). The resulting
> > binary files can be parsed from userspace because the format string metadata is
> > exported in the files. The event set can be enhanced by adding tracepoints to
> > the kernel code and by creating probe modules, which connects callbacks to the
> > tracepoints and contain the format string metainformation. Those callbacks are
> > responsible for writing the data in the trace buffers. This separation between
> > the trace buffer format string and the tracepoints is done on purpose so the
> > core kernel instrumentation (tracepoints) is not exported to userspace, which
> > will make maintainance much easier.
> 
> I've discussed this in my previous email. There is still a separation with
> the TRACE_EVENT_FORMAT and the maintainers code. The format sting is 
> "hint" only and may change without notice. LTTng could use it or ignore 
> it, it is up to the tracer to actually export that string. ftrace chose to 
> export it because it was a simple way to extract that information. My 
> utility will need to do a bit more work when the events get more complex, 
> but the way it is set up, we can do that on a case by case basis.
> 
> 
> > 
> > The tree including the trace clock patches is available at :
> > 
> > git://git.kernel.org/pub/scm/linux/kernel/git/compudj/linux-2.6-lttng.git
> > branch : 2.6.29-rc7-lttng-0.105
> > 
> > Project website : http://www.lttng.org/
> > 
> > Information about how to install and use the tracer is available at :
> > 
> > http://ltt.polymtl.ca/svn/trunk/lttv/LTTngManual.html
> > 
> > The size of the LTTng core patchset is 41 patches. The diffstat details
> > as follow :
> 
> 
> Again, this is overwhelming. This needs to be broken up into a small 
> subsets that can be examined piece by piece.
> 
> > 
> >  include/linux/ltt-core.h                                  |   35 
> >  include/linux/ltt-relay.h                                 |  161 +
> >  include/linux/ltt-tracer.h                                |   43 
> >  include/linux/marker.h                                    |  121 
> >  kernel/marker.c                                           |  353 ++
> >  kernel/module.c                                           |   31 
> >  linux-2.6-lttng/Documentation/markers.txt                 |   17 
> >  linux-2.6-lttng/MAINTAINERS                               |    7 
> >  linux-2.6-lttng/Makefile                                  |    2 
> >  linux-2.6-lttng/arch/powerpc/kernel/traps.c               |    5 
> >  linux-2.6-lttng/arch/powerpc/platforms/cell/spufs/spufs.h |    6 
> >  linux-2.6-lttng/arch/sparc/Makefile                       |    2 
> >  linux-2.6-lttng/arch/x86/kernel/dumpstack.c               |    5 
> >  linux-2.6-lttng/arch/x86/mm/fault.c                       |    1 
> >  linux-2.6-lttng/fs/ext4/fsync.c                           |    8 
> >  linux-2.6-lttng/fs/ext4/ialloc.c                          |   17 
> >  linux-2.6-lttng/fs/ext4/inode.c                           |   79 
> >  linux-2.6-lttng/fs/ext4/mballoc.c                         |   71 
> >  linux-2.6-lttng/fs/ext4/mballoc.h                         |    2 
> >  linux-2.6-lttng/fs/ext4/super.c                           |    6 
> >  linux-2.6-lttng/fs/jbd2/checkpoint.c                      |    7 
> >  linux-2.6-lttng/fs/jbd2/commit.c                          |   12 
> >  linux-2.6-lttng/fs/pipe.c                                 |    5 
> >  linux-2.6-lttng/fs/select.c                               |   41 
> >  linux-2.6-lttng/fs/seq_file.c                             |   45 
> >  linux-2.6-lttng/fs/splice.c                               |    1 
> 
> There is a lot of code above that needs to be in their own patch series.
> Maintainers do not have the time to pick through 41 patches to find out 
> which patch might deal with their code.
> 
> Thanks,
> 
> -- Steve
> 
> 
> >  linux-2.6-lttng/include/linux/immediate.h                 |   94 
> >  linux-2.6-lttng/include/linux/kvm_host.h                  |   12 
> >  linux-2.6-lttng/include/linux/ltt-channels.h              |   94 
> >  linux-2.6-lttng/include/linux/ltt-core.h                  |   47 
> >  linux-2.6-lttng/include/linux/ltt-relay.h                 |  186 +
> >  linux-2.6-lttng/include/linux/ltt-tracer.h                |  731 ++++++
> >  linux-2.6-lttng/include/linux/ltt-type-serializer.h       |  107 
> >  linux-2.6-lttng/include/linux/marker.h                    |   16 
> >  linux-2.6-lttng/include/linux/module.h                    |    6 
> >  linux-2.6-lttng/include/linux/poll.h                      |    2 
> >  linux-2.6-lttng/include/linux/seq_file.h                  |   20 
> >  linux-2.6-lttng/include/trace/ext4.h                      |  129 +
> >  linux-2.6-lttng/include/trace/jbd2.h                      |   19 
> >  linux-2.6-lttng/init/Kconfig                              |    2 
> >  linux-2.6-lttng/kernel/kallsyms.c                         |    1 
> >  linux-2.6-lttng/kernel/marker.c                           |   12 
> >  linux-2.6-lttng/kernel/module.c                           |   32 
> >  linux-2.6-lttng/ltt/Kconfig                               |  130 +
> >  linux-2.6-lttng/ltt/Makefile                              |   15 
> >  linux-2.6-lttng/ltt/ltt-channels.c                        |  338 ++
> >  linux-2.6-lttng/ltt/ltt-core.c                            |  101 
> >  linux-2.6-lttng/ltt/ltt-filter.c                          |   66 
> >  linux-2.6-lttng/ltt/ltt-kprobes.c                         |  479 +++
> >  linux-2.6-lttng/ltt/ltt-marker-control.c                  |  265 ++
> >  linux-2.6-lttng/ltt/ltt-relay-alloc.c                     |  715 +++++
> >  linux-2.6-lttng/ltt/ltt-relay-locked.c                    | 1704 ++++++++++++++
> >  linux-2.6-lttng/ltt/ltt-serialize.c                       |  685 +++++
> >  linux-2.6-lttng/ltt/ltt-trace-control.c                   | 1061 ++++++++
> >  linux-2.6-lttng/ltt/ltt-tracer.c                          | 1210 +++++++++
> >  linux-2.6-lttng/ltt/ltt-type-serializer.c                 |   96 
> >  linux-2.6-lttng/ltt/ltt-userspace-event.c                 |  131 +
> >  linux-2.6-lttng/samples/markers/Makefile                  |    2 
> >  linux-2.6-lttng/samples/markers/marker-example.c          |    4 
> >  linux-2.6-lttng/samples/markers/probe-example.c           |   10 
> >  linux-2.6-lttng/samples/markers/test-multi.c              |  120 
> >  linux-2.6-lttng/virt/kvm/kvm_trace.c                      |   12 
> >  ltt/Kconfig                                               |   24 
> >  ltt/Makefile                                              |    2 
> >  ltt/ltt-relay-alloc.c                                     |   80 
> >  65 files changed, 9445 insertions(+), 398 deletions(-)
> > 
> > 
> > Comments are welcome.
> > 
> > Mathieu
> > 
> > 
> > -- 
> > Mathieu Desnoyers
> > OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> >