From mboxrd@z Thu Jan 1 00:00:00 1970 From: xufeng.zhang@windriver.com (xufeng zhang) Date: Wed, 10 Nov 2010 15:19:23 +0800 Subject: [ltt-dev] "Kernel/LTTV event size differs for event udpv4_rcv_extended: kernel 32, LTTV 28" error when view trace In-Reply-To: <20101109115743.GA19744@Krystal> References: <4CD3A32D.6030706@windriver.com> <4CD75636.1020305@polymtl.ca> <4CD788AE.60301@windriver.com> <20101108095545.GA17190@Krystal> <4CD8F840.2080704@windriver.com> <20101109114500.GA18027@Krystal> <20101109115743.GA19744@Krystal> Message-ID: <4CDA477B.7010508@windriver.com> On 11/09/2010 07:57 PM, Mathieu Desnoyers wrote: > * Mathieu Desnoyers (compudj at krystal.dyndns.org) wrote: > >> * xufeng zhang (xufeng.zhang at windriver.com) wrote: >> >>> On 11/08/2010 05:55 PM, Mathieu Desnoyers wrote: >>> >>>> * xufeng zhang (xufeng.zhang at windriver.com) wrote: >>>> >>>> >>>>> On 11/08/2010 09:45 AM, Benjamin Poirier wrote: >>>>> >>>>> >>>>>> On 05/11/10 02:24 AM, xufeng zhang wrote: >>>>>> >>>>>> >>>>>> >>>>>>> Hi, >>>>>>> I'm using ltt-control-0.86 and lttv-0.12.31 on ARM to trace the kernel, >>>>>>> however, >>>>>>> >>>>>>> >>>>>>> >>>>>> Hello, >>>>>> >>>>>> Which version of the kernel tracer patches are you using? >>>>>> >>>>>> >>>>>> >>>>> I have query the developers who merged LTTng into kernel, they said >>>>> there is no incompatibility >>>>> between LTTV and LTTng, so does there any other reasons which lead to >>>>> this error? >>>>> >>>>> Thanks, >>>>> Xufeng Zhang >>>>> >>>>> >>>>>> -Ben >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> I cannot view the trace after I have successfully generated, here is the >>>>>>> error >>>>>>> when I run 'lttv-gui -t trace1' command: >>>>>>> ** Message: statistics viewer : background computation data ready. >>>>>>> >>>>>>> ** (lttv.real:6559): WARNING **: Cannot find pin_in in schedchange 671 >>>>>>> >>>>>>> ** ERROR **: Kernel/LTTV event size differs for event >>>>>>> udpv4_rcv_extended: kernel 32, LTTV 28 >>>>>>> >>>>>>> >>>> Please compare the /ltt/probes/net-extended-trace.c file from your >>>> distro kernel tree to the: >>>> >>>> lttng-modules package, file: /probes/net-extended-trace.c >>>> >>>> Also compare the /ltt/ltt-type-serializer.h (or was it >>>> include/linux/ltt-type-serializer.h ?) from your distro kernel with the >>>> lttng-modules package: >>>> >>>> /ltt-type-serializer.h >>>> >>>> I remember having fixed this problem in the past months. Look closely at >>>> the structure layout for "struct serialize_l214421224411111", "struct >>>> serialize_l4412228" and "struct serialize_l4421224411111" vs the types >>>> passed to the DEFINE_MARKER_TP() in net-extended-trace.c. >>>> >>>> >>> I have compare my kernel tree to lttng-modules-0.19.2, there is no >>> different >>> in ltt-type-serializer.h file, and below is the diff info for >>> net-extended-trace.c: >>> >>> --- net-extended-trace.c.kernel 2010-11-09 15:09:24.000000000 +0800 >>> +++ net-extended-trace.c 2010-11-09 15:12:47.000000000 +0800 >>> @@ -20,14 +20,15 @@ >>> >>> #include >>> #include >>> -#include >>> #include >>> #include >>> #include >>> #include >>> #include >>> >>> -void probe_net_dev_xmit_extended(struct sk_buff *skb); >>> +#include "../ltt-type-serializer.h" >>> + >>> +void probe_net_dev_xmit_extended(void *_data, struct sk_buff *skb); >>> >>> DEFINE_MARKER_TP(net, dev_xmit_extended, net_dev_xmit, >>> probe_net_dev_xmit_extended, "skb 0x%lX network_protocol #n2u%hu " >>> @@ -35,7 +36,7 @@ >>> "tot_len #n2u%hu ihl #1u%u source #n2u%hu dest #n2u%hu seq >>> #n4u%lu " >>> "ack_seq #n4u%lu doff #1u%u ack #1u%u rst #1u%u syn #1u%u fin >>> #1u%u"); >>> >>> -notrace void probe_net_dev_xmit_extended(struct sk_buff *skb) >>> +notrace void probe_net_dev_xmit_extended(void *_data, struct sk_buff *skb) >>> { >>> struct marker *marker; >>> struct serialize_l214421224411111 data; >>> @@ -70,14 +71,14 @@ >>> &data, serialize_sizeof(data), sizeof(long)); >>> } >>> >>> -void probe_tcpv4_rcv_extended(struct sk_buff *skb); >>> +void probe_tcpv4_rcv_extended(void *_data, struct sk_buff *skb); >>> >>> DEFINE_MARKER_TP(net, tcpv4_rcv_extended, net_tcpv4_rcv, >>> probe_tcpv4_rcv_extended, "skb 0x%lX saddr #n4u%lu daddr #n4u%lu " >>> "tot_len #n2u%hu ihl #1u%u source #n2u%hu dest #n2u%hu seq >>> #n4u%lu " >>> "ack_seq #n4u%lu doff #1u%u ack #1u%u rst #1u%u syn #1u%u fin >>> #1u%u"); >>> >>> -notrace void probe_tcpv4_rcv_extended(struct sk_buff *skb) >>> +notrace void probe_tcpv4_rcv_extended(void *_data, struct sk_buff *skb) >>> { >>> struct marker *marker; >>> struct serialize_l4421224411111 data; >>> @@ -104,14 +105,14 @@ >>> &data, serialize_sizeof(data), sizeof(long)); >>> } >>> >>> -void probe_udpv4_rcv_extended(struct sk_buff *skb); >>> +void probe_udpv4_rcv_extended(void *_data, struct sk_buff *skb); >>> >>> DEFINE_MARKER_TP(net, udpv4_rcv_extended, net_udpv4_rcv, >>> probe_udpv4_rcv_extended, "skb 0x%lX saddr #n4u%lu daddr #n4u%lu " >>> "unicast #1u%u ulen #n2u%hu source #n2u%hu dest #n2u%hu " >>> "data_start #8u%lx"); >>> >>> -notrace void probe_udpv4_rcv_extended(struct sk_buff *skb) >>> +notrace void probe_udpv4_rcv_extended(void *_data, struct sk_buff *skb) >>> { >>> struct marker *marker; >>> struct serialize_l4412228 data; >>> >>> >>> I don't think the difference in net-extended-trace.c will make any impact. >>> This problem has troubled me for weeks. >>> >> Can you try to reproduce with the latest LTTng version ? It's very hard >> for us to help you if we don't know the LTTng version you are using, >> especially for a bug I am aware to have fixed recently. >> > Ah, I remember it now. It had something to do with alignment of 64-bit > fields on 32-bit machines. Older LTTng versions did align these fields > on the next 64-bit multiple. New LTTng versions do the same as gcc and > align these on the next 32-bit boundary. > > Check the diff of your include/linux/ltt-core.h ltt_align() function > between a recent lttng and yours. > Could you provide the content of this patch, I think my problem is almost the same. Thank you very much. Thanks, Xufeng Zhang > Thanks, > > Mathieu > > >> Also, you might simply try to unload the net-extended-trace module (or >> disable its markers) to see if it helps getting a parseable trace. >> >> Thanks, >> >> Mathieu >> >> >>> Thanks, >>> Xufeng Zhang >>> >>>> Thanks, >>>> >>>> Mathieu >>>> >>>> >>>> >>>> >>>>>>> aborting... >>>>>>> /usr/local/bin/lttv-gui: line 10: 6559 Aborted >>>>>>> $LTTV_CMD.real -m lttvwindow -m guievents -m guifilter -m guicontrolflow >>>>>>> -m resourceview -m guistat >>>>>>> >>>>>>> >>>>>>> And I found another people met the same problem on MIPS32, a patch have >>>>>>> also been provided for MIPS32. >>>>>>> Here is the similar code in ARM trace-clock.c: >>>>>>> linux/arch/arm/mach-omap2/trace-clock.c line 537 >>>>>>> void get_trace_clock(void) >>>>>>> { >>>>>>> spin_lock(&trace_clock_lock); >>>>>>> if (trace_clock_refcount++) >>>>>>> goto end; >>>>>>> _start_trace_clock(); >>>>>>> end: >>>>>>> spin_unlock(&trace_clock_lock); >>>>>>> } >>>>>>> EXPORT_SYMBOL_GPL(get_trace_clock); >>>>>>> >>>>>>> void put_trace_clock(void) >>>>>>> { >>>>>>> spin_lock(&trace_clock_lock); >>>>>>> WARN_ON(trace_clock_refcount<= 0); >>>>>>> if (trace_clock_refcount != 1) >>>>>>> goto end; >>>>>>> _stop_trace_clock(); >>>>>>> end: >>>>>>> trace_clock_refcount--; >>>>>>> spin_unlock(&trace_clock_lock); >>>>>>> } >>>>>>> EXPORT_SYMBOL_GPL(put_trace_clock); >>>>>>> >>>>>>> Could anybody figure out what's wrong with the kernel code? Any >>>>>>> suggestion would be appreciated. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Xufeng Zhang >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> ltt-dev mailing list >>>>>>> ltt-dev at lists.casi.polymtl.ca >>>>>>> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> ltt-dev mailing list >>>>> ltt-dev at lists.casi.polymtl.ca >>>>> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev >>>>> >>>>> >>>>> >>>> >>>> >>> >> -- >> Mathieu Desnoyers >> Operating System Efficiency R&D Consultant >> EfficiOS Inc. >> http://www.efficios.com >> >> _______________________________________________ >> ltt-dev mailing list >> ltt-dev at lists.casi.polymtl.ca >> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev >> >> >