From mboxrd@z Thu Jan 1 00:00:00 1970 From: avik.sil@linaro.org (Avik Sil) Date: Tue, 13 Sep 2011 16:46:35 +0530 Subject: [ltt-dev] LTTng 2.0 on ARM In-Reply-To: References: <4E68559A.8080204@linaro.org> <4E6E4250.2000309@linaro.org> Message-ID: <4E6F3B93.5050603@linaro.org> On Tuesday 13 September 2011 12:35 AM, Mathieu Desnoyers wrote: > * Avik Sil (avik.sil at linaro.org) wrote: >> >>> Hi Avik, >>> >>> Can I get your kernel .config ? Also, adding a printk in lttng-modules >>> wrapper/vmalloc.h, just after vmalloc_sync_all_sym = (void *) >>> kallsyms_lookup_name("vmalloc_sync_all"); >>> >>> printing the vmalloc_sync_all_sym pointer value would be clearly >>> helpful. I would think ARM does not implement vmalloc_sync_all, so the >>> dummy mm/vmalloc.c vmalloc_sync_all weak symbol should be used as a >>> valid empty function. Please grep for vmalloc_sync_all under your >>> arch/arm to see if your particular omap flavor is implementing a >>> vmalloc_sync_all. >>> >> Hi Mathieu, >> >> Please find attached my kernel .config. After adding printk just after >> vmalloc_sync_all_sym = (void *)kallsyms_lookup_name("vmalloc_sync_all"); >> I got following output: >> >> [ 1139.173522] vmalloc_sync_all_sym: c00a1d14 >> [ 1139.180877] Internal error: Oops - undefined instruction: 0 [#1] >> PREEMPT SMP >> [ 1139.191284] Modules linked in: lttng_ftrace(+) >> [ 1139.198974] CPU: 1 Not tainted (3.0.0-1004-linaro-omap #6) >> [ 1139.208099] PC is at vmalloc_sync_all+0x0/0x8 >> [ 1139.215667] LR is at init_module+0x1c/0x28 [lttng_ftrace] >> [ 1139.224304] pc : [] lr : [] psr: 60000113 >> [ 1139.224304] sp : ebb11f50 ip : 00000000 fp : 00000000 >> [ 1139.242401] r10: 00000000 r9 : 00000000 r8 : c06a2f00 >> [ 1139.250793] r7 : bf8001f5 r6 : 00000000 r5 : 00182008 r4 : c00a1d14 >> [ 1139.260559] r3 : 271aed1f r2 : ebb11f44 r1 : bf8003a6 r0 : 00000034 >> [ 1139.270233] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM >> Segment user >> [ 1139.280670] Control: 50c5387d Table: abdb004a DAC: 00000015 >> [ 1139.289672] Process insmod (pid: 1368, stack limit = 0xebb102f8) >> [ 1139.298889] Stack: (0xebb11f50 to 0xebb12000) >> [ 1139.306335] 1f40: bf80044c >> c00085f7 00000000 00000001 >> [ 1139.320587] 1f60: bf80044c bf80044c 00182008 bf80044c 00182008 >> 00000000 000019de c000c364 >> [ 1139.334899] 1f80: ebb10000 c00622bb 00182018 000019de 00182008 >> 00000002 000019de beb099a0 >> [ 1139.349273] 1fa0: 00000080 c000c161 00000002 000019de 00182018 >> 000019de 00182008 402e6250 >> [ 1139.363861] 1fc0: 00000002 000019de beb099a0 00000080 00000003 >> 00004000 400b9000 00000000 >> [ 1139.378540] 1fe0: 00011f80 beb09728 00008c01 40296ae0 00000110 >> 00182018 a92aabaa aab0b0aa >> [ 1139.393310] [] (vmalloc_sync_all+0x0/0x8) from [] >> (init_module+0x1c/0x28 [lttng_ftrace]) >> [ 1139.410095] [] (init_module+0x1c/0x28 [lttng_ftrace]) from >> [] (do_one_initcall+0x6b/0xfc) >> [ 1139.427032] [] (do_one_initcall+0x6b/0xfc) from >> [] (sys_init_module+0x4b/0x11c) >> [ 1139.443176] [] (sys_init_module+0x4b/0x11c) from >> [] (ret_fast_syscall+0x1/0x50) >> [ 1139.459472] Code: e8bdb007 bf008ff0 c0672290 c0be2e54 (f85db500) >> [ 1139.475524] ---[ end trace 7f9f02516cb400bf ]--- >> Segmentation fault >> >> Also I found that vmalloc_sync_all is not implemented under arch/arm and >> it is using the mm/vmalloc.c vmalloc_sync_all weak symbol whose >> disassembly gives: >> >> bx lr > > Hrm, weird. It's possibly some linker issue around the weak symbol, or > the compiler does not generate the stub correctly. Can you look at the > full dissassembly of the weak vmalloc_sync_all ? I would like to know > the surrounding of 0xc00a1d14. > > e.g. > objdump -S --start-address=0xc00a1d14 vmlinux |head -n 30 I got following output: # objdump -S --start-address=0xc00a1d14 /usr/lib/debug/boot/vmlinux-3.0.0-1004-linaro-omap | head -n 30 /usr/lib/debug/boot/vmlinux-3.0.0-1004-linaro-omap: file format elf32-littlearm Disassembly of section .text: c00a1d14 : c00a1d14: b500 push {lr} c00a1d16: f76a fa9d bl c000c254 <__gnu_mcount_nc> c00a1d1a: 4770 bx lr c00a1d1c : c00a1d1c: e92d 41f0 stmdb sp!, {r4, r5, r6, r7, r8, lr} c00a1d20: b500 push {lr} c00a1d22: f76a fa97 bl c000c254 <__gnu_mcount_nc> c00a1d26: 2500 movs r5, #0 c00a1d28: 4604 mov r4, r0 c00a1d2a: 460f mov r7, r1 c00a1d2c: 4606 mov r6, r0 c00a1d2e: e004 b.n c00a1d3a c00a1d30: f856 0b04 ldr.w r0, [r6], #4 c00a1d34: 3501 adds r5, #1 c00a1d36: f7ff fc9b bl c00a1670 c00a1d3a: 42bd cmp r5, r7 c00a1d3c: dbf8 blt.n c00a1d30 c00a1d3e: 4620 mov r0, r4 c00a1d40: e8bd 41f0 ldmia.w sp!, {r4, r5, r6, r7, r8, lr} c00a1d44: f004 b888 b.w c00a5e58 c00a1d48 : > > We have to keep in mind that this could also be a ftrace function tracer > bug, in which case the surrounding of 0xc00a1d14 from the vmlinux image > will not match the instructions in memory. Can you dump the hex content > around 0xc00a1d14 just before the vmalloc_sync_all gets called and > compare with the disassembly ? The hex content dump shows: [ 150.810119] c00a1d14: b500 f85d eb04 4770 So it seems your guess is right, the content of vmlinux image surrounding 0xc00a1d14 is not matching the instruction in memory. By grepping objdump I found that instruction 'f85d eb04' translates to "ldr.w lr, [sp], #4". > > Also, another test: what happens if you put a "return;" at the beginning > of the wrapper/vmalloc.h vmalloc_sync_all wrapper ? > By putting "return;" the vmalloc_sync_all wrapper behaves correctly without any oops. Regards, Avik > Thanks, > > Mathieu >