From mboxrd@z Thu Jan 1 00:00:00 1970 From: mathieu.desnoyers@efficios.com (Mathieu Desnoyers) Date: Tue, 23 Jun 2015 20:21:12 +0000 (UTC) Subject: [lttng-dev] [PATCH lttng-modules] Fix: Building the event list fails on fragmented memory In-Reply-To: <1503217107.1784.1435088815111.JavaMail.zimbra@efficios.com> References: <55755E0D.7010902@leisener.de> <1905517223.241.1433920494952.JavaMail.zimbra@efficios.com> <557838F1.9020808@leisener.de> <1586680029.1815.1434223948187.JavaMail.zimbra@efficios.com> <20150623134638.GA3944@hal> <1503217107.1784.1435088815111.JavaMail.zimbra@efficios.com> Message-ID: <1501535608.1867.1435090872005.JavaMail.zimbra@efficios.com> I think I found the culprit (testing now): int lttng_metadata_printf(struct lttng_session *session, const char *fmt, ...) tmp_cache_alloc_size = max_t(unsigned int, session->metadata_cache->cache_alloc + len, session->metadata_cache->cache_alloc << 1) should be: tmp_cache_alloc_size = max_t(unsigned int, session->metadata_cache->metadata_written + len, session->metadata_cache->cache_alloc << 1); I think we have a possible memory corruption in the upstream code here. More coming soon, Thanks, Mathieu ----- On Jun 23, 2015, at 3:46 PM, Mathieu Desnoyers mathieu.desnoyers at efficios.com wrote: > ----- On Jun 23, 2015, at 9:46 AM, Jan Glauber jan.glauber at gmail.com wrote: > >> On Sat, Jun 13, 2015 at 07:32:28PM +0000, Mathieu Desnoyers wrote: >>> ----- On Jun 10, 2015, at 9:17 AM, Martin Leisener martin at leisener.de wrote: >>> >>> > Hi Mathieu, >>> > >>> > sorry for sending the broken patch, I fell for some "smart" formating of my >>> > email client, I cannot utilize it myself the way I send it. >>> > Hence I now send the patch again as it was original intended. Sent to myself, I >>> > can apply it, so I hope it now works in general as well >>> > I examined as recommended diff -urN, it seems the output can be utilized by >>> > patch the same way as git diff. >>> > Hope it is now received the way I intended. >>> >>> Hi Martin, >>> >>> I tried you patch on a x86-64 machine (my laptop), and although taking a >>> single kernel trace worked fine, it appears to OOPS when I run >>> >>> (in lttng-tools) >>> cd tests >>> ./run.sh root_regression >> >> Hi Mathieu, >> >> can you please share the oops message so I can have a look? > > Sure, > > Here is the backtrace happening with the attached patch over lttng-modules > commit 8c6e7f13c778701dec2d6549f1e2ca98970907a0. I modified your patch > slightly. > > [ 4078.314978] BUG: unable to handle kernel paging request at ffffc900038d995e > [ 4078.315824] IP: [] __memcpy+0x12/0x20 > [ 4078.315824] PGD 236c92067 PUD 236c93067 PMD bac0c067 PTE 0 > [ 4078.315824] Oops: 0000 [#1] SMP > [ 4078.315824] Modules linked in: lttng_probe_workqueue(O) lttng_probe_vmscan(O) > lttng_probe_udp(O) lttng_probe_timer(O) lttng_probe_sunrpc(O) > lttng_probe_statedump(O) lttng_probe_sock(O) lttng_probe_skb(O) > lttng_probe_signal(O) lttng_probe_scsi(O) lttng_probe_sched(O) > lttng_probe_regmap(O) lttng_probe_rcu(O) lttng_probe_random(O) > lttng_probe_printk(O) lttng_probe_power(O) lttng_probe_net(O) > lttng_probe_napi(O) lttng_probe_module(O) lttng_probe_kmem(O) > lttng_probe_jbd2(O) lttng_probe_irq(O) lttng_probe_ext4(O) > lttng_probe_compaction(O) lttng_probe_block(O) lttng_types(O) > lttng_ring_buffer_metadata_mmap_client(O) > lttng_ring_buffer_client_mmap_overwrite(O) > lttng_ring_buffer_client_mmap_discard(O) lttng_ring_buffer_metadata_client(O) > lttng_ring_buffer_client_overwrite(O) lttng_ring_buffer_client_discard(O) > lttng_tracer(O) lttng_statedump(O) lttng_kprobes(O) lttng_lib_ring_buffer(O) > lttng_kretprobes(O) virtio_blk virtio_net virtio_pci virtio_ring virtio > [ 4078.315824] CPU: 5 PID: 4258 Comm: lttng-consumerd Tainted: G O > 4.1.0 #7 > [ 4078.315824] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs > 01/01/2011 > [ 4078.315824] task: ffff8802350c3660 ti: ffff8800bae84000 task.ti: > ffff8800bae84000 > [ 4078.315824] RIP: 0010:[] [] > __memcpy+0x12/0x20 > [ 4078.315824] RSP: 0018:ffff8800bae87da0 EFLAGS: 00010246 > [ 4078.315824] RAX: ffff880235439025 RBX: 0000000000000fd8 RCX: 00000000000001fb > [ 4078.315824] RDX: 0000000000000000 RSI: ffffc900038d995e RDI: ffff880235439025 > [ 4078.315824] RBP: ffff8800bae87db8 R08: ffff8800bacecc00 R09: 0000000000008000 > [ 4078.315824] R10: 0000000000000000 R11: 0000000000000246 R12: ffff8800bae87dc8 > [ 4078.315824] R13: ffff88023466e800 R14: 0000000000000fd8 R15: 0000000000000fd8 > [ 4078.315824] FS: 00007f5d3b1cc700(0000) GS:ffff8802372a0000(0000) > knlGS:0000000000000000 > [ 4078.315824] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b > [ 4078.315824] CR2: ffffc900038d995e CR3: 00000000bb1ed000 CR4: 00000000000006e0 > [ 4078.315824] Stack: > [ 4078.315824] ffffffffa01ac797 ffff8800bb5bd480 ffff8800bb5bd4d0 > ffff8800bae87e48 > [ 4078.315824] ffffffffa0073060 ffff88023466e800 0000000000000000 > 0000000000000fd8 > [ 4078.315824] ffffffff00000001 ffff8800bacecc00 0000000000000fd8 > 0000000000008025 > [ 4078.315824] Call Trace: > [ 4078.315824] [] ? lttng_event_write+0x87/0xb0 > [lttng_ring_buffer_metadata_client] > [ 4078.315824] [] lttng_metadata_output_channel+0xd0/0x120 > [lttng_tracer] > [ 4078.315824] [] lttng_metadata_ring_buffer_ioctl+0x79/0xd0 > [lttng_tracer] > [ 4078.315824] [] do_vfs_ioctl+0x2e0/0x4e0 > [ 4078.315824] [] ? file_has_perm+0x87/0xa0 > [ 4078.315824] [] SyS_ioctl+0x81/0xa0 > [ 4078.315824] [] ? syscall_trace_leave+0xd1/0xe0 > [ 4078.315824] [] tracesys_phase2+0x84/0x89 > [ 4078.315824] Code: 5b 5d c3 66 0f 1f 44 00 00 e8 6b fc ff ff eb e1 90 90 90 90 > 90 90 90 90 90 0f 1f 44 00 00 48 89 f8 48 89 d1 48 c1 e9 03 83 e2 07 48 a5 > 89 d1 f3 a4 c3 66 0f 1f 44 00 00 48 89 f8 48 89 d1 f3 > [ 4078.315824] RIP [] __memcpy+0x12/0x20 > [ 4078.315824] RSP > [ 4078.315824] CR2: ffffc900038d995e > [ 4078.315824] ---[ end trace a05b652829ceda48 ]--- > [ 4078.433848] LTTng: block device enumeration is not supported by kernel > > > -- > Mathieu Desnoyers > EfficiOS Inc. > http://www.efficios.com > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com