From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id Ax1LEgISz2eDKwsAWB0awg (envelope-from ) for ; Mon, 10 Mar 2025 12:23:30 -0400 Authentication-Results: simark.ca; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=I++7drdW; dkim-atps=neutral Received: by simark.ca (Postfix, from userid 112) id 38E4C1E105; Mon, 10 Mar 2025 12:23:30 -0400 (EDT) X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-5.4 required=5.0 tests=ARC_SIGNED,ARC_VALID,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=unavailable autolearn_force=no version=4.0.0 Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id 556391E08E for ; Mon, 10 Mar 2025 12:23:29 -0400 (EDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id EBB753858D20 for ; Mon, 10 Mar 2025 16:23:28 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org EBB753858D20 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1741623809; bh=oQ88Ve+zdb+6V0+sQ5fxjaLkMj1Ua/NH9AoxPWJ/gak=; h=Date:Subject:To:References:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=I++7drdWBoW6NhwX3dYYWUnfXhYTY+BAFKeXw+l+R1Rj8k+T+y25wBmQ1Rb0Bnfmj SlE0Cwi9tflARWqg4mnlvCdsKP+bEusgb33+OVdCgeK1QoYdrOcwGjRZWPUgSuq+0Q fTfxNDIlyi8QT3QfwomZufNRwzeChpfPIzTDXX2k= Received: from simark.ca (simark.ca [158.69.221.121]) by sourceware.org (Postfix) with ESMTPS id 8A2F43858D28 for ; Mon, 10 Mar 2025 16:22:44 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 8A2F43858D28 ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 8A2F43858D28 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1741623764; cv=none; b=DGdxDpyma9hUFpYfBem6vQf6Fp0Upq6yG31wupB/WPxA8saOyX8k7sZ3fV3wdgGMRhYGRPvH5mKaJ7xI0eej3wRyCc03KOHEMiYgqEA70qBg3bcd3dNBAQN+0jiJ9ZrpcXrC1a+y4U8viwc7qBbjaJAV0rdUCX7HGP5KdvzB9MY= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1741623764; c=relaxed/simple; bh=kqCvIfBXSqA2LDhc6AV8KmKZJ4rKIcJGP7a0lfucZzU=; h=DKIM-Signature:DKIM-Signature:Message-ID:Date:MIME-Version: Subject:To:From; b=uy1wOkprRXKuzXlrwS4e09YwzieK3vEhvpYQWZEY7Qfnx7Faw2kqAcLgiZ2jCzJFfCSEQmfevGOBrEKB7j1+GUoEvJ5j5QdU1/eABERzCDOX7xXjKFp76yvQeBahF6fkxRQxBtxvxND2ImB61ER/NaAtKbFeUGiE9gu6VgfNPAc= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 8A2F43858D28 Received: by simark.ca (Postfix, from userid 112) id 457B31E10A; Mon, 10 Mar 2025 12:22:44 -0400 (EDT) Received: from [192.168.113.187] (modemcable238.237-201-24.mc.videotron.ca [24.201.237.238]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPSA id 989DC1E08E; Mon, 10 Mar 2025 12:22:42 -0400 (EDT) Message-ID: Date: Mon, 10 Mar 2025 12:22:42 -0400 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Multi-threading support on DragonFlyBSD To: Sergey Zigachev , gdb@sourceware.org References: Content-Language: fr In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: gdb@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gdb mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Simon Marchi via Gdb Reply-To: Simon Marchi Errors-To: gdb-bounces~public-inbox=simark.ca@sourceware.org Sender: "Gdb" On 3/5/25 5:22 AM, Sergey Zigachev via Gdb wrote: > Hi, > I'm trying to enable debugging of multi-threaded programs on DragonFlyBSD using gdb and have several questions. > > I checked out gdb 15.2 sources, applied out-of-tree patches with native-dependent code implementation for amd64 DragonFlyBSD target. These patches have basic support for enumerating registers, accessing memory etc. All the OS-specific things. Single-thread debugging works well with these patches. However, multiprocess debugging, async and non-stop modes are not implemented. As is any notion of multiple threads in the inferior. My goal is to add missing bits and pieces both in userspace (gdb's nat-dep) and kernelspace (mostly ptrace syscall). > > I'm at the stage where I have the basics working. I can report events from different threads, suspend/resume single or all threads, inspect TLS variables etc. Schedule-locking mode also supported. > > My questions concern mostly step-over ("step") command issued to multiple threads, not at once of course. Although I see approx. the same behavior with the "next" command. > I wrote a little test C program that has main thread spawning two extra threads running two separate but mostly identical functions. Each function prints message to output, sleeps for several seconds, prints another message, and then returns. Main function waits for both threads to complete. > > First test run. > I put a breakpoint at the start of the first function, run the program and input "step" command until process exits. Essentially I'm debbuging as-if it was a single-thread program. Everything works as expected. By that i mean after each "step" command gdb steps on the next source line until the end of the function. It skips instructions that are part of a single source line, and skips functions that don't have debugging symbols. This is the baseline I'm trying to replicate with stepping over multiple threads. > > Second test run. > Same as the first run but this time I put a breakpoint at the start of both functions. This results in gdb stopping at different threads when I enter "step" command. > I observe the following issues: > - each source line gets hit several times, I think by the number of asm instructions it implements > - gdb stops at functions without debug symbols; sometimes it can step over and continue; sometimes it doesn't if it can't find the end of the function > Basically, gdb's acting as if it's running "stepi" command. > > With trial and error and some printf sprinkled around I came up with a theory why that could happen. Consider this part of the function: > >> 23 printf("Running thread thr=%d sleep=%d!\n", d->thr_num, d->sleep); >> 0x0000000000400b2d <+29>: mov 0xc(%rdi),%edx >> 0x0000000000400b30 <+32>: mov $0x400cff,%edi >> 0x0000000000400b35 <+37>: xor %eax,%eax >> 0x0000000000400b37 <+39>: call 0x4007f0 >> >> 24 sleep(d->sleep); >> 0x0000000000400b3c <+44>: mov 0xc(%rbx),%edi > > When gdb stops at line 23 and I enter "step" command, it calculates step range thread_info->control.step_range_{start,end}. In this case it will be [0x400b2d-0x400b3c). Then it uses this info to step through asm instructions 4 times transparently to the user. > The happy call chain looks like this (infrun.c): > - clear_proceed_status_thread *for each thread* > - prepare_one_step (infcmd.c) -- this one updates the range > - proceed > - 4x fetch_inferior_event > - stop at next line waiting for input > This works for single-thread case because gdb clears thread_info state and then loads range for the *current* thread after clearing. > If I return event for a different thread, then its info was cleared and haven't been loaded by gdb before that. Gdb can't recognize this signal and assumes it's a "random signal". It goes downhill from there, I think most consecutive signals become "random". > > I don't want to track down heisenbugs caused by my dirty hands touching gdb thread state and accidentally braking invariants. Hence this message, any advice is appreciated. > > My questions: > 1. Does my theory make sense? > 2. Is it a job of native-dependent code to save/restore thread state before reporting a signal? > 3. If yes, what else should I store, per-thread, per-process etc? > > Thanks, > Sergey Hi Sergey, What I don't understand here is: what other event do you report for the other threads? When you step one thread, I assume that your resume method gets called with inferior_ptid (the global variable) equal to the thread being stepped, and scope_ptid equal to (pid, 0, 0)? If so, the threads other than the one being stepped should be resumed freely. In your scenario, I don't see what signal these other threads would receive. Once your stepping thread is done stepping, it should generate a SIGTRAP, after which your target should send some signal (SIGSTOP?) to ask the other threads to stop and wait for these threads to report "stopped by SIGSTOP" (since your target operates in non-stop mode). But these SIGSTOPs should not be reported to the core, as it's an implementation detail of the target. That's my understanding of how things work on Linux, perhaps things are different on DragonflyBSD. I assume that you have enabled and started at the "set debug infrun" logs? This is how you generally debug those issues, look at what infrun asks your target to do, look at what your target answers, try to see if that makes sense. Finally, it's perhaps outside of the scope of your work, but "modern" targets use non-stop (related to "maint set target-non-stop", not the user-visible "set non-stop") and target-async. I'm more used to reason in terms of non-stop & target-async, and that code path of infrun is likely more tested. It's probably no small task, but switching to that would be more future-proof. Simon