From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id YdOCJjiEompXcTwAWB0awg (envelope-from ) for ; Thu, 10 Sep 2026 06:19:36 -0400 Authentication-Results: simark.ca; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20251104 header.b=rj/WDUSu; dkim-atps=neutral Received: by simark.ca (Postfix, from userid 112) id 97C2C1E09E; Thu, 10 Sep 2026 06:19:36 -0400 (EDT) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-25) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=ARC_SIGNED,ARC_VALID,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_SBL_CSS autolearn=ham autolearn_force=no version=4.0.1 Received: from vm01.sourceware.org (vm01.sourceware.org [IPv6:2620:52:6:3111::32]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id 7D6F51E091 for ; Thu, 10 Sep 2026 06:19:35 -0400 (EDT) Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A2E394BB1C11 for ; Thu, 10 Sep 2026 10:19:34 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A2E394BB1C11 Authentication-Results: sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20251104 header.b=rj/WDUSu Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by sourceware.org (Postfix) with ESMTPS id 5848B4BAE7D0 for ; Thu, 10 Sep 2026 10:19:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5848B4BAE7D0 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 5848B4BAE7D0 Authentication-Results: sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::102d ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1789035547; cv=none; b=Z4srClPGPyE29pkcn8DVBX8C4l7jSX2hp6I7s4yQeZNgF/fwFYJbtk7GFcvp0llPpN3JjaBrQhk+V/sB4uuCnV7asa88KeRXl7Swrw7bcxGfrik+uT4ZtYqcuCqQB+y7qVP/Dqbccm+XHiLAYD/ph/LRK6RG5pCUMlSg6eP3awk= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1789035547; c=relaxed/simple; bh=0kXIvZSfR7mdOMusZ0khCGZSoTjf5+qYTI3yPEj1sEQ=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=viNDj+L6mlq7WdIv/kK6wAH7kpqKmFOzdPnPB/Uy94YNVlhRuQEE9JL09Pl/a77i1a+iUUmdrx8PV7b5h4ZeN8x7P/w7LozwyPYjDS4L3VyOrk0EJChggbioWFmq0fKCVnuGi/zz4u1+FVUMnj1OyUDicqftZJIf20w+ja2z6zg= ARC-Authentication-Results: i=1; sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20251104 header.b=rj/WDUSu DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5848B4BAE7D0 Received: by mail-pj1-x102d.google.com with SMTP id 98e67ed59e1d1-3964dfb5b9aso8164479a91.1 for ; Thu, 10 Sep 2026 03:19:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789035546; x=1789640346; darn=sourceware.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=mxrkLIddBCDBemg28bjGuHYWbdmw5HGpA+6onhpMivI=; b=rj/WDUSuQpoYd0Znxlv2VOr7eFYdd0noDfNuE9MFU1wyTbOyVPbUhjeUOLgmVk5KwT YgNWfMsCFsdPec9BK6gjYJLSnCRdA8Vl5ydYVlCnOyBfDvgBbaRMDoP4PcJJZYbdtnXV GZjcD2Gm2i1dbHWvTArPCn2+WbhoqeKt+tKo5IF7n4UL2po63SrSI495mRqxktotbcO3 hVMU85DinyMxN9iqowX1ZYkmT1C8pnoLOf3gEX8twizXjXkrmXXYk/6Ni5WKaAuJk2H3 OX30ofXEL4ImNhRNtxPC9heG3PBqVxQGTy5bk+t1CfbRYhdSGg0CLEyHqonwtruaIliD HL6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789035546; x=1789640346; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=mxrkLIddBCDBemg28bjGuHYWbdmw5HGpA+6onhpMivI=; b=D/98BQT7hrdQUao2UMY10v591L3HigRw0pgr0+g2xZ5h0FZr7CSnVXQM4pxUvKtD96 L8wsgsrEuUWMGgzzwTfL+ENMjZSjOkmq/ppP/y9QZkUqXXnScQ1d12+ruB1ln7b+CN/7 KLvPR4Hm8rBblDO2etHrcM+n1t4jSAW0nvsLFRYwT6EvYWjBRhMoRVXUm7rilG5/LqS5 LzVw12LdlotAhQuWoGGCbFPso9V+5pUgaiFFnJv+maFxFlK8rNH8R+ms+nkYUq56wB+g voqMhsCB2w3PP/UwMuxlWwPmRlVccxc6jJTkkkiIT9kKIoC/Lsa2U09zUWvRj8PCS14S dyXA== X-Gm-Message-State: AFuF++ndCk518aeFd1ZJG90B45t7eiPq4jWNWauI2a127AJiHA/OPM3p Rlt2bVejUzM2Nj0hb7dZSPMXzC3fHu0XqDhGsz0QamMTdAnx8rUHWlMH X-Gm-Gg: AYBFou19NYi/pDE8RA8blt1v8I4/21Ubf8XuRO5LXgAbbmLx5u0vK/c8TkxtnT1jBvW pEBf0rakbfBcxlbGKbEXadU9G3IxuessDUBGgz7RtY0/Xut7OgldsF3WBAs87vz8QRL0FuusQY5 p3hmosIOJ+d3tD45cvdwWScWKItnS7XqoC+V318U8x58S2fOTkox0Biq5juw9ULOBQ9xAVr+qnL c4gKeYv+nF0fX9cerALt69cQNO1kKCCWGMMzlKs8JfdvJUgNB7kM98VecGWjCFFSCUfseUvv//b aLhTmV2J3YMJWFX3I38xo6o23Op9yxHmF6+qJWZQpmI7GThcXJ2smZsbqvOkpUuha9ekUpDgJXr B63XLX6gw/sTTIKVbwvBfv3SOVREQfhNquNzTIsbwCBInVwt7Vgvl0+c8Gmh3+JRneNjUj4r7kf Sr3xDp5loIpoBdDgISaj2bUchA0zLbg2IcrdU1fJ7guwIiap4NYidfC73LnvA+8xqbv7+GEGpwW a2wC0qu24ZCD8qwslhbgayrRJ7z X-Received: by 2002:a17:90a:d44b:b0:396:4cbf:45bc with SMTP id 98e67ed59e1d1-39b26203f11mr63962524a91.16.1789035546036; Thu, 10 Sep 2026 03:19:06 -0700 (PDT) Received: from localhost.localdomain ([122.171.22.218]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-33825669695sm27328919eec.3.2026.09.10.03.19.02 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 10 Sep 2026 03:19:05 -0700 (PDT) From: Aditya Vidyadhar Kamath To: ulrich.weigand@de.ibm.com, simon.marchi@polymtl.ca, tom@tromey.com Cc: gdb-patches@sourceware.org, Aditya.Kamath1@ibm.com, sangamesh.swamy@in.ibm.com, Aditya Vidyadhar Kamath Subject: [PATCH v2] Speed up next/step while debugging multithreaded programs on AIX. Date: Thu, 10 Sep 2026 15:46:44 +0530 Message-ID: <20260910101643.85955-2-akamath996@gmail.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gdb-patches-bounces~public-inbox=simark.ca@sourceware.org From: Aditya Vidyadhar Kamath During go compiler debugging on AIX my collegue and I noticed that the next command was pretty slow compared to Linux - around 7 seconds per step when program has atleast 3 or more threads running. A simple walk through a function took around a minute. In compiler codes this is slow. TO measure this properly I wrote a benchmark pasted below. The program spawns 20 worker threads , then stops inside a function with 10 simple that GDB steps over one by one. The GDB batch script breaks at that function and issues 10 next commands and then quits. Timing the whole run gives a fair picture of how much overhead each `next` carries. Both the .c file and batch file are pasted below. Before this patch, on AIX 7.3 ppc64 with 20 background threads: real 1m10.05s user 0m17.59s sys 0m20.43s That is roughly 7 seconds per `next` step. After this patch: real 0m20.37s user 0m5.94s sys 0m6.53s About 2 seconds per step. 3.5x faster for the same workload. Now why was it slow? Every time the inferior stops -- even for a single-step trap from next - the AIX thread layer called pd_update(), which did two very expensive things unconditionally: 1. pthdb_session_update () This function walks through all of the pthread library's internal data structures inside the inferior process, reading them out via pdc_read_data() callbacks. With 20 threads that is over 300 memory reads from the inferior on every single stop, even when absolutely nothing about the threads has changed. 2. sync_threadlists() This follows the session update with a full rebuild of GDB's thread list: iterate every thread via pthdb_pthread(), call pthdb_pthread_tid() for each one, allocate two sorted arrays, diff them against GDB's internal list, and reconcile the differences. Again, all of this on every stop, even when the thread population has not changed at all since the last stop. On top of that, get_signaled_thread() was calling getthrds() asking for one kernel thread descriptor at a time, which costs one syscall per thread in the process. None of this work is wrong but it is needed when threads actually come and go. But during a next or step sequence through a function, the thread list does not change. Paying for a full rescan on every stop is something not needed I think. So what I changed in this patch is: 1. Batch getthrds() calls get_signaled_thread () The original code passed 1 as the count argument to getthrds(), so it did one syscall per kernel thread to find the one that stopped. Changed to fetch 64 descriptors per call and scan the batch in a loop. This alone cuts the getthrds() cost from O(N) syscalls to O(N/64). 2. Track the thread count - aix_thread_variables, pd_activate Added a last_thread_count field to aix_thread_variables. It records how many threads libpthdebug reported the last time a full sync was done. Initialised to -1 in the constructor so the very first stop always does a full sync. Also reset to -1 in pd_activate() so that whenever a new debug session is opened the first stop is always fully synced regardless of the stop kind. 3. Skip the sync on clean step stops pd_update() now takes a bool step_stop parameter. When that is true and last_thread_count is not -1, it skips pthdb_session_update() and sync_threadlists() entirely. When a sync is needed (not a step stop, or first stop after activation), a new helper count_pthdb_threads() does a lightweight pass through pthdb_pthread() just to count threads. It only calls sync_threadlists() if that count differs from last_thread_count. This avoids the expensive pthdb_pthread_tid() calls and array allocations on stops where the thread count has not changed. 4. Detect step stops in wait() and pass the flag down wait() already has the stop kind and signal number before it calls pd_update(). A TARGET_WAITKIND_STOPPED with GDB_SIGNAL_TRAP is what the kernel reports for a completed single-step. wait() now computes step_stop from those two fields and passes it to pd_update(). int current_count = count_pthdb_threads (data); if (current_count != data->last_thread_count) This makes sure we never miss any genuinely coming new threads and are in sync. ================== Benchmark program (bench_next.c): /* Spawns NUM_THREADS worker threads that spin, then steps through 10 assignments in do_steps() so GDB can time each next. */ #include #include #include #include #ifndef NUM_THREADS #define NUM_THREADS 20 #endif static volatile int keep_running = 1; static void * worker (void *arg) { while (keep_running) sched_yield (); return NULL; } void do_steps (void) { volatile int a = 1; volatile int b = 2; volatile int c = a + b; volatile int d = c * 2; volatile int e = d - a; volatile int f = e + c; volatile int g = f / 2; volatile int h = g + 1; volatile int i2 = h * h; volatile int j = i2 - b; (void)j; } int main (void) { pthread_t threads[NUM_THREADS]; int i; for (i = 0; i < NUM_THREADS; i++) pthread_create (&threads[i], NULL, worker, NULL); do_steps (); keep_running = 0; for (i = 0; i < NUM_THREADS; i++) pthread_join (threads[i], NULL); return 0; } ==================== Build done with gcc -O0 -g -gdwarf -maix64 -DNUM_THREADS=20 -o bench_next_bin bench_next.c -lpthread ===================== GDB batch script (bench_next.gdb): set pagination off set confirm off break do_steps run next next next next next next next next next next quit ======================== Then run, time gdb -batch -x bench_next.gdb ./bench_next_bin --- gdb/aix-thread.c | 153 ++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 130 insertions(+), 23 deletions(-) diff --git a/gdb/aix-thread.c b/gdb/aix-thread.c index 5ac71d22237..3370736810a 100644 --- a/gdb/aix-thread.c +++ b/gdb/aix-thread.c @@ -173,6 +173,11 @@ static pthdb_callbacks_t pd_callbacks = { /* Aix variable structure. */ struct aix_thread_variables { + aix_thread_variables () + : pd_able (0), pd_active (0), pd_session (0), pd_brk_addr (0), + arch64 (0), last_thread_count (-1), last_resume_step (0) + {} + /* Whether the current application is debuggable by pthdb. */ int pd_able; @@ -192,6 +197,19 @@ struct aix_thread_variables /* Describes the number of thread exit events reported. */ std::unordered_set exited_threads; + + /* Last known libpthdebug thread count. Used to skip sync_threadlists() + when the thread population has not changed since the previous stop, + which is the common case during stepping. -1 means unknown/force sync. */ + int last_thread_count; + + /* Set to non-zero by resume() when the resume was a software + single-step, i.e. single-step breakpoints were inserted for the + current thread before the inferior was set running. Cleared to + zero for any other kind of resume. Used by wait() to distinguish + a SIGTRAP from a completed single-step from one caused by a + breakpoint, since both look identical at the signal level. */ + int last_resume_step; }; /* Key to our per-inferior data. */ @@ -733,6 +751,11 @@ state2str (pthdb_state_t state) } } +/* Number of thread descriptors to fetch per getthrds() call. The original + code fetched one at a time, which costs one syscall per thread. Fetching + in batches reduces that to one syscall per GETTHRDS_BATCH threads. */ +#define GETTHRDS_BATCH 64 + /* Search through the list of all kernel threads for the thread that has stopped on a SIGTRAP signal, and return its TID. Return 0 if none found. */ @@ -740,20 +763,18 @@ state2str (pthdb_state_t state) static pthdb_tid_t get_signaled_thread (int pid) { - struct thrdsinfo64 thrinf; + struct thrdsinfo64 thrinf[GETTHRDS_BATCH]; tid_t ktid = 0; + int count, i; - while (1) + while ((count = getthrds (pid, thrinf, sizeof (thrinf[0]), + &ktid, GETTHRDS_BATCH)) > 0) { - if (getthrds (pid, &thrinf, - sizeof (thrinf), &ktid, 1) != 1) - break; - /* We also need to keep in mind Trap and interrupt or any signal that needs to be handled in pd_update (). */ - - if (thrinf.ti_cursig) - return thrinf.ti_tid; + for (i = 0; i < count; i++) + if (thrinf[i].ti_cursig) + return thrinf[i].ti_tid; } /* Didn't find any thread stopped on a SIGTRAP signal. */ @@ -861,13 +882,43 @@ sync_threadlists (pid_t pid) } } +/* Count the number of live pthreads visible to libpthdebug. + Used before sync_threadlists() to check whether the thread population + has changed since the last stop. */ + +static int +count_pthdb_threads (struct aix_thread_variables *data) +{ + pthdb_pthread_t pdtid; + int cmd, n = 0; + + for (cmd = PTHDB_LIST_FIRST;; cmd = PTHDB_LIST_NEXT) + { + int status = pthdb_pthread (data->pd_session, &pdtid, cmd); + if (status != PTHDB_SUCCESS || pdtid == PTHDB_INVALID_PTHREAD) + break; + n++; + } + return n; +} + /* Synchronize libpthdebug's state with the inferior and with GDB, generate a composite process/thread for the current thread, - Return the ptid of the event thread if one can be found, else - return a pid-only ptid with PID. */ + return the ptid of the event thread if one can be found, else + return a pid-only ptid with PID. + + STEP_STOP should be set only when the last resume() was a software + single-step and the resulting stop is a SIGTRAP. In that case, if + the thread list was already synced at a previous stop, both + pthdb_session_update() and sync_threadlists() are skipped. Those + two calls dominate the cost of every stop in a multithreaded program, + so skipping them during stepping gives a large speedup. When + STEP_STOP is false - meaning the resume was a free continue, or a + temporary breakpoint was used, or the signal is not SIGTRAP - the + full sync always runs so that newly created threads are not missed. */ static ptid_t -pd_update (pid_t pid) +pd_update (pid_t pid, bool step_stop = false) { int status; ptid_t ptid; @@ -880,15 +931,36 @@ pd_update (pid_t pid) if (!data->pd_active) return ptid_t (pid); - status = pthdb_session_update (data->pd_session); - if (status != PTHDB_SUCCESS) - return ptid_t (pid); - - /* Attempt to sync_threadlists () only when debugging object files - and not core files since list of threads never change for core - files. */ - if (target_has_execution ()) - sync_threadlists (pid); + /* Skip the session update and thread list sync when stepping. + pthdb_session_update() reads through all pthread data structures in + the inferior on every call, and sync_threadlists() follows it with + O(N) kernel calls per thread. During a next/step sequence the thread + list does not change, so both calls can be omitted as long as we have + done at least one full sync since activation (last_thread_count >= 0). + Any other stop kind (breakpoints, real signals) always does a full + sync because threads may have been created or destroyed. */ + bool need_sync = !step_stop || (data->last_thread_count < 0); + + if (need_sync) + { + status = pthdb_session_update (data->pd_session); + if (status != PTHDB_SUCCESS) + return ptid_t (pid); + + /* Attempt to sync_threadlists() only when debugging object files + and not core files since the list of threads never changes for + core files. Even for live inferiors, skip the sync when the + thread count matches the previous stop. */ + if (target_has_execution ()) + { + int current_count = count_pthdb_threads (data); + if (current_count != data->last_thread_count) + { + sync_threadlists (pid); + data->last_thread_count = current_count; + } + } + } /* Define "current thread" as one that just received a trap signal. */ @@ -921,7 +993,12 @@ pd_activate (pid_t pid) PTHDB_FLAG_REGS, &pd_callbacks, &data->pd_session); if (status == PTHDB_SUCCESS) - data->pd_active = 1; + { + data->pd_active = 1; + /* Reset last_thread_count so that the very first stop after + activation always does a full sync_threadlists() pass. */ + data->last_thread_count = -1; + } } /* AIX implementation of update_thread_list. */ @@ -1056,6 +1133,21 @@ aix_thread_target::resume (ptid_t ptid, int step, enum gdb_signal sig) data = get_thread_data_helper_for_ptid (ptid); + /* Remember whether this resume is a software single-step so that + wait() can correctly classify the matching stop. On AIX the + architecture uses software single-stepping (rs6000_software_single_step), + so the step argument passed down to target_resume is always 0. + The only reliable indicator that GDB is single-stepping is whether + single-step breakpoints were inserted for the current thread just + before this resume. A breakpoint SIGTRAP from a user breakpoint + and a SIGTRAP from a completed software single-step are otherwise + indistinguishable at this layer. */ + if (data != nullptr) + { + struct thread_info *tp = inferior_thread (); + data->last_resume_step = thread_has_single_step_breakpoints_set (tp); + } + if (ptid.tid () == 0) { scoped_restore save_inferior_ptid = make_scoped_restore (&inferior_ptid); @@ -1125,7 +1217,22 @@ aix_thread_target::wait (ptid_t ptid, struct target_waitstatus *status, pd_activate (ptid.pid ()); } - return pd_update (ptid.pid ()); + /* Decide whether to skip the expensive pthdb_session_update() and + sync_threadlists() calls in pd_update(). We can only skip them + when we know for certain this stop came from a software single-step. + On AIX, single-stepping is implemented by inserting breakpoints at + the next instruction(s) and resuming with step=0, so the step + argument to target_resume is always 0 and cannot be used here. + Instead, resume() records whether single-step breakpoints were set + at the time of the resume in last_resume_step. If that flag is set + and the stop is a SIGTRAP, this is a completed single-step and the + thread list has not changed. If last_resume_step is 0, GDB either + continued freely or planted a different kind of breakpoint, so a + new thread could exist and the sync must run. */ + bool step_stop = (data->last_resume_step + && status->kind () == TARGET_WAITKIND_STOPPED + && status->sig () == GDB_SIGNAL_TRAP); + return pd_update (ptid.pid (), step_stop); } /* Supply AIX altivec registers, both 64 and 32 bit. */ -- 2.51.2