[RFC PATCH 0/1] gdb/python: add drg_why_sleeping command

[RFC PATCH 0/1] gdb/python: add drg_why_sleeping command

[Alexandra Hájková]

When using GDB to debug a multithreaded program with a thread in
interruptible sleep, we can call `info threads` command in GDB,
which shows the userspace frame each thread is stopped at.However, 
GDB has no way to show why a thread is sleeping on the kernel
side—which kernel function it is blocked in, or the full
kernel call path that led there. Drgn, on the other hand, can
provide this information.  Drgn does not use ptrace — it reads
/proc/kcore directly which means drgn can run alongside a GDB
session without any conflict, since GDB already holds ptrace on
the inferior.

I wrote a small GDB Python command, drgn_why_sleeping, that demonstrates 
this cooperation. When invoked on a selected thread (in GDB non-stop mode, so the thread
remains in its sleep state), it retrieves the full kernel stack via drgn

When debugging a multithreaded program with GDB, `info threads` shows
the userspace frame each thread is stopped at, but GDB has no way to show
why a thread is sleeping on the kernel side -- which kernel function it is
blocked in, or the call path that led there.

To run this script drgn and kernel debug has to be installed. I'm using
to drgn's sudo helper, which opens /proc/kcore in a transient privileged
child and passes the fd back over a unix socket.  GDB itself stays unprivileged.
Running GDB itsel is another possibility.

In order to make things easier, I included sleeping_test.c program as
a part of this patch since it's RFC.

To demostrate this command, sleep_test has to be compiled and run 
separetely. Then we need to attach GDB in non-stop mode for example

./gdb/gdb --data-directory=gdb/data-directory -ex "set non-stop on" -p 1979281

then we need to resume the thread before inspecting it:
    (gdb) info threads
    (gdb) thread N
    (gdb) drgn_why_sleeping

If the thread stays GDB-stopped, the kernel stack just shows ptrace_stop
and not the real reason it was sleeping.  Resuming it (continue &) lets it
return to its kernel sleep.

      Id   Target Id                          Frame
      2    ... "sleep_test" (LWP 1979284)      0x... in read () from libc.so.6
      ...
    (gdb) thread 2

Running before resuming the thread which means GDB still has it ptrace-stopped, 
so the kernel stack only shows the ptrace stop, not the real sleep:

    (gdb) drgn_why_sleeping
    Task state: t
    context_switch
    __schedule
    schedule
    ptrace_stop
    ptrace_signal
    get_signal
    ...
    do_syscall_64
    entry_SYSCALL_64

After resuming just this thread (non-stop) so it re-enters its sleep,
and running the drgn_why_sleep again we're seeing the real reason it
is blocked:

    (gdb) continue &
    (gdb) drgn_why_sleeping
    Task state: S
    context_switch
    __schedule
    schedule
    anon_pipe_read
    new_sync_read
    vfs_read
    ksys_read
    do_syscall_64
    entry_SYSCALL_64

Given a frame name, the command prints that frame's locals instead.  Here
the file operations resolve to pipeanon_fops, confirming from the kernel
data itself that the thread is blocked on a pipe:

    (gdb) drgn_why_sleeping vfs_read
    Task state: S
    vfs_read
      file = *(struct file *)0xffff8c8a1d585600 = {
        .f_op = (const struct file_operations *)pipeanon_fops+0x0 = 0x...,
        .f_flags = (unsigned int)0,
        .f_inode = (struct inode *)0xffff8c87c87a7200,
        ...
      }

Limitations:
Secure Boot with lockdown=confidentiality) blocks /proc/kcore despite running
as root.

---
<git format-patch will append the diffstat here>

[PATCH] Add drg_why_sleeping.py

[Alexandra Hájková]

---
 .../lib/gdb/command/drgn_why_sleeping.py      | 72 ++++++++++++++++++
 sleep_test.c                                  | 76 +++++++++++++++++++
 2 files changed, 148 insertions(+)
 create mode 100644 gdb/python/lib/gdb/command/drgn_why_sleeping.py
 create mode 100644 sleep_test.c

diff --git a/gdb/python/lib/gdb/command/drgn_why_sleeping.py b/gdb/python/lib/gdb/command/drgn_why_sleeping.py
new file mode 100644
index 00000000000..ea11a73faae
--- /dev/null
+++ b/gdb/python/lib/gdb/command/drgn_why_sleeping.py
@@ -0,0 +1,72 @@
+# GDB 'drgn_why_sleeping' command.
+# Copyright (C) 2026 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Implementation of the GDB 'drgn_why_sleeping' command using the GDB Python API."""
+
+import gdb
+import drgn
+from drgn.helpers.linux.pid import find_task
+from drgn.helpers.linux.sched import task_state_to_char
+
+class DrgnWhySleeping(gdb.Command):
+    """Look at kernel stack and show what is the thread blocked at.
+
+    Usage: Run GDB as root to be able to read from /proc/kcore.
+           The command needs debugging symbols for the running kernel.
+           (gdb) source ~/binutils-gdb/gdb/python/lib/gdb/command/drgn_why_sleeping.py
+           (gdb) info threads
+           (gdb) thread n
+           (gdb) drgn_why_sleeping
+    """
+
+    def __init__(self):
+        super(DrgnWhySleeping, self).__init__(
+            name="drgn_why_sleeping", command_class=gdb.COMMAND_STATUS, prefix=False
+        )
+
+    def invoke(self, arg_str, from_tty):
+        if gdb.selected_thread() is None:
+            raise gdb.GdbError(
+                (
+                    "Can't find selected thread. "
+                )
+            )
+        tid = gdb.selected_thread().ptid[1]
+
+        try:
+            prog = drgn.program_from_kernel()
+        except Exception as e:
+            raise gdb.GdbError(
+                    "Can't read /proc/kcore, %s. Try running GDB as root. " % str(e)
+                    )
+        task = find_task(prog, tid)
+        task_state = task_state_to_char(task)
+        stack = prog.stack_trace(task)
+        print(f"Task state: {task_state}")
+        if arg_str:
+            for frame in stack:
+                if frame.name == arg_str:
+                    print(frame.name)
+                    for var in frame.locals():
+                        print(f"  {var} = {frame[var]}")
+                    break
+            else:
+                print("Frame not found: %s" % arg_str)
+        else:
+            for frame in stack:
+                print(frame.name)
+
+DrgnWhySleeping()
diff --git a/sleep_test.c b/sleep_test.c
new file mode 100644
index 00000000000..c324e55944c
--- /dev/null
+++ b/sleep_test.c
@@ -0,0 +1,76 @@
+/*
+ * sleep_test.c - reproducer for testing drgn_why_sleeping GDB command
+ *
+ * Creates three threads, each blocked in a different kind of interruptible
+ * sleep, then prints the PID and waits so GDB can attach.
+ *
+ * Compile: gcc -g -o sleep_test sleep_test.c -lpthread
+ */
+
+#include <stdio.h>
+#include <pthread.h>
+#include <unistd.h>
+
+static pthread_mutex_t mutex_a = PTHREAD_MUTEX_INITIALIZER;
+static pthread_mutex_t mutex_b = PTHREAD_MUTEX_INITIALIZER;
+static pthread_cond_t  cond    = PTHREAD_COND_INITIALIZER;
+static int pipe_fds[2];
+
+/* Blocked trying to acquire mutex_a, which main holds. */
+static void *mutex_waiter(void *arg)
+{
+    printf("[thread 1] blocking on mutex_a...\n");
+    fflush(stdout);
+    pthread_mutex_lock(&mutex_a);
+    /* never reached during the test */
+    pthread_mutex_unlock(&mutex_a);
+    return NULL;
+}
+
+/* Blocked in pthread_cond_wait; nobody will ever signal cond. */
+static void *cond_waiter(void *arg)
+{
+    pthread_mutex_lock(&mutex_b);
+    printf("[thread 2] blocking on cond...\n");
+    fflush(stdout);
+    pthread_cond_wait(&cond, &mutex_b);
+    /* never reached during the test */
+    pthread_mutex_unlock(&mutex_b);
+    return NULL;
+}
+
+/* Blocked in read(); nobody will write to the pipe. */
+static void *pipe_reader(void *arg)
+{
+    char buf[16];
+    printf("[thread 3] blocking on pipe read...\n");
+    fflush(stdout);
+    read(pipe_fds[0], buf, sizeof(buf));
+    /* never reached during the test */
+    return NULL;
+}
+
+int main(void)
+{
+    pthread_t t1, t2, t3;
+
+    pipe(pipe_fds);
+
+    /* Hold mutex_a before creating thread 1 so it blocks immediately. */
+    pthread_mutex_lock(&mutex_a);
+
+    pthread_create(&t1, NULL, mutex_waiter, NULL);
+    pthread_create(&t2, NULL, cond_waiter,  NULL);
+    pthread_create(&t3, NULL, pipe_reader,  NULL);
+
+    /* Give threads time to reach their blocking calls. */
+    sleep(1);
+
+    printf("\n[main] PID %d ready — attach GDB now\n", getpid());
+    fflush(stdout);
+
+    /* Block here so GDB has time to attach. */
+    pause();
+
+    return 0;
+}
-- 
2.52.0

Re: [PATCH] Add drg_why_sleeping.py

[Tom Tromey]

>>>>> "Alexandra" == Alexandra Hájková <ahajkova@redhat.com> writes:

Hi.  Thanks for the patch.

I'm not sure this is really something gdb ought to ship.

Alexandra>  sleep_test.c                                  | 76 +++++++++++++++++++

This file shouldn't be here.

Alexandra> +           (gdb) source ~/binutils-gdb/gdb/python/lib/gdb/command/drgn_why_sleeping.py

Normally commands are auto-installed.

Alexandra> +           (gdb) drgn_why_sleeping

This isn't a very gdb-ish name.  At the very least gdb usually uses "-"
as a separator; I always thought maybe it inherited a Lispy style from
the RMS days.  But maybe a prefix command would be better anyhow.

Alexandra> +            raise gdb.GdbError(
Alexandra> +                (
Alexandra> +                    "Can't find selected thread. "
Alexandra> +                )
Alexandra> +            )

Extra parens?

Alexandra> +                    "Can't read /proc/kcore, %s. Try running GDB as root. " % str(e)

We can't ever recommend running gdb as root.

I don't normally like to make drastic pronouncements like that, but gdb
is nowhere near hardened enough for this kind of thing.  In fact it's
more accurate to say that gdb doesn't even try, it's ridiculously
exposed.

Tom

Re: [PATCH] Add drg_why_sleeping.py

[Alexandra Petlanova Hajkova]

[-- Attachment #1: Type: text/plain, Size: 1738 bytes --]

On Tue, Sep 1, 2026 at 6:54 PM Tom Tromey <tom@tromey.com> wrote:

> >>>>> "Alexandra" == Alexandra Hájková <ahajkova@redhat.com> writes:
>
> Hi.  Thanks for the patch.
>
> I'm not sure this is really something gdb ought to ship.
>
> Alexandra>  sleep_test.c                                  | 76
> +++++++++++++++++++
>
> This file shouldn't be here.
>
> Alexandra> +           (gdb) source
> ~/binutils-gdb/gdb/python/lib/gdb/command/drgn_why_sleeping.py
>
> Normally commands are auto-installed.
>
Yeah, sorry, I originally didn't specify --data-directory so it ended up not
sourced for me.

>
> Alexandra> +           (gdb) drgn_why_sleeping
>
> This isn't a very gdb-ish name.  At the very least gdb usually uses "-"
> as a separator; I always thought maybe it inherited a Lispy style from
> the RMS days.  But maybe a prefix command would be better anyhow.
>

I'm happy to change the name, it was just the first one that came to mind.

>
>
> Extra parens?
>
> Alexandra> +                    "Can't read /proc/kcore, %s. Try running
> GDB as root. " % str(e)
>
> We can't ever recommend running gdb as root.
>
So,  you recommend just saying we can't read kcore without saying
anything about running GDB as root making things work. Sure, let's
make this bit secret.


> I don't normally like to make drastic pronouncements like that, but gdb
> is nowhere near hardened enough for this kind of thing.  In fact it's
> more accurate to say that gdb doesn't even try, it's ridiculously
> exposed.
>

So,  you recommend just saying we can't read kcore without saying
anything about running GDB as root making things work. Sure, let's
make this bit secret.




>
> Tom
>
>

[-- Attachment #2: Type: text/html, Size: 3048 bytes --]