[PATCH] Fix Windows sharing_input_terminal hang

Pedro Alves
2023-08-22T15:13:51Z

After running a number of programs under Windows gdb and detaching
them, I typed run in gdb, and got a hang, here:

(top-gdb) bt
#0 sharing_input_terminal (pid=4672) at /home/pedro/gdb/src/gdb/mingw-hdep.c:388
#1 0x00007ff71a2d8678 in sharing_input_terminal (inf=0x23bf23dafb0) at /home/pedro/gdb/src/gdb/inflow.c:269
#2 0x00007ff71a2d887b in child_terminal_save_inferior (self=0x23bf23de060) at /home/pedro/gdb/src/gdb/inflow.c:423
#3 0x00007ff71a2c80c0 in inf_child_target::terminal_save_inferior (this=0x23bf23de060) at /home/pedro/gdb/src/gdb/inf-child.c:111
#4 0x00007ff71a429c0f in target_terminal_is_ours_kind (desired_state=target_terminal_state::is_ours_for_output) at /home/pedro/gdb/src/gdb/target.c:1037
#5 0x00007ff71a429e02 in target_terminal::ours_for_output () at /home/pedro/gdb/src/gdb/target.c:1094
#6 0x00007ff71a2ccc8e in post_create_inferior (from_tty=0) at /home/pedro/gdb/src/gdb/infcmd.c:245
#7 0x00007ff71a2cd431 in run_command_1 (args=0x0, from_tty=0, run_how=RUN_NORMAL) at /home/pedro/gdb/src/gdb/infcmd.c:502
#8 0x00007ff71a2cd58b in run_command (args=0x0, from_tty=0) at /home/pedro/gdb/src/gdb/infcmd.c:527

The problem is that the loop around GetConsoleProcessList looped
forever, because there were exactly 10 processes to return.
GetConsoleProcessList's documentation says:

If the buffer is too small to hold all the valid process identifiers,
the return value is the required number of array elements. The
function will have stored no identifiers in the buffer. In this
situation, use the return value to allocate a buffer that is large
enough to store the entire list and call the function again.

In this case, the buffer wasn't too small, it was exactly the right
size, so we should have broken out of the loop. We didn't due to a
"<" check that should have been "<=". That is fixed by this patch.

Change-Id: I14e4909f2ac2fa83d0d9b6e64418b5831ac4e4e3
---
gdb/mingw-hdep.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gdb/mingw-hdep.c b/gdb/mingw-hdep.c
index 75f63a3fe17..137a63eb244 100644
--- a/gdb/mingw-hdep.c
+++ b/gdb/mingw-hdep.c
@@ -385,7 +385,7 @@ sharing_input_terminal (int pid)
len = GetConsoleProcessList (results.data (), results.size ());
/* Note that LEN == 0 is a failure, but we can treat it the same
as a "no". */
- if (len < results.size ())
+ if (len <= results.size ())
break;

results.resize (len);

base-commit: 11e3488d3f787d51196a115fd9c6085e57bd9626
--
2.34.1