Re: [PATCH] gdb/remote: fix assertions when attaching in non-stop mode

[Simon Marchi <simark@simark.ca>]

On 6/11/26 1:36 AM, Martin KOCH wrote:
> When connecting to a remote target in non-stop mode against a
> multi-threaded inferior stopped at raise(SIGSTOP), three internal-error
> assertions can fire in sequence:
> 
>   remote.c:546: mark_async_event_handler: Assertion 'this->is_async_p ()' failed.
>   thread.c:429: set_pending_waitstatus: Assertion 'this->internal_state () ==
>                 THREAD_INT_STOPPED || ...' failed.
>   thread.c:426: set_pending_waitstatus: Assertion '!this->has_pending_waitstatus ()' failed.
> 
> The first one is what PR 30630 reports and is fixed with the PR proposed
> by Mikhail Terekhov, but with the fix the two other assertions surface:
> 
>  * thread.c:429: addressed by reordering set_internal_state / set_state
>    to run before set_pending_waitstatus.
>  * thread.c:426: addressed by clearing any existing pending wait
>    status before installing the new one, when gdbserver delivers
>    multiple events for the same thread.
> 
> [1] https://sourceware.org/pipermail/gdb-patches/2023-October/202937.html
> 
> Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=30630
> Signed-off-by: Martin KOCH <Martin.KOCH@bachmann.info>

Hi Martin,

We will definitely want a test for this.

I played with the reproducer a little bit, I think that the
target_is_async_p and the reorder set_internal_state vs
set_pending_waitstatus fixes make sense.

For the "clear duplicate waitstatus" one, I'd like to understand the
root cause a bit better.  The fix is not necessarily bad, in order to
protect against misbehaving remotes, but I'm thinking that if GDB really
sees duplicate stop replies for a given thread, then perhaps gdbserver
is doing something wrong that should also be fixed.

I looked at the RSP logs, and saw this:

  [remote] Sending packet: $?#3f
  [remote] Packet received: T1206:e0419624fd7f0000;07:b8419624fd7f0000;10:520aaa1ba67f0000;thread:p194d4.194d4;core:a;
  [remote] Sending packet: $vStopped#55
  [remote] Packet received: T1206:c0ed9f1ba67f0000;07:98ed9f1ba67f0000;10:520aaa1ba67f0000;thread:p194d4.194d5;core:d;
  [remote] Sending packet: $vStopped#55
  [remote] Packet received: OK

Seems fine, one event for each thread.  But I scrolled up and saw:

  [remote] Sending packet: $QNonStop:1#8d
  [remote] Packet received: OK
  [remote] Sending packet: $qXfer:threads:read::0,1000#92
  [remote]   Notification received: Stop:T1206:e0419624fd7f0000;07:b8419624fd7f0000;10:520aaa1ba67f0000;thread:p194d4.194d4;core:a;
  [remote] Packet received: l<threads>\n<thread id="p194d4.194d4" core="10" name="test"/>\n<thread id="p194d4.194d5" core="13" name="test"/>\n</threads>\n
  [remote] Sending packet: $vStopped#55
  [remote] Packet received: OK

That's probably it: early on, GDB sent an asynchronous stop reply for
one of the threads.  So here, I end up with two stop replies for thread
194d4.

On page

  https://sourceware.org/gdb/current/onlinedocs/gdb.html/Remote-Non_002dStop.html#Remote-Non_002dStop

we read:

  In non-stop mode, the target shall respond to the ‘?’ packet as
  follows. First, any incomplete stop reply notification/‘vStopped’
  sequence in progress is abandoned. The target must begin a new
  sequence reporting stop events for all stopped threads, whether or not
  it has previously reported those events to GDB. The first stop reply
  is sent as a synchronous reply to the ‘?’ packet, and subsequent stop
  replies are sent as responses to ‘vStopped’ packets using the
  mechanism described above. The target must not send asynchronous stop
  reply notifications until the sequence is complete. If all threads are
  running when the target receives the ‘?’ packet, or if the target is
  not attached to any process, it shall respond ‘OK’.

I could be wrong, but my interpretation of the "target must not send
asynchronous stop reply notifications until the sequence is complete"
part is that gdbserver should refrain from sending async stop replies
until GDB has sent packet ? and the whole dialog for packet ? is over.
So that would be a gdbserver bug.

Another way to interpret the sentence could be that the target must not
send async stop replies only during the window that starts when
receiving the ? packet and ends when sending the final OK that ends the
? dialog.  But that would be surprising.

If we agree it's a gdbserver bug, then ideally we would have:

 - patch that fixes the gdbserver bug
 - patch that fixes the other two asserts, with a test (it should work
   fine at this point)
 - optionally, patch that adds handling for duplicate stop replies /
   misbehaving targets

To test the last one, it is always possible to add knobs to gdbserver to
make it misbehave on purpose.  Or, there are tests that run the program
once to record the RSP communication (for use with gdbreplay), modify
the log to insert something wrong (like here, you could make the target
send back two stop replies for the same thread), and replay using
gdbreplay.  But this is starting to sound like some scope creep, if we
just had the first two I'd be happy.

Also, I just saw that patches 2 and 3 in Mohamed's series here are
basically the same fixes as you.

https://inbox.sourceware.org/gdb-patches/20260518183316.127043-1-mohamed.bouhaouel@intel.com/M

Either way, I think it would be good to have a test based on the
reproducer in bug 30630.

Thanks,

Simon