[PATCHv2] gdb: better handling of 'S' packets

[Andrew Burgess <andrew.burgess@embecosm.com>]

In V2 I've reduced the patch to focus only on the minimum changes
required to fix the original bug (PR gdb/26819).

I think that this new patch is much more obviously the right thing to
do.

Once this is merged I'll put together a new patch with the extra
functionality from V1, but that can come later.

All feedback welcome.

Thanks,
Andrew

---

commit 00190f2b3958e2e822b3d6b078148175f995486c
Author: Andrew Burgess <andrew.burgess@embecosm.com>
Date:   Tue Nov 10 17:54:11 2020 +0000

    gdb: better handling of 'S' packets
    
    This commit builds on work started in the following two commits:
    
      commit 24ed6739b699f329c2c45aedee5f8c7d2f54e493
      Date:   Thu Jan 30 14:35:40 2020 +0000
    
          gdb/remote: Restore support for 'S' stop reply packet
    
      commit cada5fc921e39a1945c422eea055c8b326d8d353
      Date:   Wed Mar 11 12:30:13 2020 +0000
    
          gdb: Handle W and X remote packets without giving a warning
    
    This is related to how GDB handles remote targets that send back 'S'
    packets.
    
    In the first of the above commits we fixed GDB's ability to handle a
    single process, single threaded target that sends back 'S' packets.
    Although the 'T' packet would always be preferred to 'S' these days,
    there's nothing really wrong with 'S' for this situation.
    
    The second commit above fixed an oversight in the first commit, a
    single-process, multi-threaded target can send back a process wide
    event, for example the process exited event 'W' without including a
    process-id, this also is fine as there is no ambiguity in this case.
    
    In PR gdb/26819 we run into yet another problem with the above
    commits.  In this case we have a single process with two threads, GDB
    hits a breakpoint in thread 2 and then performs a stepi:
    
      (gdb) b main
      Breakpoint 1 at 0x1212340830: file infinite_loop.S, line 10.
      (gdb) c
      Continuing.
    
      Thread 2 hit Breakpoint 1, main () at infinite_loop.S:10
      10    in infinite_loop.S
      (gdb) set debug remote 1
      (gdb) stepi
      Sending packet: $vCont;s:2#24...Packet received: S05
      ../binutils-gdb/gdb/infrun.c:5807: internal-error: int finish_step_over(execution_control_state*): Assertion `ecs->event_thread->control.trap_expected' failed.
    
    What happens in this case is that on the RISC-V target displaced
    stepping is not supported, so when the stepi is issued GDB steps just
    thread 2.  As only a single thread was set running the target decides
    that is can get away with sending back an 'S' packet without a
    thread-id.  GDB then associates the stop with thread 1 (the first
    non-exited thread), but as thread 1 was not previously set executing
    the assertion seen above triggers.
    
    As an aside I am surprised that the target sends pack 'S' in this
    situation.  The target is happy to send back 'T' (including thread-id)
    when multiple threads are set running, so (to me) it would seem easier
    to just always use the 'T' packet when multiple threads are in use.
    However, the target only uses 'T' when multiple threads are actually
    executing, otherwise an 'S' packet it used.
    
    Still, when looking at the above situation we can see that GDB should
    be able to understand which thread the 'S' reply is referring too.
    
    The problem is that is that in commit 24ed6739b699 (above) when a stop
    reply comes in with no thread-id we look for the first non-exited
    thread and select that as the thread the stop applies too.
    
    What we should really do is check the threads executing flag too, and
    so find the first non-exited, executing thread, and associate the stop
    event with this thread.  In the above example both thread 1 and 2 are
    non-exited, but only thread 2 is executing, so this is what we should
    use.
    
    Initially I planned to always look for the first non-exited, executing
    thread, however, this doesn't always work.
    
    When GDB initially connects to a target it queries the target for a
    list of threads.  These threads are created within GDB in the
    non-executing state.  GDB then asks the target for the last stop
    reason with the '?'  packet.  If the reply to '?' doesn't include a
    thread-id then GDB needs to look through all the threads and find a
    suitable candidate.  At this point no threads will be marked
    executing, so all we can do is find the first non-exited thread (as we
    currently do).
    
    There's a test for this issue included which works with stock
    gdbserver by disabling use of the 'T' packet, and enabling
    'scheduler-locking' within GDB so only one thread is set running.
    
    gdb/ChangeLog:
    
            PR gdb/26819
            * remote.c
            (remote_target::select_thread_for_ambiguous_stop_reply): New
            member function.
            (remote_target::process_stop_reply): Call
            select_thread_for_ambiguous_stop_reply.
    
    gdb/testsuite/ChangeLog:
    
            PR gdb/26819
            * gdb.server/stop-reply-no-thread-multi.c: New file.
            * gdb.server/stop-reply-no-thread-multi.exp: New file.

diff --git a/gdb/remote.c b/gdb/remote.c
index 71f814efb36..2dc2dc59da2 100644
--- a/gdb/remote.c
+++ b/gdb/remote.c
@@ -747,6 +747,9 @@ class remote_target : public process_stratum_target
   ptid_t process_stop_reply (struct stop_reply *stop_reply,
 			     target_waitstatus *status);
 
+  ptid_t select_thread_for_ambiguous_stop_reply
+    (const struct target_waitstatus *status);
+
   void remote_notice_new_inferior (ptid_t currthread, int executing);
 
   void process_initial_stop_replies (int from_tty);
@@ -7671,75 +7674,201 @@ remote_notif_get_pending_events (remote_target *remote, notif_client *nc)
   remote->remote_notif_get_pending_events (nc);
 }
 
-/* Called when it is decided that STOP_REPLY holds the info of the
-   event that is to be returned to the core.  This function always
-   destroys STOP_REPLY.  */
+/* Called from process_stop_reply when the stop packet we are responding
+   too didn't include a process-id or thread-id.  STATUS is the stop event
+   we are responding too.
+
+   It is the task of this function to select a suitable thread (or process)
+   and return its ptid, this is the thread (or process) we will assume the
+   stop event came from.
+
+   In some cases there isn't really any choice about which thread (or
+   process) is selected, a basic remote with a single process containing a
+   single thread might choose not to send any process-id or thread-id in
+   its stop packets, this function will select and return the one and only
+   thread.
+
+   However, if a target supports multiple threads (or processes) and still
+   doesn't include a thread-id (or process-id) in its stop packet then
+   first, this is a badly behaving target, and second, we're going to have
+   to select a thread (or process) at random and use that.  This function
+   will print a warning to the user if it detects that there is the
+   possibility that GDB is guessing which thread (or process) to
+   report.  */
 
 ptid_t
-remote_target::process_stop_reply (struct stop_reply *stop_reply,
-				   struct target_waitstatus *status)
+remote_target::select_thread_for_ambiguous_stop_reply
+	(const struct target_waitstatus *status)
 {
-  ptid_t ptid;
+  /* Some stop events apply to all threads in an inferior, while others
+     only apply to a single thread.  */
+  bool is_stop_for_all_threads
+    = (status->kind == TARGET_WAITKIND_EXITED
+       || status->kind == TARGET_WAITKIND_SIGNALLED);
 
-  *status = stop_reply->ws;
-  ptid = stop_reply->ptid;
+  struct remote_state *rs = get_remote_state ();
 
-  /* If no thread/process was reported by the stub then use the first
-     non-exited thread in the current target.  */
-  if (ptid == null_ptid)
+  /* Track the possible threads in this structure.  */
+  struct thread_choices
+  {
+    /* Constructor.  */
+    thread_choices (struct remote_state *rs, bool is_stop_for_all_threads)
+      : m_rs (rs),
+	m_is_stop_for_all_threads (is_stop_for_all_threads)
+    { /* Nothing.  */ }
+
+    /* Disable/delete these.  */
+    thread_choices () = delete;
+    DISABLE_COPY_AND_ASSIGN (thread_choices);
+
+    /* Consider thread THR setting the internal thread tracking variables
+       as appropriate.  */
+    void consider_thread (thread_info *thr)
+    {
+      /* Record the first non-exited thread as a fall-back response.  This
+	 is only every used during the initial connection to the target.  */
+      if (m_non_exited_thread == nullptr)
+	m_non_exited_thread = thr;
+      else if (!m_is_stop_for_all_threads
+	       || m_non_exited_thread->ptid.pid () != thr->ptid.pid ())
+	m_non_exited_thread_multiple = true;
+
+      /* If this is an executing thread then it might be a more appropriate
+	 match than just picking the first non-exited thread.  */
+      if (thr->executing)
+	{
+	  if (m_executing_thread == nullptr)
+	    m_executing_thread = thr;
+	  else if (!m_is_stop_for_all_threads
+		   || m_executing_thread->ptid.pid () != thr->ptid.pid ())
+	    m_executing_thread_multiple = true;
+	}
+    }
+
+    /* Return a pointer to the best possible thread.  */
+    thread_info *best_thread () const
+    { return best_thread_inner ().first; }
+
+    /* Return true if there were multiple possible thread/processes and we
+       had to just pick one.  We only worry about seeing multiple
+       candidates in the category from which the BEST_THREAD was selected.  */
+    bool multiple_possible_threads_p () const
+    { return best_thread_inner ().second; }
+
+  private:
+
+    /* Return a pair.  The first item is the best thread that was found,
+       the second is a bool which is true if there were multiple suitable
+       threads seen.  If multiple suitable threads were seen then the best
+       thread that was returned was actually just the first, best, thread.
+       If the bool is false then only one suitable thread was seen.  The
+       best thread will never be nullptr.  */
+    std::pair<thread_info *,bool> best_thread_inner () const
+    {
+      /* The best thread is the first non-exited thread that was marked as
+	 executing.  */
+      std::pair <thread_info *, bool> result (m_executing_thread,
+					      m_executing_thread_multiple);
+
+      /* If there was no thread marked as executing then select a
+	 non-exited thread.  This should only happen during the initial
+	 connection to the target when all threads are marked
+	 non-executing but we still see the very first stop packet.  */
+      if (result.first == nullptr)
+	result = {m_non_exited_thread, m_non_exited_thread_multiple};
+
+      return result;
+    }
+
+    /* The remote state we are examining threads for.  */
+    struct remote_state *m_rs = nullptr;
+
+    /* Is this stop event one for all threads in a process (e.g.  process
+       exited), or an event for a single thread (e.g. thread stopped).  */
+    bool m_is_stop_for_all_threads;
+
+    /* The first thread whose executing flag is true.  */
+    thread_info *m_executing_thread = nullptr;
+
+    /* Set to true if we see multiple possible threads that could be
+       stored into M_EXECUTING_THREAD.  The variable M_EXECUTING_THREAD is
+       set to the first suitable thread seen, then on the second suitable
+       thread this flag is set to true.  */
+    unsigned int m_executing_thread_multiple = false;
+
+    /* The first non-exited thread.  */
+    thread_info *m_non_exited_thread = nullptr;
+
+    /* Set to true if we see multiple possible threads that could be
+       stored into M_NON_EXITED_THREAD.  The variable M_NON_EXITED_THREAD
+       is set to the first suitable thread seen, then on the second suitable
+       thread this flag is set to true.  */
+    unsigned int m_non_exited_thread_multiple = false;
+  } choices (rs, is_stop_for_all_threads);
+
+  /* Consider all non-exited threads to see which is the best match.  */
+  for (thread_info *thr : all_non_exited_threads (this))
+    choices.consider_thread (thr);
+
+  /* Select the best possible thread.  */
+  thread_info *thr = choices.best_thread ();
+  gdb_assert (thr != nullptr);
+
+  /* Warn if the remote target is sending ambiguous stop replies.  */
+  if (choices.multiple_possible_threads_p ())
     {
-      /* Some stop events apply to all threads in an inferior, while others
-	 only apply to a single thread.  */
-      bool is_stop_for_all_threads
-	= (status->kind == TARGET_WAITKIND_EXITED
-	   || status->kind == TARGET_WAITKIND_SIGNALLED);
+      static bool warned = false;
 
-      for (thread_info *thr : all_non_exited_threads (this))
+      if (!warned)
 	{
-	  if (ptid != null_ptid
-	      && (!is_stop_for_all_threads
-		  || ptid.pid () != thr->ptid.pid ()))
-	    {
-	      static bool warned = false;
+	  /* If you are seeing this warning then the remote target has
+	     stopped without specifying a thread-id, but the target
+	     does have multiple threads (or inferiors), and so GDB is
+	     having to guess which thread stopped.
 
-	      if (!warned)
-		{
-		  /* If you are seeing this warning then the remote target
-		     has stopped without specifying a thread-id, but the
-		     target does have multiple threads (or inferiors), and
-		     so GDB is having to guess which thread stopped.
-
-		     Examples of what might cause this are the target
-		     sending and 'S' stop packet, or a 'T' stop packet and
-		     not including a thread-id.
-
-		     Additionally, the target might send a 'W' or 'X
-		     packet without including a process-id, when the target
-		     has multiple running inferiors.  */
-		  if (is_stop_for_all_threads)
-		    warning (_("multi-inferior target stopped without "
-			       "sending a process-id, using first "
-			       "non-exited inferior"));
-		  else
-		    warning (_("multi-threaded target stopped without "
-			       "sending a thread-id, using first "
-			       "non-exited thread"));
-		  warned = true;
-		}
-	      break;
-	    }
+	     Examples of what might cause this are the target sending
+	     and 'S' stop packet, or a 'T' stop packet and not
+	     including a thread-id.
 
-	  /* If this is a stop for all threads then don't use a particular
-	     threads ptid, instead create a new ptid where only the pid
-	     field is set.  */
+	     Additionally, the target might send a 'W' or 'X packet
+	     without including a process-id, when the target has
+	     multiple running inferiors.  */
 	  if (is_stop_for_all_threads)
-	    ptid = ptid_t (thr->ptid.pid ());
+	    warning (_("multi-inferior target stopped without "
+		       "sending a process-id, using first "
+		       "non-exited inferior"));
 	  else
-	    ptid = thr->ptid;
+	    warning (_("multi-threaded target stopped without "
+		       "sending a thread-id, using first "
+		       "non-exited thread"));
+	  warned = true;
 	}
-      gdb_assert (ptid != null_ptid);
     }
 
+  /* If this is a stop for all threads then don't use a particular threads
+     ptid, instead create a new ptid where only the pid field is set.  */
+  return ((is_stop_for_all_threads) ? ptid_t (thr->ptid.pid ()) : thr->ptid);
+}
+
+/* Called when it is decided that STOP_REPLY holds the info of the
+   event that is to be returned to the core.  This function always
+   destroys STOP_REPLY.  */
+
+ptid_t
+remote_target::process_stop_reply (struct stop_reply *stop_reply,
+				   struct target_waitstatus *status)
+{
+  ptid_t ptid;
+
+  *status = stop_reply->ws;
+  ptid = stop_reply->ptid;
+
+  /* If no thread/process was reported by the stub then select a suitable
+     thread/process.  */
+  if (ptid == null_ptid)
+    ptid = select_thread_for_ambiguous_stop_reply (status);
+  gdb_assert (ptid != null_ptid);
+
   if (status->kind != TARGET_WAITKIND_EXITED
       && status->kind != TARGET_WAITKIND_SIGNALLED
       && status->kind != TARGET_WAITKIND_NO_RESUMED)
diff --git a/gdb/testsuite/gdb.server/stop-reply-no-thread-multi.c b/gdb/testsuite/gdb.server/stop-reply-no-thread-multi.c
new file mode 100644
index 00000000000..f0d86bd13c1
--- /dev/null
+++ b/gdb/testsuite/gdb.server/stop-reply-no-thread-multi.c
@@ -0,0 +1,77 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2020 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include <stdlib.h>
+#include <pthread.h>
+#include <unistd.h>
+
+volatile int worker_blocked = 1;
+volatile int main_blocked = 1;
+
+void
+unlock_worker ()
+{
+  worker_blocked = 0;
+}
+
+void
+unlock_main ()
+{
+  main_blocked = 0;
+}
+
+void
+breakpt ()
+{
+  /* Nothing.  */
+}
+
+static void*
+worker (void *data)
+{
+  unlock_main ();
+
+  while (worker_blocked)
+    ;
+
+  breakpt ();
+
+  return NULL;
+}
+
+int
+main ()
+{
+  pthread_t thr;
+  void *retval;
+
+  /* Ensure the test doesn't run forever.  */
+  alarm (99);
+
+  if (pthread_create (&thr, NULL, worker, NULL) != 0)
+    abort ();
+
+  while (main_blocked)
+    ;
+
+  unlock_worker ();
+
+  if (pthread_join (thr, &retval) != 0)
+    abort ();
+
+  return 0;
+}
diff --git a/gdb/testsuite/gdb.server/stop-reply-no-thread-multi.exp b/gdb/testsuite/gdb.server/stop-reply-no-thread-multi.exp
new file mode 100644
index 00000000000..ff80f10243f
--- /dev/null
+++ b/gdb/testsuite/gdb.server/stop-reply-no-thread-multi.exp
@@ -0,0 +1,136 @@
+# This testcase is part of GDB, the GNU debugger.
+#
+# Copyright 2020 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Test how GDB handles the case where a target either doesn't use 'T'
+# packets at all or doesn't include a thread-id in a 'T' packet, AND,
+# where the test program contains multiple threads.
+#
+# In general if multiple threads are executing and the target doesn't
+# include a thread-id in its stop response then GDB will not be able
+# to correctly figure out which thread the stop applies to.
+#
+# However, this test covers a very specific case, there are multiple
+# threads but only a single thread is actually executing.  So, when
+# the stop comes from the target, without a thread-id, GDB should be
+# able to correctly figure out which thread has stopped.
+
+load_lib gdbserver-support.exp
+
+if { [skip_gdbserver_tests] } {
+    verbose "skipping gdbserver tests"
+    return -1
+}
+
+standard_testfile
+if [prepare_for_testing "failed to prepare" $testfile $srcfile {debug pthreads}] {
+    return -1
+}
+
+# Run the tests with different features of GDBserver disabled.
+proc run_test { disable_feature } {
+    global binfile gdb_prompt decimal hex
+
+    clean_restart ${binfile}
+
+    # Make sure we're disconnected, in case we're testing with an
+    # extended-remote board, therefore already connected.
+    gdb_test "disconnect" ".*"
+
+    set packet_arg ""
+    if { $disable_feature != "" } {
+	set packet_arg "--disable-packet=${disable_feature}"
+    }
+    set res [gdbserver_start $packet_arg $binfile]
+    set gdbserver_protocol [lindex $res 0]
+    set gdbserver_gdbport [lindex $res 1]
+
+    # Disable XML-based thread listing, and multi-process extensions.
+    gdb_test_no_output "set remote threads-packet off"
+    gdb_test_no_output "set remote multiprocess-feature-packet off"
+
+    set res [gdb_target_cmd $gdbserver_protocol $gdbserver_gdbport]
+    if ![gdb_assert {$res == 0} "connect"] {
+	return
+    }
+
+    # There should be only one thread listed at this point.
+    gdb_test_multiple "info threads" "" {
+	-re "2 Thread.*$gdb_prompt $" {
+	    fail $gdb_test_name
+	}
+	-re "has terminated.*$gdb_prompt $" {
+	    fail $gdb_test_name
+	}
+	-re "\\\* 1\[\t \]*Thread\[^\r\n\]*\r\n$gdb_prompt $" {
+	    pass $gdb_test_name
+	}
+    }
+
+    gdb_breakpoint "unlock_worker"
+    gdb_continue_to_breakpoint "run to unlock_worker"
+
+    # There should be two threads at this point with thread 1 selected.
+    gdb_test "info threads" \
+	"\\\* 1\[\t \]*Thread\[^\r\n\]*\r\n  2\[\t \]*Thread\[^\r\n\]*" \
+	"second thread should now exist"
+
+    # Switch threads.
+    gdb_test "thread 2" ".*" "switch to second thread"
+
+    # Now turn on scheduler-locking so that when we step thread 2 only
+    # that one thread will be set running.
+    gdb_test_no_output "set scheduler-locking on"
+
+    # Single step thread 2.  Only the one thread will step.  When the
+    # thread stops, if the stop packet doesn't include a thread-id
+    # then GDB should still understand which thread stopped.
+    gdb_test_multiple "stepi" "" {
+	-re "Thread 1 received signal SIGTRAP" {
+	    fail $gdb_test_name
+	}
+	-re "$hex.*$decimal.*while \\(worker_blocked\\).*$gdb_prompt" {
+	    pass $gdb_test_name
+	}
+    }
+
+    # Check that thread 2 is still selected.
+    gdb_test "info threads" \
+	"  1\[\t \]*Thread\[^\r\n\]*\r\n\\\* 2\[\t \]*Thread\[^\r\n\]*" \
+	"second thread should still be selected after stepi"
+
+    # Turn scheduler locking off again so that when we continue all
+    # threads will be set running.
+    gdb_test_no_output "set scheduler-locking off"
+
+    # Continue until exit.  The server sends a 'W' with no PID.
+    # Bad GDB gave an error like below when target is nonstop:
+    #  (gdb) c
+    #  Continuing.
+    #  No process or thread specified in stop reply: W00
+    gdb_continue_to_end "" continue 1
+}
+
+# Disable different features within gdbserver:
+#
+# Tthread: Start GDBserver, with ";thread:NNN" in T stop replies disabled,
+#          emulating old gdbservers when debugging single-threaded programs.
+#
+# T: Start GDBserver with the entire 'T' stop reply packet disabled,
+#    GDBserver will instead send the 'S' stop reply.
+foreach_with_prefix to_disable { "" Tthread T } {
+    run_test $to_disable
+}