Re: [PATCH] 0001-Fix-multi-thread-debug-bug-in-AIX.patch

[Aditya Kamath1 via Gdb-patches <gdb-patches@sourceware.org>]

[-- Attachment #1: Type: text/plain, Size: 9181 bytes --]

Hi Ulrich, Tom and community.

Please find attached the patch. {See: 0001-Fix-multi-thread-debug-bug-in-AIX.patch}

>Not sure if I follow you exactly here, but my understanding is
>indeed that follow_fork should initially create an inferior for
>the new process with just a single main thread using (pid, 0, 0).
>Subsequently, aix-thread should detect that the new inferior
>uses pthreads and then switch its ptid to (pid, 0, tid).

>Not sure where exactly this goes wrong for you.  What is the
>path leading to this crash you're observing?

Let me explain this using the example code pasted below this email. Consider we now follow the child {which is the only case, previous version of the patch was failing}.

---------------------------------------------------------------------
>(gdb) set follow-fork-mode child
>(gdb) set detach-on-fork off
>(gdb) r
>Starting program: /home/aditya/gdb_tests/ultimate-multi-thread-fork
>[New Thread 258]
>[New Thread 515]
>[Attaching after Thread 515 fork to child process 10748216]
>[New inferior 2 (process 10748216)]
---------------------------------------------------------------------

GDB until here know that a fork event occurred and the child process is <10748216, 0, 0> via wait () in the rs6000-aix-nat.c file. The child in the GDB core is <pid, 0 ,0>.

GDB has three things to do here.
** It will get a new object file notifier of the new process,
** new inferior is created so the new inferior notifier {Kindly See initialize_aix_thread () in aix-thread.c where aix_thread_inferior_created () will be called }
** And follow_fork () event since GDB must decide it has to follow the child.
Note:- Our child process is <pid, 0, 0>

This is the exact order in which the above 3 things will be executed from the main GDB event loop.

So, the first task gets done. The new object file notifier has called pd_enable (), to pd_activate () to sync_threadlist () where we will change child process to <pid, 0 ,tid> if the libpthdebug session is initialised successfully.

The second task now, new inferior notifier calls aix_thread_inferior_created () to pd_enable () (), to pd_activate () to sync_threadlist ().

Now we have changed the child process from <pid, 0, 0> to <pid, 0, tid>.

But no one has informed the GDB core about this change. It is still of the view that child process is <pid, 0, 0> from the set_forked () status it got from rs6000-aix-nat wait () while a fork () event was being detected. So when it attempts to handle the third task which is to follow the child process since we have given that command, the child ptid passed to switch_to_thread is <pid, 0 ,0>. But this does not exists. Since the first two events changed it. So GDB got surprised on not finding it and failed to debug further thereby giving an option to dump the core with an assertion below. {Kindly see the back trace I pasted in the previous email. One can verify this}.

-----------------------------------------------------------------------
>thread.c:1337: internal-error: switch_to_thread: Assertion `thr != NULL' ?>failed.
>A problem internal to GDB has been detected,
>further debugging may prove unreliable.
----------------------------------------------------------------------------

The game is about when we decide to set the pd_active or say thread debugging active. We did it in two places. In the wait () and pd_enable () in aix-thread.c file. We need it to put it in the correct place which we aren’t correctly. If there is a need for any non-target function like follow_fork to use ptid_t (pid, 0 , 0) then we must not change the thread ptid though the library is initialised. This is where we are going wrong.

Let me also tell you all what I tried today and failed. I had removed pd_activate () from the pd_enable () and called pd_activate () only from wait () for all the events. {Currently we do only for trap}. But the problem is if we have many inferiors and couple of them with no private data set, since we didn’t in pd_enable () via pd_activate () as we removed it,  then at any point if we iterate_over_threads () over these inferiors we will crash since these don’t have private data set yet.

So this is a tricky problem. Wonder how Linux folks did it. If anyone knows it will be of help if shared. Kindly let me know at high level. I could not understand how they are handling the same.


>But you can determine
>the inferior associated with a PID directly via find_inferior_pid
>without ever involving any thread.

This is done in this patch. Thank you Ulrich.

Let me know what you think. Awaiting for a reply.

Have a nice day ahead.

Thanks and regards,
Aditya.

-------------------------------------------------------------------
Code

Code :-

#include <stdio.h>

#include <unistd.h>

#include <stdlib.h>

#include <pthread.h>

#include <assert.h>


pthread_barrier_t barrier;


#define NUM_THREADS 2


void *

thread_function (void *arg)

{

  /* This ensures that the breakpoint is only hit after both threads

     are created, so the test can always switch to the non-event

     thread when the breakpoint triggers.  */


  pthread_barrier_wait (&barrier);

  pid_t child;


  child = fork ();

  if (child > 0)

    printf ("I am parent \n");

  else

  {

    child = fork ();

    if (child > 0)

      printf ("I am child \n");

    else

      printf ("I am grandchild \n");

  }

  while (1); /* break here */

}


int

main (void)

{

  int i;

  pthread_t thread[NUM_THREADS];


  alarm (300);


  pthread_barrier_init (&barrier, NULL, NUM_THREADS);


  for (i = 0; i < NUM_THREADS; i++)

    {

      int res;


      res = pthread_create (&thread[i], NULL,

                            thread_function, NULL);

      assert (res == 0);

    }


  while (1)

  {

    sleep (15);

  }


  return 0;

}






From: Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
Date: Friday, 17 February 2023 at 1:16 AM
To: simark@simark.ca <simark@simark.ca>, Aditya Kamath1 <Aditya.Kamath1@ibm.com>, gdb-patches@sourceware.org <gdb-patches@sourceware.org>
Cc: Sangamesh Mallayya <sangamesh.swamy@in.ibm.com>
Subject: Re: [PATCH] 0001-Fix-multi-thread-debug-bug-in-AIX.patch
Aditya Kamath1 <Aditya.Kamath1@ibm.com> wrote:

>So what is happening is that the when after a new process is born,
>its pthread library is getting intialised and we have changed its
>ptid from ptid (pid, 0, 0) to ptid (pid, 0, tid). Since we follow
>fork the code in inferior.c file will switch to the thread child
>where the child is reported as ptid (pid, 0, 0) but exists as
>ptid (pid, 0, tid). This leads to this crash. We did try with two
>variables if you recall in the previous patch. But your point of
>pd_active being there for it was valid. So somehow something isn't
>correct that I did not understand. We have pd_activate () in only
>two places. So is follow_fork () is expecting us to switch to child
>process and then change the ptid of the child?? If yes, how do we go??
>And if not where are we going wrong here.

Not sure if I follow you exactly here, but my understanding is
indeed that follow_fork should initially create an inferior for
the new process with just a single main thread using (pid, 0, 0).
Subsequently, aix-thread should detect that the new inferior
uses pthreads and then switch its ptid to (pid, 0, tid).

Not sure where exactly this goes wrong for you.  What is the
path leading to this crash you're observing?


>Also this ptid_t (pid, 0, 0) and our main thread being
>ptid_t (pid, 0, tid) might need a smarted way to switch to the
>main thread's process space and set the right current inferior
>process in pdc_read_memory. Kindly check it in this patch and
>let me know if we can do it better.

So you currently do:

+  thread_info *thread = find_thread_ptid (current_inferior (),
+                                         ptid_t (user_current_pid));
+  /* If the pthread debug library is loaded, then we need the ptid_t (pid, 0 ,tid).
+     Since the main thread in the below for loop will be in the first iteration
+     we will break.  */
+  if (!thread)
+  {
+    for (thread_info *tp: all_threads (current_inferior ()->process_target (),
+                                       ptid_t (user_current_pid)))
+      {
+        thread = tp;
+        break;
+      }
+  }
[...]
+  {
+    scoped_restore save_inferior_ptid = make_scoped_restore (&inferior_ptid);
+    inferior_ptid = ptid_t (user_current_pid);
+    scoped_restore_current_inferior restore_inferior;
+    set_current_inferior (thread->inf);
+
+    scoped_restore_current_program_space restore_current_progspace;
+    set_current_program_space (thread->inf->pspace);
+    status = target_write_memory (addr, (gdb_byte *) buf, len);
+  }

This is overkill.  Note that at no point do you actually ever
use "thread" itself, only "thread->inf".  But you can determine
the inferior associated with a PID directly via find_inferior_pid
without ever involving any thread.

Bye,
Ulrich

[-- Attachment #2: 0001-Fix-multi-thread-debug-bug-in-AIX.patch --]
[-- Type: application/octet-stream, Size: 29654 bytes --]

From b469b0bc402666a7c06bb13b693185980c992f55 Mon Sep 17 00:00:00 2001
From: Aditya Vidyadhar Kamath <Aditya.Kamath1@ibm.com>
Date: Fri, 17 Feb 2023 03:16:34 -0600
Subject: [PATCH] Fix multi-threaded debugging under AIX

Multi-threaded debugging using the libpthdebug debug interface
is currently broken due to multiple issues.

When debugging a single inferior, we were getting assertion
failures in get_aix_thread_info as no tp->priv structure was
allocated for the main thread.

We fixed this by switching the main
thread from a (pid, 0, 0) ptid_t to a (pid, 0, tid) ptid_t and
allocaing the tp->priv structure in sync_threadlists.

As a result, the switch_to_thread call in pdc_read_data could
now fail since the main thread no longer uses (pid, 0, 0).

So we replaced the call by only switching inferior_ptid, the current
inferior, and the current address space (like proc-service.c).
Add similar switching to pdc_write_data where it was missing
completely.

When debugging multiple inferiors, an additional set of
problems prevented correct multi-threaded debugging:

First of all, aix-thread.c used to have a number of global
variables holding per-inferior information.

We switched hese
to a per-inferior data structure instead.

Also, sync_threadlists was getting confused as we were
comparing the list of threads returned by libpthdebug
for *one* process with GDB's list of threads for *all*
processes. Now we only use he GDB threads of the current
inferior instead.

Finally, the presence of the thread library in any but
the first inferior was not correctly detected due to a
bug in solib-aix.c, where the BFD file name for shared
library members was changed when the library was loaded
for the first time, which caused the library to no longer
be recognized by name when loaded a second time,
---
 gdb/aix-thread.c | 369 +++++++++++++++++++++++++++++------------------
 gdb/solib-aix.c  |  14 ++
 2 files changed, 240 insertions(+), 143 deletions(-)

diff --git a/gdb/aix-thread.c b/gdb/aix-thread.c
index e556c153576..c6153e1f71e 100644
--- a/gdb/aix-thread.c
+++ b/gdb/aix-thread.c
@@ -68,10 +68,6 @@ static bool debug_aix_thread;
 #define pthdb_tid_t	tid_t
 #endif
 
-/* Return whether to treat PID as a debuggable thread id.  */
-
-#define PD_TID(ptid)	(pd_active && ptid.tid () != 0)
-
 /* Success and failure values returned by pthdb callbacks.  */
 
 #define PDC_SUCCESS	PTHDB_SUCCESS
@@ -144,24 +140,6 @@ class aix_thread_target final : public target_ops
 
 static aix_thread_target aix_thread_ops;
 
-/* Address of the function that libpthread will call when libpthdebug
-   is ready to be initialized.  */
-
-static CORE_ADDR pd_brk_addr;
-
-/* Whether the current application is debuggable by pthdb.  */
-
-static int pd_able = 0;
-
-/* Whether a threaded application is being debugged.  */
-
-static int pd_active = 0;
-
-/* Whether the current architecture is 64-bit.  
-   Only valid when pd_able is true.  */
-
-static int arch64;
-
 /* Forward declarations for pthdb callbacks.  */
 
 static int pdc_symbol_addrs (pthdb_user_t, pthdb_symbol_t *, int);
@@ -191,9 +169,66 @@ static pthdb_callbacks_t pd_callbacks = {
   NULL
 };
 
-/* Current pthdb session.  */
+/* Aix variable structure.  */
+struct aix_thread_variables 
+{
+  /* Whether the current application is debuggable by pthdb.  */
+  int pd_able;
+
+  /* Whether a threaded application is being debugged.  */
+  int pd_active;
+
+  /* Current pthdb session.  */
+  pthdb_session_t pd_session;
+
+  /* Address of the function that libpthread will call when libpthdebug
+   is ready to be initialized.  */
+  CORE_ADDR pd_brk_addr;
+
+  /* Whether the current architecture is 64-bit.
+   Only valid when pd_able is true.  */
+  int arch64;
+};
+
+/* Key to our per-inferior data.  */
+static const registry<inferior>::key<aix_thread_variables>
+  aix_thread_variables_handle;
+
+/* Function to Get aix_thread_variables data.  */
+static struct aix_thread_variables*
+get_aix_thread_variables_data (struct inferior *inf)
+{
+  if (inf == NULL)
+    return NULL;
 
-static pthdb_session_t pd_session;
+  struct aix_thread_variables* data;
+
+  data = aix_thread_variables_handle.get (inf);
+  if (data == NULL)
+    data = aix_thread_variables_handle.emplace (inf);
+
+  return data;
+}
+
+/* Helper to get data for ptid in a function.  */
+
+static struct aix_thread_variables*
+get_thread_data_helper_for_ptid (ptid_t ptid)
+{
+  inferior *inf = find_inferior_ptid (current_inferior ()->process_target (),
+					ptid);
+  return get_aix_thread_variables_data (inf);
+}
+
+/* Helper to get data for pid in a function.  */
+
+static struct aix_thread_variables*
+get_thread_data_helper_for_pid (pid_t pid)
+{
+  inferior *inf = find_inferior_pid (current_inferior ()->process_target (),
+                                        pid);
+  return get_aix_thread_variables_data (inf);
+}
 
 /* Return a printable representation of pthdebug function return
    STATUS.  */
@@ -318,7 +353,7 @@ pid_to_prc (ptid_t *ptidp)
   ptid_t ptid;
 
   ptid = *ptidp;
-  if (PD_TID (ptid))
+  if (ptid.tid () != 0)
     *ptidp = ptid_t (ptid.pid ());
 }
 
@@ -389,6 +424,9 @@ pdc_read_regs (pthdb_user_t user_current_pid,
   double fprs[ppc_num_fprs];
   struct ptxsprs sprs64;
   struct ptsprs sprs32;
+  struct aix_thread_variables *data;
+  
+  data = get_thread_data_helper_for_pid (user_current_pid);
   
   if (debug_aix_thread)
     gdb_printf (gdb_stdlog, "pdc_read_regs tid=%d flags=%s\n",
@@ -397,7 +435,7 @@ pdc_read_regs (pthdb_user_t user_current_pid,
   /* General-purpose registers.  */
   if (flags & PTHDB_FLAG_GPRS)
     {
-      if (arch64)
+      if (data->arch64)
 	{
 	  if (!ptrace64aix (PTT_READ_GPRS, tid, 
 			    (unsigned long) gprs64, 0, NULL))
@@ -423,7 +461,7 @@ pdc_read_regs (pthdb_user_t user_current_pid,
   /* Special-purpose registers.  */
   if (flags & PTHDB_FLAG_SPRS)
     {
-      if (arch64)
+      if (data->arch64)
 	{
 	  if (!ptrace64aix (PTT_READ_SPRS, tid, 
 			    (unsigned long) &sprs64, 0, NULL))
@@ -456,6 +494,10 @@ pdc_write_regs (pthdb_user_t user_current_pid,
      this is needed, I have implemented what I think it should do,
      however this code is untested.  */
 
+  struct aix_thread_variables *data;
+  
+  data = get_thread_data_helper_for_pid (user_current_pid);
+
   if (debug_aix_thread)
     gdb_printf (gdb_stdlog, "pdc_write_regs tid=%d flags=%s\n",
 		(int) tid, hex_string (flags));
@@ -463,7 +505,7 @@ pdc_write_regs (pthdb_user_t user_current_pid,
   /* General-purpose registers.  */
   if (flags & PTHDB_FLAG_GPRS)
     {
-      if (arch64)
+      if (data->arch64)
 	ptrace64aix (PTT_WRITE_GPRS, tid, 
 		     (unsigned long) context->gpr, 0, NULL);
       else
@@ -479,7 +521,7 @@ pdc_write_regs (pthdb_user_t user_current_pid,
   /* Special-purpose registers.  */
   if (flags & PTHDB_FLAG_SPRS)
     {
-      if (arch64)
+      if (data->arch64)
 	{
 	  ptrace64aix (PTT_WRITE_SPRS, tid, 
 		       (unsigned long) &context->msr, 0, NULL);
@@ -499,7 +541,9 @@ pdc_read_data (pthdb_user_t user_current_pid, void *buf,
 	       pthdb_addr_t addr, size_t len)
 {
   int status, ret;
-
+  inferior *inf = find_inferior_pid (current_inferior ()->process_target (), 
+				     user_current_pid);
+  
   if (debug_aix_thread)
     gdb_printf (gdb_stdlog,
 		"pdc_read_data (user_current_pid = %ld, buf = 0x%lx, addr = %s, len = %ld)\n",
@@ -508,14 +552,17 @@ pdc_read_data (pthdb_user_t user_current_pid, void *buf,
   /* This is needed to eliminate the dependency of current thread
      which is null so that thread reads the correct target memory.  */
   {
-    scoped_restore_current_thread restore_current_thread;
+    scoped_restore save_inferior_ptid = make_scoped_restore (&inferior_ptid);
     /* Before the first inferior is added, we pass inferior_ptid.pid ()
        from pd_enable () which is 0.  There is no need to switch threads
        during first initialisation.  In the rest of the callbacks the
        current thread needs to be correct.  */
-    if (user_current_pid != 0)
-      switch_to_thread (current_inferior ()->process_target (),
-			ptid_t (user_current_pid));
+    inferior_ptid = ptid_t (user_current_pid);
+    scoped_restore_current_inferior restore_inferior;
+    set_current_inferior (inf);
+
+    scoped_restore_current_program_space restore_current_progspace;
+    set_current_program_space (inf->pspace);
     status = target_read_memory (addr, (gdb_byte *) buf, len);
   }
   ret = status == 0 ? PDC_SUCCESS : PDC_FAILURE;
@@ -533,13 +580,25 @@ pdc_write_data (pthdb_user_t user_current_pid, void *buf,
 		pthdb_addr_t addr, size_t len)
 {
   int status, ret;
+  inferior *inf = find_inferior_pid (current_inferior ()->process_target (), 
+                                     user_current_pid);
 
   if (debug_aix_thread)
     gdb_printf (gdb_stdlog,
 		"pdc_write_data (user_current_pid = %ld, buf = 0x%lx, addr = %s, len = %ld)\n",
 		user_current_pid, (long) buf, hex_string (addr), len);
 
-  status = target_write_memory (addr, (gdb_byte *) buf, len);
+  {
+    scoped_restore save_inferior_ptid = make_scoped_restore (&inferior_ptid);
+    inferior_ptid = ptid_t (user_current_pid);
+    scoped_restore_current_inferior restore_inferior;
+    set_current_inferior (inf);
+
+    scoped_restore_current_program_space restore_current_progspace;
+    set_current_program_space (inf->pspace);
+    status = target_write_memory (addr, (gdb_byte *) buf, len);
+  }
+
   ret = status == 0 ? PDC_SUCCESS : PDC_FAILURE;
 
   if (debug_aix_thread)
@@ -639,39 +698,6 @@ pcmp (const void *p1v, const void *p2v)
   return p1->pthid < p2->pthid ? -1 : p1->pthid > p2->pthid;
 }
 
-/* iterate_over_threads() callback for counting GDB threads.
-
-   Do not count the main thread (whose tid is zero).  This matches
-   the list of threads provided by the pthreaddebug library, which
-   does not include that main thread either, and thus allows us
-   to compare the two lists.  */
-
-static int
-giter_count (struct thread_info *thread, void *countp)
-{
-  if (PD_TID (thread->ptid))
-    (*(int *) countp)++;
-  return 0;
-}
-
-/* iterate_over_threads() callback for accumulating GDB thread pids.
-
-   Do not include the main thread (whose tid is zero).  This matches
-   the list of threads provided by the pthreaddebug library, which
-   does not include that main thread either, and thus allows us
-   to compare the two lists.  */
-
-static int
-giter_accum (struct thread_info *thread, void *bufp)
-{
-  if (PD_TID (thread->ptid))
-    {
-      **(struct thread_info ***) bufp = thread;
-      (*(struct thread_info ***) bufp)++;
-    }
-  return 0;
-}
-
 /* ptid comparison function */
 
 static int
@@ -719,7 +745,10 @@ get_signaled_thread (int pid)
 		    sizeof (thrinf), &ktid, 1) != 1)
 	break;
 
-      if (thrinf.ti_cursig == SIGTRAP)
+      /* We also need to keep in mind Trap and interrupt or any
+         signal that needs to be handled in pd_update ().  */
+
+      if (thrinf.ti_cursig)
 	return thrinf.ti_tid;
     }
 
@@ -741,7 +770,7 @@ get_signaled_thread (int pid)
        have difficulty with certain call patterns */
 
 static void
-sync_threadlists (int pid)
+sync_threadlists (pid_t pid) 
 {
   int cmd, status;
   int pcount, psize, pi, gcount, gi;
@@ -750,6 +779,11 @@ sync_threadlists (int pid)
   pthdb_pthread_t pdtid;
   pthread_t pthid;
   pthdb_tid_t tid;
+  process_stratum_target *proc_target
+            = current_inferior ()->process_target ();
+  thread_info  *tp;
+  struct aix_thread_variables *data;
+  data = get_thread_data_helper_for_pid (pid);
 
   /* Accumulate an array of libpthdebug threads sorted by pthread id.  */
 
@@ -759,11 +793,11 @@ sync_threadlists (int pid)
 
   for (cmd = PTHDB_LIST_FIRST;; cmd = PTHDB_LIST_NEXT)
     {
-      status = pthdb_pthread (pd_session, &pdtid, cmd);
+      status = pthdb_pthread (data->pd_session, &pdtid, cmd);
       if (status != PTHDB_SUCCESS || pdtid == PTHDB_INVALID_PTHREAD)
 	break;
 
-      status = pthdb_pthread_ptid (pd_session, pdtid, &pthid);
+      status = pthdb_pthread_ptid (data->pd_session, pdtid, &pthid);
       if (status != PTHDB_SUCCESS || pthid == PTHDB_INVALID_PTID)
 	continue;
 
@@ -780,7 +814,7 @@ sync_threadlists (int pid)
 
   for (pi = 0; pi < pcount; pi++)
     {
-      status = pthdb_pthread_tid (pd_session, pbuf[pi].pdtid, &tid);
+      status = pthdb_pthread_tid (data->pd_session, pbuf[pi].pdtid, &tid);
       if (status != PTHDB_SUCCESS)
 	tid = PTHDB_INVALID_TID;
       pbuf[pi].tid = tid;
@@ -790,13 +824,26 @@ sync_threadlists (int pid)
 
   /* Accumulate an array of GDB threads sorted by pid.  */
 
+  /* gcount is GDB thread count and pcount is pthreadlib thread count.  */
+
   gcount = 0;
-  iterate_over_threads (giter_count, &gcount);
+  for (thread_info *tp : all_threads (proc_target, ptid_t (pid)))
+    gcount++;
   g = gbuf = XNEWVEC (struct thread_info *, gcount);
-  iterate_over_threads (giter_accum, &g);
+  for (thread_info *tp : all_threads (proc_target, ptid_t (pid)))
+    *g++ = tp;
   qsort (gbuf, gcount, sizeof *gbuf, gcmp);
 
+  tp = find_thread_ptid (proc_target, ptid_t (pid));
+
+  /* If the pthreadlibrary is not ready to debug 
+     then this is just a main process which needs 
+     a priv to be set.  The if condition below does 
+     the same.  Otherwise we go to the for loop to 
+     sync the pthread and GDB thread lists.  */
+
   /* Apply differences between the two arrays to GDB's thread list.  */
+
   for (pi = gi = 0; pi < pcount || gi < gcount;)
     {
       if (pi == pcount)
@@ -810,8 +857,6 @@ sync_threadlists (int pid)
 	  priv->pdtid = pbuf[pi].pdtid;
 	  priv->tid = pbuf[pi].tid;
 
-	  process_stratum_target *proc_target
-	    = current_inferior ()->process_target ();
 	  thread = add_thread_with_info (proc_target,
 					 ptid_t (pid, 0, pbuf[pi].pthid),
 					 priv);
@@ -841,13 +886,27 @@ sync_threadlists (int pid)
 	    }
 	  else if (cmp_result > 0)
 	    {
-	      delete_thread (gbuf[gi]);
-	      gi++;
+	      /* This is to make the main process thread now look
+                 like a thread.  */
+
+	      if (gptid.is_pid ())
+		{
+		  thread_change_ptid (proc_target, gptid, pptid);
+		  aix_thread_info *priv = new aix_thread_info;
+                  priv->pdtid = pbuf[pi].pdtid;
+                  priv->tid = pbuf[pi].tid;
+		  tp->priv.reset (priv);
+		  gi++;
+		  pi++;
+		}
+	      else
+		{
+		  delete_thread (gbuf[gi]);
+		  gi++;
+		}
 	    }
 	  else
 	    {
-	      process_stratum_target *proc_target
-		= current_inferior ()->process_target ();
 	      thread = add_thread (proc_target, pptid);
 
 	      aix_thread_info *priv = new aix_thread_info;
@@ -881,17 +940,20 @@ iter_tid (struct thread_info *thread, void *tidp)
    return a pid-only ptid with PID.  */
 
 static ptid_t
-pd_update (int pid)
+pd_update (pid_t pid)
 {
   int status;
   ptid_t ptid;
   pthdb_tid_t tid;
   struct thread_info *thread = NULL;
+  struct aix_thread_variables *data;
+
+  data = get_thread_data_helper_for_pid (pid);
 
-  if (!pd_active)
+  if (!data->pd_active)
     return ptid_t (pid);
 
-  status = pthdb_session_update (pd_session);
+  status = pthdb_session_update (data->pd_session);
   if (status != PTHDB_SUCCESS)
     return ptid_t (pid);
 
@@ -915,34 +977,23 @@ pd_update (int pid)
    for that thread.  Otherwise, return a ptid-only ptid using PID.  */
 
 static ptid_t
-pd_activate (int pid)
+pd_activate (pid_t pid)
 {
   int status;
-		
-  status = pthdb_session_init (pid, arch64 ? PEM_64BIT : PEM_32BIT,
+  struct aix_thread_variables *data;
+  data = get_thread_data_helper_for_pid (pid);
+  
+  status = pthdb_session_init (pid, data->arch64 ? PEM_64BIT : PEM_32BIT,
 			       PTHDB_FLAG_REGS, &pd_callbacks, 
-			       &pd_session);
+			       &data->pd_session);
   if (status != PTHDB_SUCCESS)
     {
       return ptid_t (pid);
     }
-  pd_active = 1;
+  data->pd_active = 1;
   return pd_update (pid);
 }
 
-/* Undo the effects of pd_activate().  */
-
-static void
-pd_deactivate (void)
-{
-  if (!pd_active)
-    return;
-  pthdb_session_destroy (pd_session);
-  
-  pid_to_prc (&inferior_ptid);
-  pd_active = 0;
-}
-
 /* An object file has just been loaded.  Check whether the current
    application is pthreaded, and if so, prepare for thread debugging.  */
 
@@ -952,13 +1003,19 @@ pd_enable (void)
   int status;
   char *stub_name;
   struct bound_minimal_symbol ms;
+  struct aix_thread_variables *data;
+
+  if (!inferior_ptid.pid ())
+    return;
+  
+  data = get_thread_data_helper_for_ptid (inferior_ptid);
 
   /* Don't initialize twice.  */
-  if (pd_able)
+  if (data->pd_able)
     return;
 
   /* Check application word size.  */
-  arch64 = register_size (target_gdbarch (), 0) == 8;
+  data->arch64 = register_size (target_gdbarch (), 0) == 8;
 
   /* Check whether the application is pthreaded.  */
   stub_name = NULL;
@@ -972,13 +1029,13 @@ pd_enable (void)
   ms = lookup_minimal_symbol (stub_name, NULL, NULL);
   if (ms.minsym == NULL)
     return;
-  pd_brk_addr = ms.value_address ();
-  if (!create_thread_event_breakpoint (target_gdbarch (), pd_brk_addr))
+  data->pd_brk_addr = ms.value_address ();
+  if (!create_thread_event_breakpoint (target_gdbarch (), data->pd_brk_addr))
     return;
 
   /* Prepare for thread debugging.  */
   current_inferior ()->push_target (&aix_thread_ops);
-  pd_able = 1;
+  data->pd_able = 1; 
 
   /* If we're debugging a core file or an attached inferior, the
      pthread library may already have been initialized, so try to
@@ -991,28 +1048,31 @@ pd_enable (void)
 static void
 pd_disable (void)
 {
-  if (!pd_able)
+  struct aix_thread_variables *data;
+  data = get_thread_data_helper_for_ptid (inferior_ptid);
+
+  if (!data->pd_able)
+    return;
+  if (!data->pd_active)
     return;
-  if (pd_active)
-    pd_deactivate ();
-  pd_able = 0;
+  pthdb_session_destroy (data->pd_session);
+ 
+  pid_to_prc (&inferior_ptid);
+  data->pd_active = 0;
+  data->pd_able = 0;
   current_inferior ()->unpush_target (&aix_thread_ops);
 }
 
 /* new_objfile observer callback.
 
    If OBJFILE is non-null, check whether a threaded application is
-   being debugged, and if so, prepare for thread debugging.
-
-   If OBJFILE is null, stop debugging threads.  */
+   being debugged, and if so, prepare for thread debugging.  */
 
 static void
 new_objfile (struct objfile *objfile)
 {
   if (objfile)
     pd_enable ();
-  else
-    pd_disable ();
 }
 
 /* Attach to process specified by ARGS.  */
@@ -1042,8 +1102,11 @@ aix_thread_target::resume (ptid_t ptid, int step, enum gdb_signal sig)
 {
   struct thread_info *thread;
   pthdb_tid_t tid[2];
+  struct aix_thread_variables *data;
+  
+  data = get_thread_data_helper_for_ptid (ptid);
 
-  if (!PD_TID (ptid))
+  if (ptid.tid () == 0)
     {
       scoped_restore save_inferior_ptid = make_scoped_restore (&inferior_ptid);
       
@@ -1065,7 +1128,7 @@ aix_thread_target::resume (ptid_t ptid, int step, enum gdb_signal sig)
 	       ptid.lwp ());
       tid[1] = 0;
 
-      if (arch64)
+      if (data->arch64)
 	ptrace64aix (PTT_CONTINUE, tid[0], (long long) 1,
 		     gdb_signal_to_host (sig), (PTRACE_TYPE_ARG5) tid);
       else
@@ -1082,6 +1145,7 @@ ptid_t
 aix_thread_target::wait (ptid_t ptid, struct target_waitstatus *status,
 			 target_wait_flags options)
 {
+  struct aix_thread_variables *data;
   {
     pid_to_prc (&ptid);
 
@@ -1095,8 +1159,10 @@ aix_thread_target::wait (ptid_t ptid, struct target_waitstatus *status,
      pid-only ptids.  */
   gdb_assert (ptid.is_pid ());
 
+  data = get_thread_data_helper_for_ptid (ptid);
+
   /* Check whether libpthdebug might be ready to be initialized.  */
-  if (!pd_active && status->kind () == TARGET_WAITKIND_STOPPED
+  if (!data->pd_active && status->kind () == TARGET_WAITKIND_STOPPED
       && status->sig () == GDB_SIGNAL_TRAP)
     {
       process_stratum_target *proc_target
@@ -1105,7 +1171,7 @@ aix_thread_target::wait (ptid_t ptid, struct target_waitstatus *status,
       struct gdbarch *gdbarch = regcache->arch ();
 
       if (regcache_read_pc (regcache)
-	  - gdbarch_decr_pc_after_break (gdbarch) == pd_brk_addr)
+	  - gdbarch_decr_pc_after_break (gdbarch) == data->pd_brk_addr)
 	return pd_activate (ptid.pid ());
     }
 
@@ -1229,18 +1295,20 @@ fetch_regs_user_thread (struct regcache *regcache, pthdb_pthread_t pdtid)
   ppc_gdbarch_tdep *tdep = gdbarch_tdep<ppc_gdbarch_tdep> (gdbarch);
   int status, i;
   pthdb_context_t ctx;
+  struct aix_thread_variables *data;
+  data = get_thread_data_helper_for_ptid (inferior_ptid);
 
   if (debug_aix_thread)
     gdb_printf (gdb_stdlog, 
 		"fetch_regs_user_thread %lx\n", (long) pdtid);
-  status = pthdb_pthread_context (pd_session, pdtid, &ctx);
+  status = pthdb_pthread_context (data->pd_session, pdtid, &ctx);
   if (status != PTHDB_SUCCESS)
     error (_("aix-thread: fetch_registers: pthdb_pthread_context returned %s"),
 	   pd_status2str (status));
 
   /* General-purpose registers.  */
 
-  if (arch64)
+  if (data->arch64)
     supply_gprs64 (regcache, ctx.gpr);
   else
     for (i = 0; i < ppc_num_gprs; i++)
@@ -1253,7 +1321,7 @@ fetch_regs_user_thread (struct regcache *regcache, pthdb_pthread_t pdtid)
 
   /* Special registers.  */
 
-  if (arch64)
+  if (data->arch64)
     supply_sprs64 (regcache, ctx.iar, ctx.msr, ctx.cr, ctx.lr, ctx.ctr,
 			     ctx.xer, ctx.fpscr);
   else
@@ -1288,18 +1356,21 @@ fetch_regs_kernel_thread (struct regcache *regcache, int regno,
   struct ptxsprs sprs64;
   struct ptsprs sprs32;
   int i;
+  struct aix_thread_variables *data;
+  
+  data = get_thread_data_helper_for_ptid (regcache->ptid ());
 
   if (debug_aix_thread)
     gdb_printf (gdb_stdlog,
 		"fetch_regs_kernel_thread tid=%lx regno=%d arch64=%d\n",
-		(long) tid, regno, arch64);
+		(long) tid, regno, data->arch64);
 
   /* General-purpose registers.  */
   if (regno == -1
       || (tdep->ppc_gp0_regnum <= regno
 	  && regno < tdep->ppc_gp0_regnum + ppc_num_gprs))
     {
-      if (arch64)
+      if (data->arch64)
 	{
 	  if (!ptrace64aix (PTT_READ_GPRS, tid, 
 			    (unsigned long) gprs64, 0, NULL))
@@ -1331,7 +1402,7 @@ fetch_regs_kernel_thread (struct regcache *regcache, int regno,
 
   if (regno == -1 || special_register_p (gdbarch, regno))
     {
-      if (arch64)
+      if (data->arch64)
 	{
 	  if (!ptrace64aix (PTT_READ_SPRS, tid, 
 			    (unsigned long) &sprs64, 0, NULL))
@@ -1362,12 +1433,16 @@ aix_thread_target::fetch_registers (struct regcache *regcache, int regno)
 {
   struct thread_info *thread;
   pthdb_tid_t tid;
+  thread = find_thread_ptid (current_inferior ()->process_target (), ptid_t (regcache->ptid ().pid (), 0, regcache->ptid ().tid ()));
 
-  if (!PD_TID (regcache->ptid ()))
+  /* If a new inferior is born, then its pthread debug library is yet to
+     initialised and hence has no private data. So the below if condition
+     exists.  */
+
+  if (regcache->ptid ().tid () == 0)
     beneath ()->fetch_registers (regcache, regno);
   else
     {
-      thread = find_thread_ptid (current_inferior (), regcache->ptid ());
       aix_thread_info *priv = get_aix_thread_info (thread);
       tid = priv->tid;
 
@@ -1511,6 +1586,8 @@ store_regs_user_thread (const struct regcache *regcache, pthdb_pthread_t pdtid)
   pthdb_context_t ctx;
   uint32_t int32;
   uint64_t int64;
+  struct aix_thread_variables *data;
+  data = get_thread_data_helper_for_ptid (inferior_ptid);
 
   if (debug_aix_thread)
     gdb_printf (gdb_stdlog, 
@@ -1518,7 +1595,7 @@ store_regs_user_thread (const struct regcache *regcache, pthdb_pthread_t pdtid)
 
   /* Retrieve the thread's current context for its non-register
      values.  */
-  status = pthdb_pthread_context (pd_session, pdtid, &ctx);
+  status = pthdb_pthread_context (data->pd_session, pdtid, &ctx);
   if (status != PTHDB_SUCCESS)
     error (_("aix-thread: store_registers: pthdb_pthread_context returned %s"),
 	   pd_status2str (status));
@@ -1528,7 +1605,7 @@ store_regs_user_thread (const struct regcache *regcache, pthdb_pthread_t pdtid)
   for (i = 0; i < ppc_num_gprs; i++)
     if (REG_VALID == regcache->get_register_status (tdep->ppc_gp0_regnum + i))
       {
-	if (arch64)
+	if (data->arch64)
 	  {
 	    regcache->raw_collect (tdep->ppc_gp0_regnum + i, (void *) &int64);
 	    ctx.gpr[i] = int64;
@@ -1545,7 +1622,7 @@ store_regs_user_thread (const struct regcache *regcache, pthdb_pthread_t pdtid)
     fill_fprs (regcache, ctx.fpr);
 
   /* Special registers (always kept in ctx as 64 bits).  */
-  if (arch64)
+  if (data->arch64)
     {
       fill_sprs64 (regcache, &ctx.iar, &ctx.msr, &ctx.cr, &ctx.lr, &ctx.ctr,
 			     &ctx.xer, &ctx.fpscr);
@@ -1576,7 +1653,7 @@ store_regs_user_thread (const struct regcache *regcache, pthdb_pthread_t pdtid)
 	ctx.fpscr = tmp_fpscr;
     }
 
-  status = pthdb_pthread_setcontext (pd_session, pdtid, &ctx);
+  status = pthdb_pthread_setcontext (data->pd_session, pdtid, &ctx);
   if (status != PTHDB_SUCCESS)
     error (_("aix-thread: store_registers: "
 	     "pthdb_pthread_setcontext returned %s"),
@@ -1602,6 +1679,9 @@ store_regs_kernel_thread (const struct regcache *regcache, int regno,
   double fprs[ppc_num_fprs];
   struct ptxsprs sprs64;
   struct ptsprs  sprs32;
+  struct aix_thread_variables *data;
+  
+  data = get_thread_data_helper_for_ptid (regcache->ptid ());
 
   if (debug_aix_thread)
     gdb_printf (gdb_stdlog, 
@@ -1613,7 +1693,7 @@ store_regs_kernel_thread (const struct regcache *regcache, int regno,
       || (tdep->ppc_gp0_regnum <= regno
 	  && regno < tdep->ppc_gp0_regnum + ppc_num_fprs))
     {
-      if (arch64)
+      if (data->arch64)
 	{
 	  /* Pre-fetch: some regs may not be in the cache.  */
 	  ptrace64aix (PTT_READ_GPRS, tid, (unsigned long) gprs64, 0, NULL);
@@ -1646,7 +1726,7 @@ store_regs_kernel_thread (const struct regcache *regcache, int regno,
 
   if (regno == -1 || special_register_p (gdbarch, regno))
     {
-      if (arch64)
+      if (data->arch64)
 	{
 	  /* Pre-fetch: some registers won't be in the cache.  */
 	  ptrace64aix (PTT_READ_SPRS, tid, 
@@ -1703,7 +1783,7 @@ aix_thread_target::store_registers (struct regcache *regcache, int regno)
   struct thread_info *thread;
   pthdb_tid_t tid;
 
-  if (!PD_TID (regcache->ptid ()))
+  if (regcache->ptid ().tid () == 0)
     beneath ()->store_registers (regcache, regno);
   else
     {
@@ -1741,7 +1821,7 @@ aix_thread_target::mourn_inferior ()
 {
   target_ops *beneath = this->beneath ();
 
-  pd_deactivate ();
+  pd_disable ();
   beneath->mourn_inferior ();
 }
 
@@ -1750,7 +1830,7 @@ aix_thread_target::mourn_inferior ()
 bool
 aix_thread_target::thread_alive (ptid_t ptid)
 {
-  if (!PD_TID (ptid))
+  if (ptid.tid () == 0)
     return beneath ()->thread_alive (ptid);
 
   /* We update the thread list every time the child stops, so all
@@ -1766,7 +1846,7 @@ aix_thread_target::thread_alive (ptid_t ptid)
 std::string
 aix_thread_target::pid_to_str (ptid_t ptid)
 {
-  if (!PD_TID (ptid))
+  if (ptid.tid () == 0)
     return beneath ()->pid_to_str (ptid);
 
   return string_printf (_("Thread %s"), pulongest (ptid.tid ()));
@@ -1786,8 +1866,11 @@ aix_thread_target::extra_thread_info (struct thread_info *thread)
   pthdb_detachstate_t detachstate;
   int cancelpend;
   static char *ret = NULL;
+  struct aix_thread_variables *data;
+
+  data = get_thread_data_helper_for_ptid (thread->ptid);
 
-  if (!PD_TID (thread->ptid))
+  if (thread->ptid.tid () == 0)
     return NULL;
 
   string_file buf;
@@ -1800,24 +1883,24 @@ aix_thread_target::extra_thread_info (struct thread_info *thread)
     /* i18n: Like "thread-identifier %d, [state] running, suspended" */
     buf.printf (_("tid %d"), (int)tid);
 
-  status = pthdb_pthread_state (pd_session, pdtid, &state);
+  status = pthdb_pthread_state (data->pd_session, pdtid, &state);
   if (status != PTHDB_SUCCESS)
     state = PST_NOTSUP;
   buf.printf (", %s", state2str (state));
 
-  status = pthdb_pthread_suspendstate (pd_session, pdtid, 
+  status = pthdb_pthread_suspendstate (data->pd_session, pdtid, 
 				       &suspendstate);
   if (status == PTHDB_SUCCESS && suspendstate == PSS_SUSPENDED)
     /* i18n: Like "Thread-Id %d, [state] running, suspended" */
     buf.printf (_(", suspended"));
 
-  status = pthdb_pthread_detachstate (pd_session, pdtid, 
+  status = pthdb_pthread_detachstate (data->pd_session, pdtid, 
 				      &detachstate);
   if (status == PTHDB_SUCCESS && detachstate == PDS_DETACHED)
     /* i18n: Like "Thread-Id %d, [state] running, detached" */
     buf.printf (_(", detached"));
 
-  pthdb_pthread_cancelpend (pd_session, pdtid, &cancelpend);
+  pthdb_pthread_cancelpend (data->pd_session, pdtid, &cancelpend);
   if (status == PTHDB_SUCCESS && cancelpend)
     /* i18n: Like "Thread-Id %d, [state] running, cancel pending" */
     buf.printf (_(", cancel pending"));
diff --git a/gdb/solib-aix.c b/gdb/solib-aix.c
index f483f54de13..6be81064ebd 100644
--- a/gdb/solib-aix.c
+++ b/gdb/solib-aix.c
@@ -618,6 +618,20 @@ solib_aix_bfd_open (const char *pathname)
       if (member_name == bfd_get_filename (object_bfd.get ()))
 	break;
 
+      std::string s = bfd_get_filename (object_bfd.get ());
+
+      /* For every inferior after first int bfd system we 
+	 will have the pathname instead of the member name
+	 registered. Hence the below condition exists.  */
+
+      if (s.find ('(') != std::string::npos)
+	{
+	  int pos = s.find ('(');
+	  int len = s.find (')') - s.find ('(');
+	  if (s.substr (pos+1, len-1) == member_name) 
+	    return object_bfd;
+	}
+
       object_bfd = gdb_bfd_openr_next_archived_file (archive_bfd.get (),
 						     object_bfd.get ());
     }
-- 
2.31.1