[PATCH v1 2/2] Add TLS support for shared libraries in AIX

[PATCH v1 2/2] Add TLS support for shared libraries in AIX

[Aditya Vidyadhar Kamath]

From: Aditya Kamath <Aditya.Kamath1@ibm.com>

This patch adds support to debug thread local variables defined in shared libraries in AIX.

Sample debug output of this patch is as below
Thread 3 hit Breakpoint 1, thread_runner (arg=0x2) at tls_main.c:36
36        volatile int bp_here = 0; (void)bp_here;

$3 = 20
$4 = 40
thread 2: my_tls_var=20  lib_tls_var=40
[Thread 1 (tid 101646715) (id 1) exited]
[Thread 515 (tid 88015327) (id 3) exited]
[Inferior 1 (process 21758254) exited normally]

where lib_tls_var=40 is a variable from a thread library.

For initial-exec (R_TLS_IE): read the runtime TP-relative offset from the
.loader section TOC slot written by the AIX loader, returning it with bit 0
set as a sentinel so get_thread_local_address() uses it directly without
re-adding the static symbol value.

For global-dynamic/local-dynamic (R_TLSM): scan the .loader section for the
R_TLSM relocation, read the 8-byte mod_id from the inferior's TOC, and
dereference it as a TP-relative pointer to the per-thread block, then add the
intra-module variable offset.

Local-exec (main executable) continues to use the existing tp + offset path.

This patch implements the same as mentioned above.
---
 gdb/rs6000-aix-tdep.c | 205 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 179 insertions(+), 26 deletions(-)

diff --git a/gdb/rs6000-aix-tdep.c b/gdb/rs6000-aix-tdep.c
index aa68654ff41..db81c91524c 100644
--- a/gdb/rs6000-aix-tdep.c
+++ b/gdb/rs6000-aix-tdep.c
@@ -40,6 +40,11 @@
 #include "trad-frame.h"
 #include "frame-unwind.h"
 #include "inferior.h"
+#include "bfd.h"
+#include "coff/internal.h"
+#include "libcoff.h"
+#include "coff/xcoff.h"
+#include "libxcoff.h"
 
 /* If the kernel has to deliver a signal, it pushes a sigcontext
    structure on the stack and then calls the signal handler, passing
@@ -69,6 +74,9 @@
 /* Minimum possible text address in AIX.  */
 #define AIX_TEXT_SEGMENT_BASE 0x10000000
 
+/* R_TLSM relocation type stored in the low byte of internal_ldrel.l_rtype.  */
+#define XCOFF_R_TLSM 0x24
+
 struct rs6000_aix_reg_vrreg_offset
 {
   int vr0_offset;
@@ -1358,45 +1366,165 @@ rs6000_aix_core_xfer_shared_libraries_aix (struct gdbarch *gdbarch,
 
 /* For AIX, use the rs6000_aix_fetch_tls_load_module_address gdbarch method.
 
-   Thread-local variables accessed via a simple TP-relative offset (the Local
-   Exec and Initial Exec TLS models) are only valid for the main executable and
-   for shared libraries that were pulled in at program startup.  Variables
-   belonging to a library loaded later via dlopen() have their storage
-   allocated dynamically; their addresses must be resolved through the DTV
-   (Dynamic Thread Vector) and cannot be computed with a plain TP offset.
+   AIX 64-bit TLS defines four access models, they are:
+
+   local-exec:
+     The TLS variable belongs to the main executable.
+
+   initial-exec:
+     The TLS variable may be in the main executable or a shared library
+     that was present at program startup (i.e. not dlopen'd).
+
+   local-dynamic:
+     The TLS variable is in the same module but it may be a shared library.
+
+   global-dynamic:
+     The general case for shared libraries.  The compiler generates a
+     per-module TOC entry pair:
+       TOC[mod_entry]  -- filled with "mod_id" by the R_TLSM loader relocation
+       TOC[off_entry]  -- filled with the intra-module variable offset by R_TLS
+     mod_id is a signed TP-relative offset to the per-thread block pointer
+     for this module's TLS storage:
+       per_thread_block_ptr = *(thread_pointer + mod_id)
+       variable_addr        = per_thread_block_ptr + xcoff_symbol_value
 
-   Until full DTV-based lookup is implemented, reject any objfile that is a
-   shared library (OBJF_SHARED) which has been dlopen'd and only
-   allow the main executable (OBJF_MAINLINE).  Returning 0 here causes
-   rs6000_aix_get_thread_local_address() to be called with lm_addr == 0,
-   which it treats as the signal to use the TP-relative path.  */
+   In practice, local-exec and initial-exec both resolve via a plain
+   TP-relative offset (lm_addr == 0).  local-dynamic and global-dynamic
+   both require the mod_id indirection (lm_addr != 0) and are handled
+   in rs6000_aix_fetch_tls_load_module_address ().
+
+   To obtain mod_id for a shared library we scan the library's loader
+   section for the first R_TLSM relocation, read the virtual address of the
+   TOC entry it targets, apply the objfile's data-section relocation to get
+   the runtime address, and then read the 8-byte mod_id from the inferior.  */
 
 static CORE_ADDR
 rs6000_aix_fetch_tls_load_module_address (struct objfile *objfile)
 {
-  /* TLS variables from shared libraries cannot be directly fetched
-     via the thread pointer if they were loaded by dlopen().  */
-  if (objfile->flags & OBJF_SHARED)
+  /* Main executable so return 0.  */
+  if (!(objfile->flags & OBJF_SHARED))
+    return 0;
+
+  bfd *abfd = objfile->obfd.get ();
+  if (abfd == nullptr)
     throw_error (TLS_GENERIC_ERROR,
-		 _("TLS lookup via thread pointer is not supported for "
-		   "shared library \"%s\"; full DTV-based lookup is not "
-		   "yet implemented for AIX"),
+		 _("Cannot resolve TLS for \"%s\": no BFD"),
 		 objfile_name (objfile));
 
-  return 0;
+  /* Read the .loader section, which contains the header, symbol table,
+     and relocation table for the shared library.  */
+  asection *loader_sec = bfd_get_section_by_name (abfd, ".loader");
+  if (loader_sec == nullptr)
+    throw_error (TLS_GENERIC_ERROR,
+		 _("Cannot resolve TLS for \"%s\": no .loader section"),
+		 objfile_name (objfile));
+
+  bfd_size_type loader_size = bfd_section_size (loader_sec);
+  gdb::byte_vector loader_buf (loader_size);
+  if (!bfd_get_section_contents (abfd, loader_sec, loader_buf.data (),
+				 0, loader_size))
+    throw_error (TLS_GENERIC_ERROR,
+		 _("Cannot resolve TLS for \"%s\": cannot read .loader section"),
+		 objfile_name (objfile));
+
+  /* Parse the loader header to find the relocation table offset and count.  */
+  struct internal_ldhdr ldhdr;
+  bfd_xcoff_swap_ldhdr_in (abfd, loader_buf.data (), &ldhdr);
+
+  bfd_vma reloc_start = bfd_xcoff_loader_reloc_offset (abfd, &ldhdr);
+  bfd_size_type reloc_size = bfd_xcoff_ldrelsz (abfd);
+
+  if (reloc_start + ldhdr.l_nreloc * reloc_size > loader_size)
+    throw_error (TLS_GENERIC_ERROR,
+		 _("Cannot resolve TLS for \"%s\": .loader section truncated"),
+		 objfile_name (objfile));
+
+  /* Scan loader relocations for TLS reloc types.
+     l_rtype is a 16-bit field: high byte = reloc size, low byte = reloc type
+
+     R_TLS_IE (0x21) = initial-exec: the TOC slot holds the complete
+       TP-relative address of the variable written at load time by the
+       AIX loader.  For shared libraries this value differs from the
+       static XCOFF symbol value.  We must read the runtime TOC slot.
+       Bit 0 of lm_addr is set as a sentinel to distinguish this path
+
+     R_TLSM (0x24) = global-dynamic / local-dynamic: the TOC slot holds
+       mod_id, a signed TP-relative offset to the per-thread block pointer.
+       get_thread_local_address() then dereferences it and adds offset.  */
+
+  const gdb_byte *reloc_ptr = loader_buf.data () + reloc_start;
+  for (size_t i = 0; i < ldhdr.l_nreloc; i++, reloc_ptr += reloc_size)
+    {
+      struct internal_ldrel ldrel;
+      bfd_xcoff_swap_ldrel_in (abfd, reloc_ptr, &ldrel);
+
+      int rtype = ldrel.l_rtype & 0xff;
+      if (rtype != R_TLS_IE && rtype != XCOFF_R_TLSM)
+	continue;
+
+      /* ldrel.l_vaddr is the static (pre-relocation) address of the TOC
+	 entry the loader fills with the TLS value.  Apply the .data section
+	 relocation offset to get the runtime TOC slot address.  */
+      CORE_ADDR toc_addr = ldrel.l_vaddr + objfile->data_section_offset ();
+
+      /* Read the 8-byte value the AIX loader wrote into the TOC slot.  */
+      gdb_byte buf[8];
+      if (target_read_memory (toc_addr, buf, sizeof buf) != 0)
+	throw_error (TLS_GENERIC_ERROR,
+		     _("Cannot resolve TLS for \"%s\": "
+		       "failed to read TLS TOC slot from inferior"),
+		     objfile_name (objfile));
+
+      CORE_ADDR toc_val
+	= extract_signed_integer (buf, sizeof buf, BFD_ENDIAN_BIG);
+
+      /* A zero or positive value means the loader has not yet written the
+	 TLS offset (storage not yet allocated).  */
+      if ((LONGEST) toc_val >= 0)
+	throw_error (TLS_NOT_ALLOCATED_YET_ERROR,
+		     _("TLS storage not yet allocated for \"%s\""),
+		     objfile_name (objfile));
+
+      if (rtype == R_TLS_IE)
+	{
+	  /* initial-exec: return runtime TP-relative offset with bit 0 set
+	     so get_thread_local_address() uses it directly without adding
+	     the symbol offset (which is the static .tdata offset, not the
+	     runtime TP offset for a shared library).  */
+	  return (CORE_ADDR) toc_val | 1;
+	}
+
+      /* R_TLSM: return mod_id as-is (bit 0 is always 0).  */
+      return (CORE_ADDR) toc_val;
+    }
+
+  throw_error (TLS_GENERIC_ERROR,
+	       _("Cannot resolve TLS for \"%s\": "
+		 "no TLS relocation (R_TLS_IE or R_TLSM) found "
+		 "in .loader section"),
+	       objfile_name (objfile));
 }
 
 /* Use the AIX get_thread_local_address gdbarch function.
 
-   On 64-bit AIX the thread pointer is in R13.  For the Local Exec TLS model
-   (used by the main executable) the XCOFF symbol value is a signed
-   TP-relative offset, so the per-thread address is:
+   On 64-bit AIX the thread pointer is in R13.
+
+   lm_addr == 0  (local-exec, main executable):
+     The XCOFF symbol value is a signed TP-relative offset baked in at
+     link time:
+       address = tp + (int64_t) offset
 
-     address = thread_pointer + (int64_t) offset
+   lm_addr has bit 0 set  (initial-exec, shared library):
+     The full runtime TP-relative offset was read from the R_TLS_IE TOC
+     slot by rs6000_aix_fetch_tls_load_module_address().  Do not add the
+     symbol offset again:
+       address = tp + (int64_t)(lm_addr & ~1)
 
-   This only works correctly when the variable's storage is allocated
-   statically relative to the thread pointer, which is guaranteed for the
-   main executable. */
+   lm_addr != 0 and bit 0 clear  (global-dynamic / local-dynamic):
+     lm_addr is mod_id from the R_TLSM TOC slot a signed TP-relative
+     offset to the per-thread block pointer.  Dereference it, then add
+     the intra-module symbol offset:
+       address = *(tp + (int64_t) lm_addr) + (int64_t) offset  */
 
 static CORE_ADDR
 rs6000_aix_get_thread_local_address (struct gdbarch *gdbarch, ptid_t ptid,
@@ -1420,7 +1548,32 @@ rs6000_aix_get_thread_local_address (struct gdbarch *gdbarch, ptid_t ptid,
     throw_error (TLS_GENERIC_ERROR,
 		 _("Unable to fetch thread pointer for TLS lookup"));
 
-  return tp + (CORE_ADDR)(int64_t) offset;
+  if (lm_addr == 0)
+    {
+      /* local-exec (main executable): XCOFF symbol value is TP-relative.  */
+      return tp + (CORE_ADDR)(int64_t) offset;
+    }
+  else if (lm_addr & 1)
+    {
+      /* initial-exec (shared library): lm_addr holds the full runtime
+	 TP-relative offset from the R_TLS_IE TOC slot, with bit 0 set as
+	 a sentinel.  Strip the sentinel and compute the address directly.  */
+      return tp + (CORE_ADDR)(int64_t)(lm_addr & ~(CORE_ADDR)1);
+    }
+  else
+    {
+      /* global-dynamic / local-dynamic: lm_addr is mod_id from R_TLSM.
+	 Dereference the per-thread block pointer, then add the symbol offset.  */
+      CORE_ADDR region_ptr_addr = tp + (CORE_ADDR)(int64_t) lm_addr;
+      gdb_byte buf[8];
+      if (target_read_memory (region_ptr_addr, buf, sizeof buf) != 0)
+	throw_error (TLS_GENERIC_ERROR,
+		     _("Cannot read TLS region pointer for thread-local lookup"));
+
+      CORE_ADDR region_base
+	= extract_unsigned_integer (buf, sizeof buf, BFD_ENDIAN_BIG);
+      return region_base + (CORE_ADDR)(int64_t) offset;
+    }
 }
 
 static void
-- 
2.51.2

Re: [PATCH v1 2/2] Add TLS support for shared libraries in AIX

[Ulrich Weigand]

Aditya Vidyadhar Kamath <akamath996@gmail.com> wrote:

>For initial-exec (R_TLS_IE): read the runtime TP-relative offset from
>the .loader section TOC slot written by the AIX loader, returning
>it with bit 0 set as a sentinel so get_thread_local_address() uses
>it directly without re-adding the static symbol value.

>For global-dynamic/local-dynamic (R_TLSM): scan the .loader section
>for the R_TLSM relocation, read the 8-byte mod_id from the inferior's
>TOC, and dereference it as a TP-relative pointer to the per-thread
>block, then add the intra-module variable offset.

This is an interesting approach, which is quite different from what
Linux is doing.   The one question/concern I have this: are these
relocations present in the module where the TLS symbol is *defined*,
or rather in the module where the TLS symbol is being *used*.  On
Linux (ELF), it would be the latter - but I don't know all the AIX
(XCOFF) details here.

If it is the latter, then this approach has the drawback that the
debugger can only access symbols that are also accessed somewhere
within the program being debugged (in fact, with the current patch,
only symbols that are used in the same module they are defined in).
With the approach used on Linux, it is in principle possible to
access TLS symbols whether they are used or not.  (If there's no
better way to implement this, that of course may still a limitation
that can be accepted.)

Either way, it would be good to add test cases for the various
scenarios.

Bye,
Ulrich

Re: [PATCH v1 2/2] Add TLS support for shared libraries in AIX

[Aditya Kamath]

[-- Attachment #1: Type: text/plain, Size: 3824 bytes --]

Hi Ulrich and community members,

>This is an interesting approach, which is quite different from what
>Linux is doing.   The one question/concern I have this: are these
>relocations present in the module where the TLS symbol is *defined*,
>or rather in the module where the TLS symbol is being *used*.  On
>Linux (ELF), it would be the latter - but I don't know all the AIX
>(XCOFF) details here.

>If it is the latter, then this approach has the drawback that the
>debugger can only access symbols that are also accessed somewhere
>within the program being debugged (in fact, with the current patch,
>only symbols that are used in the same module they are defined in).
>With the approach used on Linux, it is in principle possible to
>access TLS symbols whether they are used or not.  (If there's no
>better way to implement this, that of course may still a limitation
>that can be accepted.)

>Either way, it would be good to add test cases for the various
>scenarios.


Yes, we are on the same page here.

Let me take this opportunity to explain more and give more information.


# cat tls_lib.c


#include <stdio.h>


_Thread_local int lib_tls_var = 42;


int
lib_get_tls (void)
{
  return lib_tls_var;
}

void
lib_set_tls (int val)
{
  lib_tls_var = val;
}


Assume this is my C code that I am using to create tls_lib.so

The way I am computing TLS variable address currently is


dump -X64 -Hr tls_lib.so | grep "0x0021\|0x0024"
        0x110000528  0x00000018     0      0  0x003f    0x0021

The above gives me 0x110000528 - reloc type 0x24 (R_TLSM, global-dynamic) or 0x21 (R_TLS_IE, initial-exec) which is my static TOC slot address.


CORE_ADDR toc_addr = ldrel.l_vaddr + objfile->data_section_offset ();

The above line the patch is doing this where ldrel.l_vaddr =  0x110000528.


bash-5.3# dump -X64 -tv tls_lib.so | grep lib_get_tls
[16]    m   0x100000480     .text     1  extern                    .lib_get_tls
[20]    m   0x1100004f8     .data     1  extern                    lib_get_tls

This gives me static address lib_get_tls.


/home/aditya/binutils-gdb/gdb/gdb ./tls_test
Reading symbols from ./tls_test...
(gdb) break tls_main.c:36
Breakpoint 1 at 0x100000a78: file tls_main.c, line 36.
(gdb) r
Starting program: /home/aditya/tls_test/tls_test
main: lib_tls_var initial = 42
[New Thread 258 (tid 131072299) (id 2)]
[Switching to thread 2 (Thread 258 (tid 131072299))]

Thread 2 hit Breakpoint 1, thread_runner (arg=0x1) at tls_main.c:36
36    volatile int bp_here = 0; (void)bp_here;
(gdb) p/x &lib_get_tls
$1 = 0x900000000e42480

The above is the dynamic address of lib_get_tls () inside the shared library.


(gdb) x/1gx 0x110000528 + 0x900000000e42480 - 0x1100004f8
0x900000000e424b0 <lib_set_tls>:      0x9061fff48061fff4

So the objfile->data_section_offset () is helping me to get 0x900000000e42480 - 0x1100004f8 which is the offset to the data section.

So, the plan is to inspect the link time symbol table for the TOC entries associated with the TLS variable in its defining module. From there identify the load time address of the appropriate TOC entries and then use the values to perform TLS address calculations.

Unfortunately, compilers [GCC or clang] on AIX will not generate the TOC entries in a translation unit if the variable is unreferenced.

So, we cannot be in sync with the way Linux via ELF does. In Linux DTV lookup can be done for unreferenced variables. Even in stripped binaries TLS debugging will not work in AIX with this approach.

Until this is implemented in both compilers, I would like to propose this method so AIX users can debug thread local variables.

Let me know what the community thinks.

Have a nice day ahead.

Thanks and regards,
Aditya.



[-- Attachment #2: Type: text/html, Size: 12865 bytes --]