How to avoid stepping inside libpthread

How to avoid stepping inside libpthread

[Atsushi Nemoto]

[-- Attachment #1: Type: Text/Plain, Size: 2999 bytes --]

When I keep typing "next" command in debug session, gdb might try to
do single stepping inside libpthread.  And on atomic operations (LL/SC
loop, as I'm using Linux/MIPS), the single stepping never ends.

GNU gdb 6.6
...
This GDB was configured as "mips-linux"...
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) b func		# thread's entry point
Breakpoint 1 at 0x400700: file foo.c, line 6.
(gdb) run
Starting program: ./foo
[Thread debugging using libthread_db enabled]
[New Thread 715939536 (LWP 756)]
[New Thread 726549712 (LWP 759)]
[Switching to Thread 726549712 (LWP 759)]

Breakpoint 1, func (arg=0x0) at foo.c:6 
6               return arg;		/* last statement of the thread */
(gdb) n
7       }
(gdb) 
0x2ab0ae88 in start_thread () from /lib/libpthread.so.0
(gdb) 
Single stepping until exit from function start_thread, 
which has no line number information.

The gdb stops.  

There is a LL/SC loop (atomic_decrement_and_test(&__nptl_nthreads))
after the 0x2ab0ae88 in start_thread(), so it is not wonder the single
stepping never ends (SC always fail due to a breakpoint exception).


Are there any way to avoid falling into such situations?  Only I can
do is be careful to not enter "next" command inside libpthread?


Once I fell into this situation, C-c can might stop the gdb, but
sometimes it cause internal-error.

../../gdb-6.6/gdb/linux-nat.c:1333: internal-error: wait_lwp: Assertion `lp->status == 0' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n) 

I have not investigated this error further.


And current gdb snapshot (6.6.50.20070711) behaves differently.

GNU gdb 6.6.50.20070711
...
This GDB was configured as "mips-linux"...
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) b func
Breakpoint 1 at 0x400700: file foo.c, line 6.
(gdb) run
Starting program: ./foo
warning: Can not parse XML target description; XML support was disabled at compile time
[New LWP 754]
[Switching to LWP 754]

Breakpoint 1, func (arg=0x0) at foo.c:6
6               return arg;		/* last statement of the thread */
(gdb) 
7       }
(gdb) 
warning: GDB can't find the start of the function at 0x2ab0ae88.

    GDB is unable to find the start of the function at 0x2ab0ae88
and thus can't determine the size of that function's stack frame.
This means that GDB may be unable to access that stack frame, or
the frames below it.
    This problem is most likely caused by an invalid program counter or
stack pointer.
    However, if you think GDB should simply search farther back
from 0x2ab0ae88 for code which looks like the beginning of a
function, you can increase the range of the search using the `set
heuristic-fence-post' command.
warning: GDB can't find the start of the function at 0x2ab0ae87.
0x2ab0ae88 in ?? ()
(gdb) n
Cannot find bounds of current function


It seems current gdb snapshot can not find symbol for 0x2ab0ae88.  Is
this behavior intentional?

---
Atsushi Nemoto

[-- Attachment #2: foo.c --]
[-- Type: Text/Plain, Size: 228 bytes --]

#include <stdio.h>
#include <pthread.h>

void *func(void *arg)
{
	return arg;
}

int main(int argc, char **argv) 
{
	pthread_t id;
	pthread_create(&id, NULL, func, NULL);
	pthread_join(id, NULL);
	printf("done\n");
	return 0;
}

Re: How to avoid stepping inside libpthread

[Andreas Schwab]

Atsushi Nemoto <anemo@mba.ocn.ne.jp> writes:

> There is a LL/SC loop (atomic_decrement_and_test(&__nptl_nthreads))
> after the 0x2ab0ae88 in start_thread(), so it is not wonder the single
> stepping never ends (SC always fail due to a breakpoint exception).
>
>
> Are there any way to avoid falling into such situations?

See deal_with_atomic_sequence in rs6000-tdep.c, which tries to solve the
same issue.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: How to avoid stepping inside libpthread

[Daniel Jacobowitz]

On Wed, Jul 11, 2007 at 03:18:20PM +0900, Atsushi Nemoto wrote:
> When I keep typing "next" command in debug session, gdb might try to
> do single stepping inside libpthread.  And on atomic operations (LL/SC
> loop, as I'm using Linux/MIPS), the single stepping never ends.

Andreas answered this; we could fix it for MIPS, now.

> (gdb) 
> warning: GDB can't find the start of the function at 0x2ab0ae88.

I don't know why this happens, no.  Something must be wrong.  info
shared still works, right?


-- 
Daniel Jacobowitz
CodeSourcery

Re: How to avoid stepping inside libpthread

[Atsushi Nemoto]

On Wed, 11 Jul 2007 10:59:58 +0200, Andreas Schwab <schwab@suse.de> wrote:
> > There is a LL/SC loop (atomic_decrement_and_test(&__nptl_nthreads))
> > after the 0x2ab0ae88 in start_thread(), so it is not wonder the single
> > stepping never ends (SC always fail due to a breakpoint exception).
> >
> >
> > Are there any way to avoid falling into such situations?
> 
> See deal_with_atomic_sequence in rs6000-tdep.c, which tries to solve the
> same issue.

Thank you.  I'll try to implement the feature for mips.

---
Atsushi Nemoto

Re: How to avoid stepping inside libpthread

[Atsushi Nemoto]

On Thu, 12 Jul 2007 11:15:40 +0900 (JST), Atsushi Nemoto <anemo@mba.ocn.ne.jp> wrote:
> > See deal_with_atomic_sequence in rs6000-tdep.c, which tries to solve the
> > same issue.
> 
> Thank you.  I'll try to implement the feature for mips.

Thanks, it worked.  Here is a patch against gdb-6.6.

I'll port this patch to current gdb, but while it seems current gdb
cannot find libpthread symbol, another test code (or fixing gdb) would
be needed first ...

--- gdb-6.6.org/gdb/mips-tdep.c	2006-08-09 06:32:37.000000000 +0900
+++ gdb-6.6/gdb/mips-tdep.c	2007-07-12 14:11:46.000000000 +0900
@@ -2177,6 +2177,86 @@ mips_addr_bits_remove (CORE_ADDR addr)
     return addr;
 }
 
+/* Instruction masks used during single-stepping of atomic sequences.  */
+#define LLSC_MASK 0xfc000000
+#define LL_INSTRUCTION 0xc0000000
+#define LLD_INSTRUCTION 0xd0000000
+#define SC_INSTRUCTION 0xe0000000
+#define SCD_INSTRUCTION 0xf0000000
+
+/* Checks for an atomic sequence of instructions beginning with a LL/LLD
+   instruction and ending with a SC/SCD instruction.  If such a sequence
+   is found, attempt to step through it.  A breakpoint is placed at the end of 
+   the sequence.  */
+
+static int
+deal_with_atomic_sequence (CORE_ADDR pc)
+{
+  CORE_ADDR breaks[2] = {-1, -1};
+  CORE_ADDR loc = pc;
+  CORE_ADDR branch_bp; /* Breakpoint at branch instruction's destination.  */
+  int insn;
+  int insn_count;
+  int index;
+  int last_breakpoint = 0; /* Defaults to 0 (no breakpoints placed).  */  
+  const int atomic_sequence_length = 16; /* Instruction sequence length.  */
+
+  if (pc & 0x01)
+    return 0;
+
+  insn = mips_fetch_instruction (loc);
+  /* Assume all atomic sequences start with a ll/lld instruction.  */
+  if ((insn & LLSC_MASK) != LL_INSTRUCTION
+      && (insn & LLSC_MASK) != LLD_INSTRUCTION)
+    return 0;
+
+  /* Assume that no atomic sequence is longer than "atomic_sequence_length" 
+     instructions.  */
+  for (insn_count = 0; insn_count < atomic_sequence_length; ++insn_count)
+    {
+      loc += MIPS_INSN32_SIZE;
+      insn = mips_fetch_instruction (loc);
+
+      /* Assume that there is at most one branch in the atomic
+	 sequence.  If a branch is found, put a breakpoint in its
+	 destination address.  */
+      branch_bp = mips_next_pc (loc);
+      if (branch_bp != loc + MIPS_INSN32_SIZE)
+	{
+	  if (last_breakpoint >= 1)
+	    return 0; /* More than one branch found, fallback to the
+			 standard single-step code.  */
+	  breaks[1] = branch_bp;
+	  last_breakpoint++;
+	}
+
+      if ((insn & LLSC_MASK) == SC_INSTRUCTION
+	  || (insn & LLSC_MASK) == SCD_INSTRUCTION)
+	break;
+    }
+
+  /* Assume that the atomic sequence ends with a sc/scd instruction.  */
+  if ((insn & LLSC_MASK) != SC_INSTRUCTION
+      && (insn & LLSC_MASK) != SCD_INSTRUCTION)
+    return 0;
+
+  loc += MIPS_INSN32_SIZE;
+
+  /* Insert a breakpoint right after the end of the atomic sequence.  */
+  breaks[0] = loc;
+
+  /* Check for duplicated breakpoints.  Check also for a breakpoint
+     placed (branch instruction's destination) in the atomic sequence */
+  if (last_breakpoint && pc <= breaks[1] && breaks[1] <= breaks[0])
+    last_breakpoint = 0;
+
+  /* Effectively inserts the breakpoints.  */
+  for (index = 0; index <= last_breakpoint; index++)
+      insert_single_step_breakpoint (breaks[index]);
+
+  return 1;
+}
+
 /* mips_software_single_step() is called just before we want to resume
    the inferior, if we want to single-step it but there is no hardware
    or kernel single-step support (MIPS on GNU/Linux for example).  We find
@@ -2193,6 +2273,9 @@ mips_software_single_step (enum target_s
   if (insert_breakpoints_p)
     {
       pc = read_register (mips_regnum (current_gdbarch)->pc);
+      if (deal_with_atomic_sequence (pc))
+	return;
+
       next_pc = mips_next_pc (pc);
 
       insert_single_step_breakpoint (next_pc);

Re: How to avoid stepping inside libpthread

[Daniel Jacobowitz]

On Thu, Jul 12, 2007 at 03:30:49PM +0900, Atsushi Nemoto wrote:
> +      /* Assume that there is at most one branch in the atomic
> +	 sequence.  If a branch is found, put a breakpoint in its
> +	 destination address.  */
> +      branch_bp = mips_next_pc (loc);
> +      if (branch_bp != loc + MIPS_INSN32_SIZE)

That's not what you want to do.  mips_next_pc decides based on the
current registers whether each branch is taken or not taken, so this
will not detect untaken branches (and it's got the wrong set of
registers, since loc != current pc).  I think we'll have to make
mips_next_pc return whether a control flow changing instruction was
found.  Possibly whether an acceptable branch was found, to avoid
subroutine calls, if that's going to be a problem.

-- 
Daniel Jacobowitz
CodeSourcery

Re: How to avoid stepping inside libpthread

[Atsushi Nemoto]

On Thu, 12 Jul 2007 07:10:21 -0400, Daniel Jacobowitz <drow@false.org> wrote:
> > +      /* Assume that there is at most one branch in the atomic
> > +	 sequence.  If a branch is found, put a breakpoint in its
> > +	 destination address.  */
> > +      branch_bp = mips_next_pc (loc);
> > +      if (branch_bp != loc + MIPS_INSN32_SIZE)
> 
> That's not what you want to do.  mips_next_pc decides based on the
> current registers whether each branch is taken or not taken, so this
> will not detect untaken branches (and it's got the wrong set of
> registers, since loc != current pc).  I think we'll have to make
> mips_next_pc return whether a control flow changing instruction was
> found.  Possibly whether an acceptable branch was found, to avoid
> subroutine calls, if that's going to be a problem.

Oh yes, thank you for suggestion.  Here is an updated patch against
current snapshot.

--- gdb-6.6.50.20070711.org/gdb/mips-tdep.c	2007-06-19 02:45:26.000000000 +0900
+++ gdb-6.6.50.20070711/gdb/mips-tdep.c	2007-07-13 16:02:01.000000000 +0900
@@ -2293,6 +2293,111 @@ mips_addr_bits_remove (CORE_ADDR addr)
     return addr;
 }
 
+/* Instructions used during single-stepping of atomic sequences.  */
+#define LL_OPCODE 0x30
+#define LLD_OPCODE 0x34
+#define SC_OPCODE 0x38
+#define SCD_OPCODE 0x3c
+
+/* Checks for an atomic sequence of instructions beginning with a LL/LLD
+   instruction and ending with a SC/SCD instruction.  If such a sequence
+   is found, attempt to step through it.  A breakpoint is placed at the end of 
+   the sequence.  */
+
+static int
+deal_with_atomic_sequence (CORE_ADDR pc)
+{
+  CORE_ADDR breaks[2] = {-1, -1};
+  CORE_ADDR loc = pc;
+  CORE_ADDR branch_bp; /* Breakpoint at branch instruction's destination.  */
+  unsigned long insn;
+  int insn_count;
+  int index;
+  int last_breakpoint = 0; /* Defaults to 0 (no breakpoints placed).  */  
+  const int atomic_sequence_length = 16; /* Instruction sequence length.  */
+
+  if (pc & 0x01)
+    return 0;
+
+  insn = mips_fetch_instruction (loc);
+  /* Assume all atomic sequences start with a ll/lld instruction.  */
+  if (itype_op (insn) != LL_OPCODE && itype_op (insn) != LLD_OPCODE)
+    return 0;
+
+  /* Assume that no atomic sequence is longer than "atomic_sequence_length" 
+     instructions.  */
+  for (insn_count = 0; insn_count < atomic_sequence_length; ++insn_count)
+    {
+      int is_branch = 0;
+      loc += MIPS_INSN32_SIZE;
+      insn = mips_fetch_instruction (loc);
+
+      /* Assume that there is at most one branch in the atomic
+	 sequence.  If a branch is found, put a breakpoint in its
+	 destination address.  */
+      switch (itype_op (insn))
+	{
+	case 0: /* SPECIAL */
+	  if (rtype_funct (insn) >> 1 == 4) /* JR, JALR */
+	    return 0; /* fallback to the standard single-step code. */
+	  break;
+	case 1: /* REGIMM */
+	  is_branch = ((itype_rt (insn) & 0xc0) == 0); /* B{LT,GE}Z* */
+	  break;
+	case 2: /* J */
+	case 3: /* JAL */
+	  return 0; /* fallback to the standard single-step code. */
+	case 4: /* BEQ */
+	case 5: /* BNE */
+	case 6: /* BLEZ */
+	case 7: /* BGTZ */
+	case 20: /* BEQL */
+	case 21: /* BNEL */
+	case 22: /* BLEZL */
+	case 23: /* BGTTL */
+	  is_branch = 1;
+	  break;
+	case 17: /* COP1 */
+	case 18: /* COP2 */
+	case 19: /* COP3 */
+	  is_branch = (itype_rs (insn) == 8); /* BCzF, BCzFL, BCzT, BCzTL */
+	  break;
+	}
+      if (is_branch)
+	{
+	  branch_bp = loc + mips32_relative_offset (insn) + 4;
+	  if (last_breakpoint >= 1)
+	    return 0; /* More than one branch found, fallback to the
+			 standard single-step code.  */
+	  breaks[1] = branch_bp;
+	  last_breakpoint++;
+	}
+
+      if (itype_op (insn) == SC_OPCODE || itype_op (insn) == SCD_OPCODE)
+	break;
+    }
+
+  /* Assume that the atomic sequence ends with a sc/scd instruction.  */
+  if (itype_op (insn) != SC_OPCODE && itype_op (insn) != SCD_OPCODE)
+    return 0;
+
+  loc += MIPS_INSN32_SIZE;
+
+  /* Insert a breakpoint right after the end of the atomic sequence.  */
+  breaks[0] = loc;
+
+  /* Check for duplicated breakpoints.  Check also for a breakpoint
+     placed (branch instruction's destination) in the atomic sequence */
+  if (last_breakpoint && pc <= breaks[1] && breaks[1] <= breaks[0])
+    last_breakpoint = 0;
+
+  /* Effectively inserts the breakpoints.  */
+  for (index = 0; index <= last_breakpoint; index++)
+      insert_single_step_breakpoint (breaks[index]);
+
+  return 1;
+}
+
 /* mips_software_single_step() is called just before we want to resume
    the inferior, if we want to single-step it but there is no hardware
    or kernel single-step support (MIPS on GNU/Linux for example).  We find
@@ -2304,6 +2409,9 @@ mips_software_single_step (struct frame_
   CORE_ADDR pc, next_pc;
 
   pc = get_frame_pc (frame);
+  if (deal_with_atomic_sequence (pc))
+    return 1;
+
   next_pc = mips_next_pc (frame, pc);
 
   insert_single_step_breakpoint (next_pc);

Re: How to avoid stepping inside libpthread

[Daniel Jacobowitz]

On Fri, Jul 13, 2007 at 04:03:41PM +0900, Atsushi Nemoto wrote:
> Oh yes, thank you for suggestion.  Here is an updated patch against
> current snapshot.

Hi Atsushi,

I'm sorry I never got back to you about this.  I tried your patch
today and it seems to work.  But do you have an FSF copyright
assignment, or are you planning on getting one?  This is too
substantial for us to merge it without one.

-- 
Daniel Jacobowitz
CodeSourcery