[RFHelp] Linux, threads, and regsets in thread_db

[RFHelp] Linux, threads, and regsets in thread_db

[Daniel Jacobowitz]

I spent a while this summer making thread debugging work on some more
non-x86 targets, with great frustration.  Eventually I reached the
decision that my problem was in fact a glibc bug; the attached messages
explain it.  Ulrich has categorically (and without explanation, when
asked.  sigh.) refused the patch.

Could someone on this list at least sanity check my patch?  I have a
(disgusting) workaround for GDB if the header is not changed, but I'd
much rather someone convince Ulrich that I'm not crazy.  Especially
since I can not get the workaround to compile without warnings.  C
behaves badly in the presence of casts to/from array types.


----- Forwarded message from Daniel Jacobowitz <dmj+@andrew.cmu.edu> -----

Date: Mon, 18 Jun 2001 12:06:08 -0700
From: Daniel Jacobowitz <dmj+@andrew.cmu.edu>
Subject: gregset vs prgregset vs elf_gregset in thread_db
To: libc-alpha@sources.redhat.com
Mail-Followup-To: libc-alpha@sources.redhat.com

The thread-db routines are defined to take arguments of type 'prgregset_t'. 
On about half the supported architectures, prgregset_t is typedef'd to be
'gregset_t'; on the other half it is 'elf_gregset_t'.  I don't know where
the gregset_t definitions came from, especially on platforms that have
always been ELF, but they are substantially less useful than their elf
counterparts - what we really want to fill in is the data we can get from
ptrace, and pt_regs is usually fairly similar to an elf_gregset_t and an
elf_fpregset_t.

For instance, on PowerPC the NIP is not in any of r0 - r31, and a gregset_t
only has 32 elements.  No way to convey the thread's NIP, which makes
debugging all but meaningless.

The only consumer of the prgregset_t and prfpregset_t types is thread-db. 
Is there any objection to changing them to the elf_* variants, and possibly
bumping the thread-db soname?  Although I'm not sure the latter is even
necessary; gdb is the only consumer of thread-db I've ever found on linux,
and it doesn't work with threads on the architectures affected yet anyway.

-- 
Daniel Jacobowitz                           Debian GNU/Linux Developer
Monta Vista Software                              Debian Security Team


----- End forwarded message -----
----- Forwarded message from Daniel Jacobowitz <dmj+@andrew.cmu.edu> -----

Date: Tue, 26 Jun 2001 22:53:23 -0700
From: Daniel Jacobowitz <dmj+@andrew.cmu.edu>
Subject: Re: gregset vs prgregset vs elf_gregset in thread_db
To: Geoff Keating <geoffk@redhat.com>
Cc: libc-alpha@sources.redhat.com
Mail-Followup-To: Geoff Keating <geoffk@redhat.com>,
	libc-alpha@sources.redhat.com

On Tue, Jun 26, 2001 at 09:59:32AM -0700, Geoff Keating wrote:
> > Date: Mon, 25 Jun 2001 22:15:37 -0700
> > From: Daniel Jacobowitz <dmj+@andrew.cmu.edu>
> 
> > On Mon, Jun 25, 2001 at 10:25:48PM -0700, Geoff Keating wrote:
> > > I don't believe anything other than gdb uses these structures.  They
> > > don't reflect anything that the kernel produces, do they?
> > 
> > That's right.  They're solely an interface for debuggers, and I do not
> > know of any Linux debugger other than gdb which uses them, so I figure
> > changing them to be more convenient to gdb would be a reasonable thing.
> 
> That's what I'd thought.  In that case, they can be changed in any way
> that gdb finds convenient.

In that case, is this patch OK?

2001-06-26  Daniel Jacobowitz  <dan@debian.org>
	* sysdeps/unix/sysv/linux/alpha/sys/procfs.h: Change prgregset_t
	and prfpregset_t to match their elf counterparts.
	* sysdeps/unix/sysv/linux/ia64/sys/procfs.h: Likewise.
	* sysdeps/unix/sysv/linux/mips/sys/procfs.h: Likewise.
	* sysdeps/unix/sysv/linux/sys/procfs.h: Likewise.

Index: sysdeps/unix/sysv/linux/alpha/sys/procfs.h
===================================================================
RCS file: /cvs/glibc/libc/sysdeps/unix/sysv/linux/alpha/sys/procfs.h,v
retrieving revision 1.7
diff -u -r1.7 procfs.h
--- procfs.h	1999/11/18 00:11:49	1.7
+++ procfs.h	2001/06/27 05:48:39
@@ -99,8 +99,8 @@
 typedef void *psaddr_t;
 
 /* Register sets.  Linux has different names.  */
-typedef gregset_t prgregset_t;
-typedef fpregset_t prfpregset_t;
+typedef elf_gregset_t prgregset_t;
+typedef elf_fpregset_t prfpregset_t;
 
 /* We don't have any differences between processes and threads,
    therefore habe only ine PID type.  */
Index: sysdeps/unix/sysv/linux/ia64/sys/procfs.h
===================================================================
RCS file: /cvs/glibc/libc/sysdeps/unix/sysv/linux/ia64/sys/procfs.h,v
retrieving revision 1.1
diff -u -r1.1 procfs.h
--- procfs.h	2000/06/20 03:34:47	1.1
+++ procfs.h	2001/06/27 05:48:40
@@ -104,8 +104,8 @@
 typedef void *psaddr_t;
 
 /* Register sets.  Linux has different names.  */
-typedef gregset_t prgregset_t;
-typedef fpregset_t prfpregset_t;
+typedef elf_gregset_t prgregset_t;
+typedef elf_fpregset_t prfpregset_t;
 
 /* We don't have any differences between processes and threads,
    therefore habe only ine PID type.  */
Index: sysdeps/unix/sysv/linux/mips/sys/procfs.h
===================================================================
RCS file: /cvs/glibc/libc/sysdeps/unix/sysv/linux/mips/sys/procfs.h,v
retrieving revision 1.6
diff -u -r1.6 procfs.h
--- procfs.h	2000/01/10 14:27:50	1.6
+++ procfs.h	2001/06/27 05:48:40
@@ -101,8 +101,8 @@
 typedef void *psaddr_t;
 
 /* Register sets.  Linux has different names.  */
-typedef gregset_t prgregset_t;
-typedef fpregset_t prfpregset_t;
+typedef elf_gregset_t prgregset_t;
+typedef elf_fpregset_t prfpregset_t;
 
 /* We don't have any differences between processes and threads,
    therefore habe only ine PID type.  */
Index: sysdeps/unix/sysv/linux/sys/procfs.h
===================================================================
RCS file: /cvs/glibc/libc/sysdeps/unix/sysv/linux/sys/procfs.h,v
retrieving revision 1.11
diff -u -r1.11 procfs.h
--- procfs.h	1999/11/15 07:59:41	1.11
+++ procfs.h	2001/06/27 05:48:40
@@ -99,8 +99,8 @@
 typedef void *psaddr_t;
 
 /* Register sets.  Linux has different names.  */
-typedef gregset_t prgregset_t;
-typedef fpregset_t prfpregset_t;
+typedef elf_gregset_t prgregset_t;
+typedef elf_fpregset_t prfpregset_t;
 
 /* We don't have any differences between processes and threads,
    therefore habe only ine PID type.  */


-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer


----- End forwarded message -----
----- Forwarded message from Ulrich Drepper <drepper@redhat.com> -----

Date: 14 Sep 2001 19:56:47 -0700
From: Ulrich Drepper <drepper@redhat.com>
Subject: Re: gregset vs prgregset vs elf_gregset in thread_db
To: Daniel Jacobowitz <drow@mvista.com>
Cc: Geoff Keating <geoffk@redhat.com>, libc-alpha@sources.redhat.com
Reply-To: drepper@cygnus.com (Ulrich Drepper)

Daniel Jacobowitz <drow@mvista.com> writes:

> I posted this patch three months ago and no one objected; could it
> please be applied?

There is absolutely no reason to apply this.

-- 
---------------.                          ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Red Hat          `--' drepper at redhat.com   `------------------------


----- End forwarded message -----

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: [RFHelp] Linux, threads, and regsets in thread_db

[Mark Kettenis]

Daniel Jacobowitz <drow@mvista.com> writes:

> I spent a while this summer making thread debugging work on some more
> non-x86 targets, with great frustration.  Eventually I reached the
> decision that my problem was in fact a glibc bug; the attached messages
> explain it.  Ulrich has categorically (and without explanation, when
> asked.  sigh.) refused the patch.

Seems that Ulrich is still confused about the regset issues.  I asked
the port maintainers to clean this up when I fixed it for i386.  Jakub
Jellinek cleaned up sparc, but nothing else happened.

> Could someone on this list at least sanity check my patch?  I have a
> (disgusting) workaround for GDB if the header is not changed, but I'd
> much rather someone convince Ulrich that I'm not crazy.  Especially
> since I can not get the workaround to compile without warnings.  C
> behaves badly in the presence of casts to/from array types.

In principle, your patch is the right thing.  There may be
interactions with <sys/user.h> and <sys/elf.h> though.  Look at the
i386 port to see how things should be done.

Mark

PATCH: Re: [RFHelp] Linux, threads, and regsets in thread_db

[H . J . Lu]

On Wed, Oct 03, 2001 at 09:10:07PM -0400, Daniel Jacobowitz wrote:
> > > 
> > 
> > How far have you gone on Linux/mips? I know gdb 5.1 doesn't work with
> > threads on Linux/mips. Does your change fix it? I'd like to clean up
> > glibc for mips on this.
> 
> Yes.  Debugging threads works reasonably well.  I'm going to do more
> work about this in the next couple of months too, I think; after I go
> back to gdbserver for a little while.

I am still having problems with threads on mips. I cannot set break
point after target has run:

(gdb) run
...
(gdb) b main
reading register sp (#29): No such process.

The problem is gdb tries to read sp register in thread mode. This patch
seems to work for me.

Another problem is when I set a break point in a thread function, only
the first thread stops. I got

(gdb) b process
Breakpoint 1 at 0x400a38: file Examples/ex1.c, line 14.
(gdb) r
Starting program: /export/build/gnu/glibc/build-mipsel-linux/linuxthreads/ex1 
[New Thread 1024 (LWP 26552)]
[New Thread 2049 (LWP 26569)]
[New Thread 1026 (LWP 26570)]
[Switching to Thread 1026 (LWP 26570)]

Breakpoint 1, process (arg=0x1000) at Examples/ex1.c:14
14        fprintf (stderr, "Starting process %s\n", (char *) arg);
...
Program received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 1024 (LWP 26552)]
0x2ab8de24 in __syscall_rt_sigsuspend () at soinit.c:56
56      soinit.c: No such file or directory.

But it may be a kernel bug.


H.J.
---
2001-11-14  H.J. Lu  (hjl@gnu.org)

	* mips-tdep.c (heuristic_proc_desc): Read register only when
	target is running.

--- gdb/mips-tdep.c.break	Mon Aug  6 17:13:24 2001
+++ gdb/mips-tdep.c	Wed Nov 14 13:28:17 2001
@@ -1860,7 +1860,12 @@ static mips_extra_func_info_t
 heuristic_proc_desc (CORE_ADDR start_pc, CORE_ADDR limit_pc,
 		     struct frame_info *next_frame)
 {
-  CORE_ADDR sp = read_next_frame_reg (next_frame, SP_REGNUM);
+  CORE_ADDR sp;
+  
+  if (target_has_execution)
+    sp = read_next_frame_reg (next_frame, SP_REGNUM);
+  else
+    sp = 0;
 
   if (start_pc == 0)
     return NULL;

Re: PATCH: Re: [RFHelp] Linux, threads, and regsets in thread_db

[Daniel Jacobowitz]

On Wed, Nov 14, 2001 at 01:45:21PM -0800, H . J . Lu wrote:
> On Wed, Oct 03, 2001 at 09:10:07PM -0400, Daniel Jacobowitz wrote:
> > > > 
> > > 
> > > How far have you gone on Linux/mips? I know gdb 5.1 doesn't work with
> > > threads on Linux/mips. Does your change fix it? I'd like to clean up
> > > glibc for mips on this.
> > 
> > Yes.  Debugging threads works reasonably well.  I'm going to do more
> > work about this in the next couple of months too, I think; after I go
> > back to gdbserver for a little while.
> 
> I am still having problems with threads on mips. I cannot set break
> point after target has run:
> 
> (gdb) run
> ...
> (gdb) b main
> reading register sp (#29): No such process.
> 
> The problem is gdb tries to read sp register in thread mode. This patch
> seems to work for me.

This should not be a problem.  It works fine for me.  I'll step through
it later tonight and try to figure out what the difference is from what
you're seeing.

> Another problem is when I set a break point in a thread function, only
> the first thread stops. I got
> 
> (gdb) b process
> Breakpoint 1 at 0x400a38: file Examples/ex1.c, line 14.
> (gdb) r
> Starting program: /export/build/gnu/glibc/build-mipsel-linux/linuxthreads/ex1 
> [New Thread 1024 (LWP 26552)]
> [New Thread 2049 (LWP 26569)]
> [New Thread 1026 (LWP 26570)]
> [Switching to Thread 1026 (LWP 26570)]
> 
> Breakpoint 1, process (arg=0x1000) at Examples/ex1.c:14
> 14        fprintf (stderr, "Starting process %s\n", (char *) arg);
> ...
> Program received signal SIGTRAP, Trace/breakpoint trap.
> [Switching to Thread 1024 (LWP 26552)]
> 0x2ab8de24 in __syscall_rt_sigsuspend () at soinit.c:56
> 56      soinit.c: No such file or directory.
> 
> But it may be a kernel bug.

Something is suspicious here.  The process in sigsuspend should have
been stopped with SIGSTOP, not SIGTRAP...  is this an example program
from glibc?

> H.J.
> ---
> 2001-11-14  H.J. Lu  (hjl@gnu.org)
> 
> 	* mips-tdep.c (heuristic_proc_desc): Read register only when
> 	target is running.

I can't see why this changes behavior in the way you're seeing.  I'm
pretty sure it isn't right.


-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: PATCH: Re: [RFHelp] Linux, threads, and regsets in thread_db

[H . J . Lu]

On Thu, Nov 15, 2001 at 11:48:41AM -0500, Daniel Jacobowitz wrote:
> On Wed, Nov 14, 2001 at 01:45:21PM -0800, H . J . Lu wrote:
> > On Wed, Oct 03, 2001 at 09:10:07PM -0400, Daniel Jacobowitz wrote:
> > > > > 
> > > > 
> > > > How far have you gone on Linux/mips? I know gdb 5.1 doesn't work with
> > > > threads on Linux/mips. Does your change fix it? I'd like to clean up
> > > > glibc for mips on this.
> > > 
> > > Yes.  Debugging threads works reasonably well.  I'm going to do more
> > > work about this in the next couple of months too, I think; after I go
> > > back to gdbserver for a little while.
> > 
> > I am still having problems with threads on mips. I cannot set break
> > point after target has run:
> > 
> > (gdb) run
> > ...
> > (gdb) b main
> > reading register sp (#29): No such process.
> > 
> > The problem is gdb tries to read sp register in thread mode. This patch
> > seems to work for me.
> 
> This should not be a problem.  It works fine for me.  I'll step through
> it later tonight and try to figure out what the difference is from what
> you're seeing.

In my case, gdb in thread mode will cal "fetch_register" in infptrace.c.
Since nothing is running, I got

reading register sp (#29): No such process.

I don't know why it works for you.

> 
> > Another problem is when I set a break point in a thread function, only
> > the first thread stops. I got
> > 
> > (gdb) b process
> > Breakpoint 1 at 0x400a38: file Examples/ex1.c, line 14.
> > (gdb) r
> > Starting program: /export/build/gnu/glibc/build-mipsel-linux/linuxthreads/ex1 
> > [New Thread 1024 (LWP 26552)]
> > [New Thread 2049 (LWP 26569)]
> > [New Thread 1026 (LWP 26570)]
> > [Switching to Thread 1026 (LWP 26570)]
> > 
> > Breakpoint 1, process (arg=0x1000) at Examples/ex1.c:14
> > 14        fprintf (stderr, "Starting process %s\n", (char *) arg);
> > ...
> > Program received signal SIGTRAP, Trace/breakpoint trap.
> > [Switching to Thread 1024 (LWP 26552)]
> > 0x2ab8de24 in __syscall_rt_sigsuspend () at soinit.c:56
> > 56      soinit.c: No such file or directory.
> > 
> > But it may be a kernel bug.
> 
> Something is suspicious here.  The process in sigsuspend should have
> been stopped with SIGSTOP, not SIGTRAP...  is this an example program
> from glibc?

Yes, linuxthreads/ex1 from glibc.

> 
> > H.J.
> > ---
> > 2001-11-14  H.J. Lu  (hjl@gnu.org)
> > 
> > 	* mips-tdep.c (heuristic_proc_desc): Read register only when
> > 	target is running.
> 
> I can't see why this changes behavior in the way you're seeing.  I'm

Why do you think that? Why does anyone want to read a register when
nothing is running?

> pretty sure it isn't right.



H.J.

Re: PATCH: Re: [RFHelp] Linux, threads, and regsets in thread_db

[Daniel Jacobowitz]

On Thu, Nov 15, 2001 at 08:55:08AM -0800, H . J . Lu wrote:
> In my case, gdb in thread mode will cal "fetch_register" in infptrace.c.
> Since nothing is running, I got
> 
> reading register sp (#29): No such process.
> 
> I don't know why it works for you.

OK, I misunderstood what you meant by "after the process has run".  I
see the problem.  I'll send another patch that I think is more accurate
later today, once I test it a bit.

> > > Another problem is when I set a break point in a thread function, only
> > > the first thread stops. I got
> > > 
> > > (gdb) b process
> > > Breakpoint 1 at 0x400a38: file Examples/ex1.c, line 14.
> > > (gdb) r
> > > Starting program: /export/build/gnu/glibc/build-mipsel-linux/linuxthreads/ex1 
> > > [New Thread 1024 (LWP 26552)]
> > > [New Thread 2049 (LWP 26569)]
> > > [New Thread 1026 (LWP 26570)]
> > > [Switching to Thread 1026 (LWP 26570)]
> > > 
> > > Breakpoint 1, process (arg=0x1000) at Examples/ex1.c:14
> > > 14        fprintf (stderr, "Starting process %s\n", (char *) arg);
> > > ...
> > > Program received signal SIGTRAP, Trace/breakpoint trap.
> > > [Switching to Thread 1024 (LWP 26552)]
> > > 0x2ab8de24 in __syscall_rt_sigsuspend () at soinit.c:56
> > > 56      soinit.c: No such file or directory.
> > > 
> > > But it may be a kernel bug.
> > 
> > Something is suspicious here.  The process in sigsuspend should have
> > been stopped with SIGSTOP, not SIGTRAP...  is this an example program
> > from glibc?
> 
> Yes, linuxthreads/ex1 from glibc.

OK, I can reproduce this too.  This has something to do with
mishandling thread exit.

> > > 2001-11-14  H.J. Lu  (hjl@gnu.org)
> > > 
> > > 	* mips-tdep.c (heuristic_proc_desc): Read register only when
> > > 	target is running.
> > 
> > I can't see why this changes behavior in the way you're seeing.  I'm
> 
> Why do you think that? Why does anyone want to read a register when
> nothing is running?

We never want to read that register from after_prologue;  the current
stack pointer is not relevant.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer