gdb-6.5 produces infinite backtrace on ARM

gdb-6.5 produces infinite backtrace on ARM

[Zarges, Olav]

Hi all,

Environment:
cross gdb-6.5 running on cygwin, gcc-3.4.4-glibc-2.3.5 for ARM,
Eclipse and CDT as a front end.

When debugging a multithreaded application on ARM I get inifinite
backtrace showing pthread_start_thread(). When using Eclipse as a
GDB front-end I had to use "set backtrace limit" to limit the
backtrace to a sensible value, otherwise the Eclipse debugger hangs
forever when the program hits a breakpoint.

While the thread with the breakpoint has inifinite backtrace the
other thread shows "Previous frame identical to this frame (corrupt
stack?)" (Eclipse doesn't show a call stack at all; command line
GDB shows the frames up to this message).

Reading the newsgroups I found that many people had similar problems
(is there a solution available?).

I found the hint to use DWARF CFI to set the function called by
pthread_create() or maybe pthread_create() itself to non-unwindable.
Can anybody tell me how to do it for this sample program?

Regards Olav

>>>>

compiler-run:
arm-softfloat-linux-gnu-g++ -O0 -g3 -Wall -c -fmessage-length=0
-MMD -MP -MF"threads.d" -MT"threads.d" -o"threads.o" "../threads.cpp"

linker-run:
arm-softfloat-linux-gnu-g++  -o"threads"  ./threads.o   -lpthread

>>>>

#include <pthread.h>
#include <stdio.h>
#include <unistd.h>


void* thread_function1(void*) {
    int a = 0;

    while (1) {
        for (int i=0 ; i<100; i++) {
            a += i;
        }
        printf("Thread 1: %d\n", a);
        sleep(3);
    }
    return NULL;
}

void* thread_function2(void*) {
  int a = 0;

  while (1) {
      for (int i=0 ; i<100; i++) {
        a += i;
      }
      printf("Thread 2: %d\n", a);
      sleep(2);
  }
  return NULL;
}

int main(int argc, char **argv) {
  pthread_t thread1, thread2;

  // create all the threads
  pthread_create( &thread1, NULL, thread_function1, NULL );
  pthread_create( &thread2, NULL, thread_function2, NULL );
  sleep(1);

  // wait for threads to end (.. I know this can't happen)
  pthread_join( thread1, NULL );
  pthread_join( thread2, NULL );

  return 0;
}







-- 
Olav A. Zarges

imc Meßsysteme GmbH
Voltastr. 5
13355 Berlin
Deutschland
Tel: 030 / 467090-68
Fax: 030 / 4631576

Re: gdb-6.5 produces infinite backtrace on ARM

[Daniel Jacobowitz]

On Tue, Aug 15, 2006 at 10:12:14AM +0200, Zarges, Olav wrote:
> Hi all,
> 
> Environment:
> cross gdb-6.5 running on cygwin, gcc-3.4.4-glibc-2.3.5 for ARM,
> Eclipse and CDT as a front end.
> 
> When debugging a multithreaded application on ARM I get inifinite
> backtrace showing pthread_start_thread(). When using Eclipse as a
> GDB front-end I had to use "set backtrace limit" to limit the
> backtrace to a sensible value, otherwise the Eclipse debugger hangs
> forever when the program hits a breakpoint.

Normally you should get an ugly end of the backtrace, not an infinite
one.  If you'd like, I can take a look at what's gone wrong.  I would
need enough to reproduce the problem - which probably means a tarball
including the source, compiled executable, and shared libraries which
it uses, since this may be specific to something about your build
environment.  If you want to do that, let me know; it's too big to post
to the list.

> I found the hint to use DWARF CFI to set the function called by
> pthread_create() or maybe pthread_create() itself to non-unwindable.
> Can anybody tell me how to do it for this sample program?

You have to modify the glibc source to do it, which is unpleasant.
Maybe we need to take stronger measures for dealing with the pthread
start functions, as Joel proposed some time ago...

-- 
Daniel Jacobowitz
CodeSourcery

Re: gdb-6.5 produces infinite backtrace on ARM

[Zarges, Olav]

Daniel Jacobowitz wrote:

> [ ... ]
> environment.  If you want to do that, let me know; it's too big to post
> to the list.

Yes, I would like to do that. I created a tar.gz-file with the necessary files.
The size is about 1MB.

> [ ... ]
> You have to modify the glibc source to do it, which is unpleasant.
> Maybe we need to take stronger measures for dealing with the pthread
> start functions, as Joel proposed some time ago...


If that's possible it would be great.

Regards Olav

Re: gdb-6.5 produces infinite backtrace on ARM

[Daniel Jacobowitz]

On Tue, Aug 15, 2006 at 08:40:54AM -0400, Daniel Jacobowitz wrote:
> Normally you should get an ugly end of the backtrace, not an infinite
> one.  If you'd like, I can take a look at what's gone wrong.  I would
> need enough to reproduce the problem - which probably means a tarball
> including the source, compiled executable, and shared libraries which
> it uses, since this may be specific to something about your build
> environment.  If you want to do that, let me know; it's too big to post
> to the list.

Thanks a lot for the test case.  The underlying cause was that
pthread_start_thread is marked noreturn; GCC seems to have omitted
the LR register save in this case.  I thought it wouldn't do that,
but in any case, we should cope.

I know have a fairly general patch set for this problem, which
produces:

(gdb) bt
...
#4 0xxxxxxxx in pthread_start_thread () from /lib/libpthread.so.0
Backtrace stopped: frame did not save the PC
(gdb)

This isn't ideal - we could detect the pthread_start_thread function
name and stop automatically, which might be a wise addition - but it's
better than going off into the woods.

There's about 250 lines of changes involved, to one of the more
complicated parts of GDB, so I will need to go over the patches again
and post them separately.  But I'll try to make sure this is fixed
soon.

-- 
Daniel Jacobowitz
CodeSourcery

Re: gdb-6.5 produces infinite backtrace on ARM

[Zarges, Olav]

Daniel Jacobowitz wrote:
> [ ... ]
> I know have a fairly general patch set for this problem, which
> produces:
> 
> (gdb) bt
> ...
> #4 0xxxxxxxx in pthread_start_thread () from /lib/libpthread.so.0
> Backtrace stopped: frame did not save the PC
> (gdb)
> 
> This isn't ideal - we could detect the pthread_start_thread function
> name and stop automatically, which might be a wise addition - but it's
> better than going off into the woods.
> 
> There's about 250 lines of changes involved, to one of the more
> complicated parts of GDB, so I will need to go over the patches again
> and post them separately.  But I'll try to make sure this is fixed
> soon.


Sounds pretty good. Good job! I can't wait to lay my hands on the patch...

Another question arose from you answer: what is a gdb/mi front-end supposed
to do when a message like

	"Backtrace stopped: frame did not save the PC" or
	"Previous frame identical to this frame (corrupt stack?)" or
	...

is returned at the end of the backtrace? Eclipse just throws away the
complete backtrace for the corresponding thread.

Re: gdb-6.5 produces infinite backtrace on ARM

[Daniel Jacobowitz]

On Mon, Aug 21, 2006 at 01:32:04PM +0200, Zarges, Olav wrote:
> Sounds pretty good. Good job! I can't wait to lay my hands on the patch...
> 
> Another question arose from you answer: what is a gdb/mi front-end supposed
> to do when a message like
> 
> 	"Backtrace stopped: frame did not save the PC" or
> 	"Previous frame identical to this frame (corrupt stack?)" or
> 	...
> 
> is returned at the end of the backtrace? Eclipse just throws away the
> complete backtrace for the corresponding thread.

What other versions of GDB are you looking at?  With older versions, it was an
^error; with newer it's a ~"console output message".  I would not
expect Eclipse to throw away the backtrace if there's a console output
message.

-- 
Daniel Jacobowitz
CodeSourcery

Re: gdb-6.5 produces infinite backtrace on ARM

[Zarges, Olav]

[-- Attachment #1: Type: text/plain, Size: 1248 bytes --]



Daniel Jacobowitz wrote:
> On Mon, Aug 21, 2006 at 01:32:04PM +0200, Zarges, Olav wrote:
>> Sounds pretty good. Good job! I can't wait to lay my hands on the patch...
>>
>> Another question arose from you answer: what is a gdb/mi front-end supposed
>> to do when a message like
>>
>> 	"Backtrace stopped: frame did not save the PC" or
>> 	"Previous frame identical to this frame (corrupt stack?)" or
>> 	...
>>
>> is returned at the end of the backtrace? Eclipse just throws away the
>> complete backtrace for the corresponding thread.
> 
> What other versions of GDB are you looking at?  With older versions, it was an
> ^error; with newer it's a ~"console output message".  I would not
> expect Eclipse to throw away the backtrace if there's a console output
> message.

I just verified it with gdb-6.3 which doesn't show any backtrace at all when
setting backtrace limit (--> Backtrace limit of 20 exceeded, see dump-6.3.txt).

The current version gdb-6.5 shows backtrace and the message "Previous frame
identical to this frame (corrupt stack?)" (see dump-6.5.txt). Eclipse then
fails to show any bt info at all for this thread (I can supply a screenshot
of this behaviour, 109 KByte PNG).

btw: I'm using the latest Eclipse and CDT release.


[-- Attachment #2: dump-6.5.txt --]
[-- Type: text/plain, Size: 7864 bytes --]

(gdb) t a all bt

Thread 4 (thread 32771):
#0  0x402a319c in nanosleep ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libc.so.6
#1  0x402a3078 in sleep ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libc.so.6
#2  0x00008784 in f2 () at ../threads.cpp:31
#3  0x000087a4 in thread_function2 () at ../threads.cpp:36
#4  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#5  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#6  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#7  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#8  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#9  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#10 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#11 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
---Type <return> to continue, or q <return> to quit---
#12 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#13 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#14 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#15 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#16 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#17 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#18 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#19 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0

Thread 3 (thread 32769):
#0  0x402c8d60 in poll ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libc.so.6
#1  0x40024114 in __pthread_manager ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#2  0x40024304 in __pthread_manager_event ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-lin-ux-gnu/lib/libpthread.so.0
--Type <return> to continue, or q <return> to quit---
#3  0x40024304 in __pthread_manager_event ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
Previous frame identical to this frame (corrupt stack?)

Thread 2 (thread 16386):
#0  f1 () at ../threads.cpp:13
#1  0x0000870c in thread_function1 () at ../threads.cpp:18
#2  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#3  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#4  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#5  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#6  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#7  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#8  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#9  0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#10 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
--Type <return> to continue, or q <return> to quit---
#11 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#12 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#13 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#14 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#15 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#16 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#17 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#18 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#19 0x40024410 in pthread_start_thread ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0

Thread 1 (thread 16384):
#0  0x40027148 in __pthread_sigsuspend ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#1  0x40026fcc in __pthread_wait_for_restart_signal ()
---Type <return> to continue, or q <return> to quit---
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#2  0x40023cfc in pthread_join ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libpthread.so.0
#3  0x00008818 in main (argc=1, argv=0xbeb71ea4) at ../threads.cpp:50
13              sleep(3);
(gdb)

[-- Attachment #3: dump-6.3.txt --]
[-- Type: text/plain, Size: 318 bytes --]

(gdb) t a all bt
Backtrace limit of 20 exceeded

Thread 4 (Thread 32771):
#-1 0x402a319c in nanosleep ()
   from /opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libc.so.6
Backtrace limit of 20 exceeded
Backtrace limit of 20 exceeded
13              sleep(3);
(gdb)

Re: gdb-6.5 produces infinite backtrace on ARM

[Daniel Jacobowitz]

On Mon, Aug 21, 2006 at 04:34:34PM +0200, Zarges, Olav wrote:
> The current version gdb-6.5 shows backtrace and the message "Previous frame
> identical to this frame (corrupt stack?)" (see dump-6.5.txt). Eclipse then
> fails to show any bt info at all for this thread (I can supply a screenshot
> of this behaviour, 109 KByte PNG).
> 
> btw: I'm using the latest Eclipse and CDT release.

This is very strange.  I will have to give it a try myself - it will be
a little while before I have time, but I do have a CDT setup.

-- 
Daniel Jacobowitz
CodeSourcery

Re: gdb-6.5 produces infinite backtrace on ARM

[Zarges, Olav]

Daniel Jacobowitz wrote:
>>
>> is returned at the end of the backtrace? Eclipse just throws away the
>> complete backtrace for the corresponding thread.
> 
> What other versions of GDB are you looking at?  With older versions, it was an
> ^error; with newer it's a ~"console output message".  I would not
> expect Eclipse to throw away the backtrace if there's a console output
> message.

verifying again which component is responsible for "eating the backtrace"
I found that those messages are created by method error(_("message...."))
e.g. called from gdb/frame.c.

error() finally throws an exception being caught by catch_exception(...)
in function mi_execute_command(...) in gdb/mi/mi-main.c. Therefore no backtrace
info is deliverd to eclipse via GDB/MI.

I'm using GDB 6.5 and it looks to me that it still creates an ^error. What
exactly means 'with older versions'? How can I upgrade.

Re: gdb-6.5 produces infinite backtrace on ARM

[Daniel Jacobowitz]

On Wed, Sep 06, 2006 at 03:32:39PM +0200, Zarges, Olav wrote:
> Daniel Jacobowitz wrote:
> >>
> >> is returned at the end of the backtrace? Eclipse just throws away the
> >> complete backtrace for the corresponding thread.
> > 
> > What other versions of GDB are you looking at?  With older versions, it was an
> > ^error; with newer it's a ~"console output message".  I would not
> > expect Eclipse to throw away the backtrace if there's a console output
> > message.
> 
> verifying again which component is responsible for "eating the backtrace"
> I found that those messages are created by method error(_("message...."))
> e.g. called from gdb/frame.c.
> 
> error() finally throws an exception being caught by catch_exception(...)
> in function mi_execute_command(...) in gdb/mi/mi-main.c. Therefore no backtrace
> info is deliverd to eclipse via GDB/MI.
> 
> I'm using GDB 6.5 and it looks to me that it still creates an ^error. What
> exactly means 'with older versions'? How can I upgrade.

The error should be caught in backtrace_command now.  The finished
output should still be printed.

-- 
Daniel Jacobowitz
CodeSourcery

Re: gdb-6.5 produces infinite backtrace on ARM

[Zarges, Olav]

Daniel Jacobowitz wrote:
> The error should be caught in backtrace_command now.  The finished
> output should still be printed.

Eclipse seems to read the stack via gdb/mi commands stack-info-depth and stack-list-frames and that
was what I was referring to (sorry). Please see log below. I tried to print the message with
warning(...) and return NULL instead of calling error(...) which seems to work but I don't know
about the side-effects this might involve.


[298.906]
207^done,new-thread-id="4",frame={level="0",addr="0x402c8d60",func="poll",args=[],from="/opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libc.so.6"}
[298.906] (gdb)
[298.906] 208-stack-info-depth
[298.968] &"Previous frame identical to this frame (corrupt stack?)\n"
[298.968] 208^error,msg="Previous frame identical to this frame (corrupt stack?)"
[298.968] (gdb)
[298.968] 209-stack-info-depth
[299.000] 209^done,depth="4"
[299.000] (gdb)
[299.000] 210-stack-info-depth
[299.015] 210^done,depth="4"
[299.015] (gdb)
[299.015] 211-thread-select 3
[299.031]
211^done,new-thread-id="3",frame={level="0",func="f2",args=[],file="../threads.cpp",fullname="/cygdrive/e/eclipse/ws_threads/Threads-arm/threads.cpp",line="30"},line="30",file="../threads.cpp"
[299.031] (gdb)
[299.031] 212-thread-select 4
[299.046]
212^done,new-thread-id="4",frame={level="0",addr="0x402c8d60",func="poll",args=[],from="/opt/crosstool/gcc-3.4.4-glibc-2.3.5-linux-2.6.12/arm-softfloat-linux-gnu/arm-softfloat-linux-gnu/lib/libc.so.6"}
[299.046] (gdb)
[299.046] 213-stack-list-frames 0 4
[299.140] &"Previous frame identical to this frame (corrupt stack?)\n"
[299.140] 213^error,msg="Previous frame identical to this frame (corrupt stack?)"
[299.140] (gdb)
[299.140] 214-thread-select 3
[299.156]

Re: gdb-6.5 produces infinite backtrace on ARM

[Daniel Jacobowitz]

On Wed, Sep 06, 2006 at 05:03:10PM +0200, Zarges, Olav wrote:
> 
> Daniel Jacobowitz wrote:
> > The error should be caught in backtrace_command now.  The finished
> > output should still be printed.
> 
> Eclipse seems to read the stack via gdb/mi commands stack-info-depth
> and stack-list-frames and that was what I was referring to (sorry).
> Please see log below. I tried to print the message with warning(...)
> and return NULL instead of calling error(...) which seems to work but
> I don't know about the side-effects this might involve.

Oh, I see.  Interesting.  We'll see what we can do about this; that
does make more sense.

However, since it has obviously listed a bunch of frames, CDT should
not be discarding the ones that did succeed!

-- 
Daniel Jacobowitz
CodeSourcery

Re: gdb-6.5 produces infinite backtrace on ARM

[Zarges, Olav]

Daniel Jacobowitz wrote:
> Oh, I see.  Interesting.  We'll see what we can do about this; that
> does make more sense.
> 
> However, since it has obviously listed a bunch of frames, CDT should
> not be discarding the ones that did succeed!
> 

The stack-list-frames command does not return any stack info at all if
error(...) is called. Therefore CDT is not discarding anything. The is
just the error message.

Regards
Olav

Re: gdb-6.5 produces infinite backtrace on ARM

[Daniel Jacobowitz]

On Wed, Sep 06, 2006 at 05:37:59PM +0200, Zarges, Olav wrote:
> 
> Daniel Jacobowitz wrote:
> > Oh, I see.  Interesting.  We'll see what we can do about this; that
> > does make more sense.
> > 
> > However, since it has obviously listed a bunch of frames, CDT should
> > not be discarding the ones that did succeed!
> > 
> 
> The stack-list-frames command does not return any stack info at all if
> error(...) is called. Therefore CDT is not discarding anything. The is
> just the error message.

Sorry - I misread your trace.  OK, I see what's generally wrong...

-- 
Daniel Jacobowitz
CodeSourcery

Re: gdb-6.5 produces infinite backtrace on ARM

[Zarges, Olav]

Daniel Jacobowitz wrote:
>> The stack-list-frames command does not return any stack info at all if
>> error(...) is called. Therefore CDT is not discarding anything. The is
>> just the error message.
> 
> Sorry - I misread your trace.  OK, I see what's generally wrong...

Glad to hear...

In order to supply other members of our dev-team with a working gdb
I tried to apply the patch you published on

http://sourceware.org/ml/gdb-patches/2006-08/msg00131.html

to the gdb-6.5-release I use. But the files did not match. CVS-head didn't
work either. You probably have a revised version.

Then I just added my changes (warnings instead of errors) and your new
frame_register_unwind_location() function and its invocation to
gdb-6.5-release source tree. Gdb then crashes after hitting the second
breakpoint.

It works if you patch your patch:

===================================================================
                                int *optimizedp, enum lval_type *lvalp,
                                CORE_ADDR *addrp, int *realnump)
{
-	while (this_frame->level >= 0)
+	while (this_frame != 0 && this_frame->level >= 0)


Do you plan to publish a patch on top of gdb-6.5-release for this issue?


-- 
Olav A. Zarges

Re: gdb-6.5 produces infinite backtrace on ARM

[Daniel Jacobowitz]

On Thu, Sep 07, 2006 at 06:15:49PM +0200, Zarges, Olav wrote:
> Then I just added my changes (warnings instead of errors) and your new
> frame_register_unwind_location() function and its invocation to
> gdb-6.5-release source tree. Gdb then crashes after hitting the second
> breakpoint.

You can't do that.  Frames should never be NULL; having a return
in a path which returns a NULL frame is invalid.

> Do you plan to publish a patch on top of gdb-6.5-release for this issue?

Maybe, but probably not.  We don't have enough manpower for that sort
of thing; I will try to get HEAD fixed, but that's all I can promise.

-- 
Daniel Jacobowitz
CodeSourcery

Re: gdb-6.5 produces infinite backtrace on ARM

[Daniel Jacobowitz]

On Wed, Sep 06, 2006 at 11:19:15AM -0400, Daniel Jacobowitz wrote:
> On Wed, Sep 06, 2006 at 05:03:10PM +0200, Zarges, Olav wrote:
> > 
> > Daniel Jacobowitz wrote:
> > > The error should be caught in backtrace_command now.  The finished
> > > output should still be printed.
> > 
> > Eclipse seems to read the stack via gdb/mi commands stack-info-depth
> > and stack-list-frames and that was what I was referring to (sorry).
> > Please see log below. I tried to print the message with warning(...)
> > and return NULL instead of calling error(...) which seems to work but
> > I don't know about the side-effects this might involve.
> 
> Oh, I see.  Interesting.  We'll see what we can do about this; that
> does make more sense.

Hi Olav,

Does this patch work for you?  I am not sure if it is the right
solution or not - it's careless about discarding errors - but I want
to make sure it works for you also before I start any discussion on
that.

-- 
Daniel Jacobowitz
CodeSourcery

Index: Makefile.in
===================================================================
RCS file: /cvs/src/src/gdb/Makefile.in,v
retrieving revision 1.840
diff -u -p -r1.840 Makefile.in
--- Makefile.in	22 Aug 2006 19:08:31 -0000	1.840
+++ Makefile.in	10 Sep 2006 14:48:01 -0000
@@ -3067,7 +3067,7 @@ mi-cmds.o: $(srcdir)/mi/mi-cmds.c $(defs
 	$(CC) -c $(INTERNAL_CFLAGS) $(srcdir)/mi/mi-cmds.c
 mi-cmd-stack.o: $(srcdir)/mi/mi-cmd-stack.c $(defs_h) $(target_h) $(frame_h) \
 	$(value_h) $(mi_cmds_h) $(ui_out_h) $(symtab_h) $(block_h) \
-	$(stack_h) $(dictionary_h) $(gdb_string_h)
+	$(stack_h) $(dictionary_h) $(exceptions_h) $(gdb_string_h)
 	$(CC) -c $(INTERNAL_CFLAGS) $(srcdir)/mi/mi-cmd-stack.c
 mi-cmd-var.o: $(srcdir)/mi/mi-cmd-var.c $(defs_h) $(mi_cmds_h) $(ui_out_h) \
 	$(mi_out_h) $(varobj_h) $(value_h) $(gdb_string_h)
Index: mi/mi-cmd-stack.c
===================================================================
RCS file: /cvs/src/src/gdb/mi/mi-cmd-stack.c,v
retrieving revision 1.30
diff -u -p -r1.30 mi-cmd-stack.c
--- mi/mi-cmd-stack.c	7 Sep 2006 16:40:18 -0000	1.30
+++ mi/mi-cmd-stack.c	10 Sep 2006 14:48:01 -0000
@@ -29,6 +29,7 @@
 #include "block.h"
 #include "stack.h"
 #include "dictionary.h"
+#include "exceptions.h"
 #include "gdb_string.h"
 
 static void list_args_or_locals (int locals, int values, struct frame_info *fi);
@@ -46,6 +47,7 @@ mi_cmd_stack_list_frames (char *command,
   int i;
   struct cleanup *cleanup_stack;
   struct frame_info *fi;
+  volatile struct gdb_exception e;
 
   if (argc > 2 || argc == 1)
     error (_("mi_cmd_stack_list_frames: Usage: [FRAME_LOW FRAME_HIGH]"));
@@ -75,16 +77,23 @@ mi_cmd_stack_list_frames (char *command,
 
   cleanup_stack = make_cleanup_ui_out_list_begin_end (uiout, "stack");
 
-  /* Now let;s print the frames up to frame_high, or until there are
-     frames in the stack. */
+  /* Now let's print the frames up to frame_high, or until there are
+     no more frames in the stack. */
   for (;
        fi && (i <= frame_high || frame_high == -1);
-       i++, fi = get_prev_frame (fi))
+       i++)
     {
       QUIT;
       /* Print the location and the address always, even for level 0.
          args == 0: don't print the arguments. */
       print_frame_info (fi, 1, LOC_AND_ADDRESS, 0 /* args */ );
+
+      TRY_CATCH (e, RETURN_MASK_ERROR)
+	{
+	  fi = get_prev_frame (fi);
+	}
+      if (e.reason == RETURN_ERROR)
+	break;
     }
 
   do_cleanups (cleanup_stack);
@@ -98,6 +107,7 @@ mi_cmd_stack_info_depth (char *command, 
   int frame_high;
   int i;
   struct frame_info *fi;
+  volatile struct gdb_exception e;
 
   if (argc > 1)
     error (_("mi_cmd_stack_info_depth: Usage: [MAX_DEPTH]"));
@@ -111,8 +121,16 @@ mi_cmd_stack_info_depth (char *command, 
 
   for (i = 0, fi = get_current_frame ();
        fi && (i < frame_high || frame_high == -1);
-       i++, fi = get_prev_frame (fi))
-    QUIT;
+       i++)
+    {
+      QUIT;
+      TRY_CATCH (e, RETURN_MASK_ERROR)
+	{
+	  fi = get_prev_frame (fi);
+	}
+      if (e.reason == RETURN_ERROR)
+	break;
+    }
 
   ui_out_field_int (uiout, "depth", i);
 
@@ -161,6 +179,7 @@ mi_cmd_stack_list_args (char *command, c
   int i;
   struct frame_info *fi;
   struct cleanup *cleanup_stack_args;
+  volatile struct gdb_exception e;
 
   if (argc < 1 || argc > 3 || argc == 2)
     error (_("mi_cmd_stack_list_args: Usage: PRINT_VALUES [FRAME_LOW FRAME_HIGH]"));
@@ -194,7 +213,7 @@ mi_cmd_stack_list_args (char *command, c
      frames in the stack. */
   for (;
        fi && (i <= frame_high || frame_high == -1);
-       i++, fi = get_prev_frame (fi))
+       i++)
     {
       struct cleanup *cleanup_frame;
       QUIT;
@@ -202,6 +221,13 @@ mi_cmd_stack_list_args (char *command, c
       ui_out_field_int (uiout, "level", i);
       list_args_or_locals (0, atoi (argv[0]), fi);
       do_cleanups (cleanup_frame);
+
+      TRY_CATCH (e, RETURN_MASK_ERROR)
+	{
+	  fi = get_prev_frame (fi);
+	}
+      if (e.reason == RETURN_ERROR)
+	break;
     }
 
   do_cleanups (cleanup_stack_args);

Re: gdb-6.5 produces infinite backtrace on ARM

[Michael Snyder]

On Sat, 2006-08-19 at 01:24 -0400, Daniel Jacobowitz wrote:
> On Tue, Aug 15, 2006 at 08:40:54AM -0400, Daniel Jacobowitz wrote:
> > Normally you should get an ugly end of the backtrace, not an infinite
> > one.  If you'd like, I can take a look at what's gone wrong.  I would
> > need enough to reproduce the problem - which probably means a tarball
> > including the source, compiled executable, and shared libraries which
> > it uses, since this may be specific to something about your build
> > environment.  If you want to do that, let me know; it's too big to post
> > to the list.
> 
> Thanks a lot for the test case.  The underlying cause was that
> pthread_start_thread is marked noreturn; GCC seems to have omitted
> the LR register save in this case.  I thought it wouldn't do that,
> but in any case, we should cope.
> 
> I know have a fairly general patch set for this problem, which
> produces:
> 
> (gdb) bt
> ...
> #4 0xxxxxxxx in pthread_start_thread () from /lib/libpthread.so.0
> Backtrace stopped: frame did not save the PC
> (gdb)
> 
> This isn't ideal - we could detect the pthread_start_thread function
> name and stop automatically, which might be a wise addition - but it's
> better than going off into the woods.
> 
> There's about 250 lines of changes involved, to one of the more
> complicated parts of GDB, so I will need to go over the patches again
> and post them separately.  But I'll try to make sure this is fixed
> soon.

Daniel, did you ever get around to posting your patch?

I have now run into this self-same problem myself, and 
would be glad to work with you on getting it handled.
Rather not duplicate effort, though...

Thanks,
Michael

Re: gdb-6.5 produces infinite backtrace on ARM

[Daniel Jacobowitz]

On Mon, Sep 18, 2006 at 06:39:53PM -0700, Michael Snyder wrote:
> > This isn't ideal - we could detect the pthread_start_thread function
> > name and stop automatically, which might be a wise addition - but it's
> > better than going off into the woods.
> > 
> > There's about 250 lines of changes involved, to one of the more
> > complicated parts of GDB, so I will need to go over the patches again
> > and post them separately.  But I'll try to make sure this is fixed
> > soon.
> 
> Daniel, did you ever get around to posting your patch?
> 
> I have now run into this self-same problem myself, and 
> would be glad to work with you on getting it handled.
> Rather not duplicate effort, though...

Yes.  You can find them here:
  http://sourceware.org/ml/gdb-patches/2006-08/msg00129.html
  http://sourceware.org/ml/gdb-patches/2006-08/msg00130.html
  http://sourceware.org/ml/gdb-patches/2006-08/msg00131.html

There was some discussion and I haven't been back to it yet.

-- 
Daniel Jacobowitz
CodeSourcery