Infinite backtrace on arm

Infinite backtrace on arm

[Jon Ringle]

Hi,

I backported arm kgdb to run in an embedded arm target running 2.2.16 kernel. 
I can debug the the target for the most part with gdb-5.3 over a serial 
connection. However, sometimes when I ask for a backtrace, the bt gets stuck 
recursing at the bottom of the bt. Here is an example:

Breakpoint 9, dimp_proc_bring_dsp_down (p_dsp=0x1022dc00, reason=7) at 
common/micro/dim/dimutl.c:247
(gdb) bt
#0  dimp_proc_bring_dsp_down (p_dsp=0x1022dc00, reason=7) at 
common/micro/dim/dimutl.c:247
#1  0x1005dff4 in dimp_bring_dsp_down (p_dsp=0x1022dc00, 
reason=DIM_DSP_ERR_OK) at common/micro/dim/dimutl.c:74
#2  0x1005ae20 in dimp_check_dsp_msgs () at common/micro/dim/dimdsp.c:1831
#3  0x100557a8 in dim_process_poll () at common/micro/dim/dimcomm.c:107
#4  0x10067d78 in dsp_timer1intHandler (irq=0, dev_id=0x0, regs=0x1018a820) at 
dspdriver.c:169
#5  0x1000c004 in do_IRQ (irq=1, regs=0x10ffdfa8) at irq.c:247
#6  0x1000b200 in linux_VECTOR_IRQ ()
#7  0x1000b200 in linux_VECTOR_IRQ ()
#8  0x1000b200 in linux_VECTOR_IRQ ()
[repeated ad infinitum...]

I found the following code check at blockframe.c:496 that is supposed to trap 
this situation:

  /* If ->frame and ->pc are unchanged, we are in the process of getting
     ourselves into an infinite backtrace.  Some architectures check this
     in FRAME_CHAIN or thereabouts, but it seems like there is no reason
     this can't be an architecture-independent check.  */
  if (next_frame != NULL)
    {
      if (prev->frame == next_frame->frame
        && prev->pc == next_frame->pc)
        {
          next_frame->prev = NULL;
          obstack_free (&frame_cache_obstack, prev);
          return NULL;
        }
    }

However, I found by debugging gdb that frame was changing by framesize. I 
think (but not confirmed) that this is happening because this is not caught 
by arm_frame_chain() and it is returning with:
    return fi->frame + fi->extra_info->framesize;

I fixed my problem with the following:
--- gdb/blockframe.c~   2003-10-26 00:17:13.000000000 -0400
+++ gdb/blockframe.c    2003-10-26 00:17:53.000000000 -0400
@@ -499,8 +499,7 @@
      this can't be an architecture-independent check.  */
   if (next_frame != NULL)
     {
-      if (prev->frame == next_frame->frame
-         && prev->pc == next_frame->pc)
+      if (prev->pc == next_frame->pc)
        {
          next_frame->prev = NULL;
          obstack_free (&frame_cache_obstack, prev);

I don't think this is the right thing to do, and that a fix is really needed 
in arm_frame_chain(). But I'm not sure what that might be. Does anyone have a 
suggestion?

Thanks,

Jon

Re: Infinite backtrace on arm

[Daniel Jacobowitz]

On Sun, Oct 26, 2003 at 12:28:13AM -0400, Jon Ringle wrote:
> Hi,
> 
> I backported arm kgdb to run in an embedded arm target running 2.2.16 kernel. 
> I can debug the the target for the most part with gdb-5.3 over a serial 
> connection. However, sometimes when I ask for a backtrace, the bt gets stuck 
> recursing at the bottom of the bt. Here is an example:
> 
> Breakpoint 9, dimp_proc_bring_dsp_down (p_dsp=0x1022dc00, reason=7) at 
> common/micro/dim/dimutl.c:247
> (gdb) bt
> #0  dimp_proc_bring_dsp_down (p_dsp=0x1022dc00, reason=7) at 
> common/micro/dim/dimutl.c:247
> #1  0x1005dff4 in dimp_bring_dsp_down (p_dsp=0x1022dc00, 
> reason=DIM_DSP_ERR_OK) at common/micro/dim/dimutl.c:74
> #2  0x1005ae20 in dimp_check_dsp_msgs () at common/micro/dim/dimdsp.c:1831
> #3  0x100557a8 in dim_process_poll () at common/micro/dim/dimcomm.c:107
> #4  0x10067d78 in dsp_timer1intHandler (irq=0, dev_id=0x0, regs=0x1018a820) at 
> dspdriver.c:169
> #5  0x1000c004 in do_IRQ (irq=1, regs=0x10ffdfa8) at irq.c:247
> #6  0x1000b200 in linux_VECTOR_IRQ ()
> #7  0x1000b200 in linux_VECTOR_IRQ ()
> #8  0x1000b200 in linux_VECTOR_IRQ ()
> [repeated ad infinitum...]
> 
> I found the following code check at blockframe.c:496 that is supposed to trap 
> this situation:
> 
>   /* If ->frame and ->pc are unchanged, we are in the process of getting
>      ourselves into an infinite backtrace.  Some architectures check this
>      in FRAME_CHAIN or thereabouts, but it seems like there is no reason
>      this can't be an architecture-independent check.  */
>   if (next_frame != NULL)
>     {
>       if (prev->frame == next_frame->frame
>         && prev->pc == next_frame->pc)
>         {
>           next_frame->prev = NULL;
>           obstack_free (&frame_cache_obstack, prev);
>           return NULL;
>         }
>     }
> 
> However, I found by debugging gdb that frame was changing by framesize. I 
> think (but not confirmed) that this is happening because this is not caught 
> by arm_frame_chain() and it is returning with:
>     return fi->frame + fi->extra_info->framesize;
> 
> I fixed my problem with the following:
> --- gdb/blockframe.c~   2003-10-26 00:17:13.000000000 -0400
> +++ gdb/blockframe.c    2003-10-26 00:17:53.000000000 -0400
> @@ -499,8 +499,7 @@
>       this can't be an architecture-independent check.  */
>    if (next_frame != NULL)
>      {
> -      if (prev->frame == next_frame->frame
> -         && prev->pc == next_frame->pc)
> +      if (prev->pc == next_frame->pc)
>         {
>           next_frame->prev = NULL;
>           obstack_free (&frame_cache_obstack, prev);
> 
> I don't think this is the right thing to do, and that a fix is really needed 
> in arm_frame_chain(). But I'm not sure what that might be. Does anyone have a 
> suggestion?

Well, for starters, I can tell you what the long-term "correct" thing
to do is:

  - Use a CVS version of GDB
  - Enable DWARF2-based unwinding for ARM; it's disabled in GDB because
    of some problems with Thumb unwind info.  I'll be working on this
    sometime soon.
  - Use a version of binutils which supports .cfi_* directives
    on ARM - I have patches for this but haven't posted them yet, I'll
    try again to do it this week.
  - Find a convention for unwind info that should tell GDB to terminate
    the backtrace.  I don't think there is one right now.
  - Annotate your copy of linux_VECTOR_IRQ, which is presumably an
    assembly stub, with information to make the backtrace stop.  Or to
    continue to its caller if it has one and isn't just called through
    an interrupt handler.

The problem is that there's not much GDB can do with hand-coded
assembly functions.

However, GDB ought to be able to detect and stop that loop without any
of this.  It seems to me that the problem is not two frames with the
same PC, but two frames with their PC "saved" in the same place - i.e.
GDB failing to figure out where the PC is saved.  This is made a little
tricky in current versions of GDB, because the PC is normally unwound
using frame_pc_unwind (which doesn't tell us where it was saved, since
the interface doesn't assume it's a single normal register), and the
core code doesn't have a concept of a "PC register" any more.

Andrew, any idea on how to do this?

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Infinite backtrace on arm

[Andrew Cagney]

> The problem is that there's not much GDB can do with hand-coded
> assembly functions.
> 
> However, GDB ought to be able to detect and stop that loop without any
> of this.  It seems to me that the problem is not two frames with the
> same PC, but two frames with their PC "saved" in the same place - i.e.
> GDB failing to figure out where the PC is saved.  This is made a little
> tricky in current versions of GDB, because the PC is normally unwound
> using frame_pc_unwind (which doesn't tell us where it was saved, since
> the interface doesn't assume it's a single normal register), and the
> core code doesn't have a concept of a "PC register" any more.
> 
> Andrew, any idea on how to do this?

Before anything else, find out what a current GDB does - its code to 
detect a "corrupt stack" is now much improved.

Andrew

Re: Infinite backtrace on arm

[Daniel Jacobowitz]

On Mon, Oct 27, 2003 at 10:21:54AM -0500, Andrew Cagney wrote:
> >The problem is that there's not much GDB can do with hand-coded
> >assembly functions.
> >
> >However, GDB ought to be able to detect and stop that loop without any
> >of this.  It seems to me that the problem is not two frames with the
> >same PC, but two frames with their PC "saved" in the same place - i.e.
> >GDB failing to figure out where the PC is saved.  This is made a little
> >tricky in current versions of GDB, because the PC is normally unwound
> >using frame_pc_unwind (which doesn't tell us where it was saved, since
> >the interface doesn't assume it's a single normal register), and the
> >core code doesn't have a concept of a "PC register" any more.
> >
> >Andrew, any idea on how to do this?
> 
> Before anything else, find out what a current GDB does - its code to 
> detect a "corrupt stack" is now much improved.

I'll check - I don't have my test for this problem handy any more but I
know as of a month or two ago the problem still existed.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Infinite backtrace on arm

[Jon Ringle]

[-- Attachment #1: Type: text/plain, Size: 1261 bytes --]

On Monday 27 October 2003 10:23 am, Daniel Jacobowitz wrote:
> On Mon, Oct 27, 2003 at 10:21:54AM -0500, Andrew Cagney wrote:
> > >The problem is that there's not much GDB can do with hand-coded
> > >assembly functions.
> > >
> > >However, GDB ought to be able to detect and stop that loop without any
> > >of this.  It seems to me that the problem is not two frames with the
> > >same PC, but two frames with their PC "saved" in the same place - i.e.
> > >GDB failing to figure out where the PC is saved.  This is made a little
> > >tricky in current versions of GDB, because the PC is normally unwound
> > >using frame_pc_unwind (which doesn't tell us where it was saved, since
> > >the interface doesn't assume it's a single normal register), and the
> > >core code doesn't have a concept of a "PC register" any more.
> > >
> > >Andrew, any idea on how to do this?
> >
> > Before anything else, find out what a current GDB does - its code to
> > detect a "corrupt stack" is now much improved.
>
> I'll check - I don't have my test for this problem handy any more but I
> know as of a month or two ago the problem still existed.

Here is a debug session debugging gdb 2003-10-27-cvs. I enabled frame_debug
which I hope will contain useful information:

Jon



[-- Attachment #2: gdb-session.log --]
[-- Type: text/x-log, Size: 16048 bytes --]

GNU gdb 5.3-22mdk (Mandrake Linux)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i586-mandrake-linux-gnu"...
Setting up the environment for debugging gdb.
Breakpoint 1 at 0x811e895: file ../../src/gdb/utils.c, line 805.
Breakpoint 2 at 0x8079c48: file ../../src/gdb/cli/cli-cmds.c, line 190.
(top-gdb) b get_prev_frame 
Breakpoint 3 at 0x8124700: file ../../src/gdb/frame.c, line 1743.
(top-gdb) run -b 115200 -x ~/sp/sandbox/WipSoundpipe/linux/.gdbinit ~/sp/sandbox/WipSoundpipe/linux/vmlinux 
Starting program: /home/eringlej/gdb-cvs/gdb-6.0-arm-build/gdb/gdb -b 115200 -x ~/sp/sandbox/WipSoundpipe/linux/.gdbinit ~/sp/sandbox/WipSoundpipe/linux/vmlinux
GNU gdb 2003-10-27-cvs
Copyright 2003 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "--host=i686-pc-linux-gnu --target=arm-linux"...
Setting up the environment for debugging gdb.
.gdbinit:5: Error in sourced command file:
Function "internal_error" not defined.
(gdb) open-extended-debug 
During symbol reading, struct/union type gets multiply defined: struct dentry.

Breakpoint 3, get_prev_frame (this_frame=0x82a8f70)
    at ../../src/gdb/frame.c:1743
1743	  if (frame_debug)
(top-gdb) set frame_debug = 1
(top-gdb) c
Continuing.
{ get_prev_frame (this_frame=-1) { frame_register_unwind (frame=-1,regnum="pc",...) -> *optimizedp=0 *lvalp=2 *addrp=0x3c *bufferp=[c0310110] }
{ frame_pc_unwind (this_frame=-1) -> 0x100131c0 }
-> {level=0,type=UNKNOWN_FRAME,unwind=<unknown>,pc=0x100131c0,id=<unknown>,func=<unknown>} }
{ deprecated_update_frame_pc_hack (frame=0,pc=0x100131c0) }
breakpoint () at kgdb-stub.c:1148
1148	}
warning: shared library handler failed to enable breakpoint
(gdb) b dimutl.c:247
Breakpoint 1 at 0x1005e2f0: file common/micro/dim/dimutl.c, line 247.
(gdb) c
{ frame_register_unwind (frame=-1,regnum="sp",...) -> *optimizedp=0 *lvalp=2 *addrp=0x34 *bufferp=[f0ffff10] }
{ frame_register_unwind (frame=-1,regnum="sp",...) -> *optimizedp=0 *lvalp=2 *addrp=0x34 *bufferp=[f0ffff10] }
Continuing.
<<< lots of output from program suppressed >>>
{ flush_cached_frames () }
{ frame_id_p (l={stack=0x0,code=0x0,special=0x0}) -> 0 }
{ create_sentinel_frame (...) -> {level=-1,type=NORMAL_FRAME,unwind=0x824a3c0,pc=<unknown>,id={stack=0x0,code=0x0,special=0x0},func=<unknown>} }

Breakpoint 3, get_prev_frame (this_frame=0x82a8f70)
    at ../../src/gdb/frame.c:1743
1743	  if (frame_debug)
(top-gdb) c
Continuing.
{ get_prev_frame (this_frame=-1) { frame_register_unwind (frame=-1,regnum="pc",...) -> *optimizedp=0 *lvalp=2 *addrp=0x3c *bufferp=[f0e20510] }
{ frame_pc_unwind (this_frame=-1) -> 0x1005e2f0 }
-> {level=0,type=UNKNOWN_FRAME,unwind=<unknown>,pc=0x1005e2f0,id=<unknown>,func=<unknown>} }
{ deprecated_update_frame_pc_hack (frame=0,pc=0x1005e2f0) }
{ frame_register_unwind (frame=-1,regnum="sp",...) -> *optimizedp=0 *lvalp=2 *addrp=0x34 *bufferp=[f0ffff10] }
{ frame_register_unwind (frame=-1,regnum="r6",...) -> *optimizedp=0 *lvalp=2 *addrp=0x18 *bufferp=[00dc2210] }
{ frame_register_unwind (frame=-1,regnum="r11",...) -> *optimizedp=0 *lvalp=2 *addrp=0x2c *bufferp=[04dfff10] }

Breakpoint 1, dimp_proc_bring_dsp_down (p_dsp=0x1022dc00, reason=7) at common/micro/dim/dimutl.c:247
247		    dim_mcb.a_dim_tcids[p_channel->p_dim_tcid->tcid].p_channel = NULL;
(gdb) bt
{ frame_register_unwind (frame=-1,regnum="r6",...) -> *optimizedp=0 *lvalp=2 *addrp=0x18 *bufferp=[00dc2210] }
#0  dimp_proc_bring_dsp_down (p_dsp=0x1022dc00, reason=7) at common/micro/dim/dimutl.c:247

Breakpoint 3, get_prev_frame (this_frame=0x82a8fc4)
    at ../../src/gdb/frame.c:1743
1743	  if (frame_debug)
(top-gdb) c
Continuing.
{ get_prev_frame (this_frame=0) { get_frame_id (fi=0) { frame_func_unwind (fi=-1) -> 0x1005e17c }
-> {stack=0x10ffdf08,code=0x1005e17c,special=0x0} }
{ frame_id_p (l={stack=0x10ffdf08,code=0x1005e17c,special=0x0}) -> 1 }
{ frame_register_unwind (frame=0,regnum="pc",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf00 *bufferp=[f4df0510] }
{ frame_pc_unwind (this_frame=0) -> 0x1005dff4 }
-> {level=1,type=UNKNOWN_FRAME,unwind=<unknown>,pc=0x1005dff4,id=<unknown>,func=<unknown>} }
{ frame_register_unwind (frame=0,regnum="r4",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdedc *bufferp=[00dc2210] }
{ frame_register_unwind (frame=0,regnum="r1",...) { frame_register_unwind (frame=-1,regnum="r1",...) -> *optimizedp=0 *lvalp=2 *addrp=0x4 *bufferp=[00000000] }
-> *optimizedp=0 *lvalp=2 *addrp=0x4 *bufferp=[00000000] }
#1  0x1005dff4 in dimp_bring_dsp_down (p_dsp=0x1022dc00, reason=DIM_DSP_ERR_OK) at common/micro/dim/dimutl.c:74

Breakpoint 3, get_prev_frame (this_frame=0x82a9160)
    at ../../src/gdb/frame.c:1743
1743	  if (frame_debug)
(top-gdb) c
Continuing.
{ get_prev_frame (this_frame=1) { get_frame_id (fi=1) { frame_register_unwind (frame=0,regnum="r11",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdef8 *bufferp=[1cdfff10] }
{ frame_func_unwind (fi=0) -> 0x1005df78 }
{ frame_id_eq (l={stack=0x10ffdf08,code=0x1005e17c,special=0x0},r={stack=0x10ffdf20,code=0x1005df78,special=0x0}) -> 0 }
-> {stack=0x10ffdf20,code=0x1005df78,special=0x0} }
{ frame_id_p (l={stack=0x10ffdf20,code=0x1005df78,special=0x0}) -> 1 }
{ frame_id_inner (l={stack=0x10ffdf20,code=0x1005df78,special=0x0},r={stack=0x10ffdf08,code=0x1005e17c,special=0x0}) -> 0 }
{ frame_id_eq (l={stack=0x10ffdf20,code=0x1005df78,special=0x0},r={stack=0x10ffdf08,code=0x1005e17c,special=0x0}) -> 0 }
{ frame_register_unwind (frame=1,regnum="pc",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf18 *bufferp=[20ae0510] }
{ frame_pc_unwind (this_frame=1) -> 0x1005ae20 }
-> {level=2,type=UNKNOWN_FRAME,unwind=<unknown>,pc=0x1005ae20,id=<unknown>,func=<unknown>} }
#2  0x1005ae20 in dimp_check_dsp_msgs () at common/micro/dim/dimdsp.c:1831

Breakpoint 3, get_prev_frame (this_frame=0x82a92fc)
    at ../../src/gdb/frame.c:1743
1743	  if (frame_debug)
(top-gdb) c
Continuing.
{ get_prev_frame (this_frame=2) { get_frame_id (fi=2) { frame_register_unwind (frame=1,regnum="r11",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf10 *bufferp=[54dfff10] }
{ frame_func_unwind (fi=1) -> 0x1005a5e4 }
{ frame_id_eq (l={stack=0x10ffdf20,code=0x1005df78,special=0x0},r={stack=0x10ffdf58,code=0x1005a5e4,special=0x0}) -> 0 }
-> {stack=0x10ffdf58,code=0x1005a5e4,special=0x0} }
{ frame_id_p (l={stack=0x10ffdf58,code=0x1005a5e4,special=0x0}) -> 1 }
{ frame_id_inner (l={stack=0x10ffdf58,code=0x1005a5e4,special=0x0},r={stack=0x10ffdf20,code=0x1005df78,special=0x0}) -> 0 }
{ frame_id_eq (l={stack=0x10ffdf58,code=0x1005a5e4,special=0x0},r={stack=0x10ffdf20,code=0x1005df78,special=0x0}) -> 0 }
{ frame_register_unwind (frame=2,regnum="pc",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf50 *bufferp=[a8570510] }
{ frame_pc_unwind (this_frame=2) -> 0x100557a8 }
-> {level=3,type=UNKNOWN_FRAME,unwind=<unknown>,pc=0x100557a8,id=<unknown>,func=<unknown>} }
#3  0x100557a8 in dim_process_poll () at common/micro/dim/dimcomm.c:107

Breakpoint 3, get_prev_frame (this_frame=0x82a9498)
    at ../../src/gdb/frame.c:1743
1743	  if (frame_debug)
(top-gdb) c
Continuing.
{ get_prev_frame (this_frame=3) { get_frame_id (fi=3) { frame_register_unwind (frame=2,regnum="r11",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf48 *bufferp=[68dfff10] }
{ frame_func_unwind (fi=2) -> 0x100556e4 }
{ frame_id_eq (l={stack=0x10ffdf58,code=0x1005a5e4,special=0x0},r={stack=0x10ffdf6c,code=0x100556e4,special=0x0}) -> 0 }
-> {stack=0x10ffdf6c,code=0x100556e4,special=0x0} }
{ frame_id_p (l={stack=0x10ffdf6c,code=0x100556e4,special=0x0}) -> 1 }
{ frame_id_inner (l={stack=0x10ffdf6c,code=0x100556e4,special=0x0},r={stack=0x10ffdf58,code=0x1005a5e4,special=0x0}) -> 0 }
{ frame_id_eq (l={stack=0x10ffdf6c,code=0x100556e4,special=0x0},r={stack=0x10ffdf58,code=0x1005a5e4,special=0x0}) -> 0 }
{ frame_register_unwind (frame=3,regnum="pc",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf64 *bufferp=[787d0610] }
{ frame_pc_unwind (this_frame=3) -> 0x10067d78 }
-> {level=4,type=UNKNOWN_FRAME,unwind=<unknown>,pc=0x10067d78,id=<unknown>,func=<unknown>} }
{ frame_register_unwind (frame=3,regnum="r0",...) { frame_register_unwind (frame=2,regnum="r0",...) { frame_register_unwind (frame=1,regnum="r0",...) { frame_register_unwind (frame=0,regnum="r0",...) { frame_register_unwind (frame=-1,regnum="r0",...) -> *optimizedp=0 *lvalp=2 *addrp=0x0 *bufferp=[00000000] }
-> *optimizedp=0 *lvalp=2 *addrp=0x0 *bufferp=[00000000] }
-> *optimizedp=0 *lvalp=2 *addrp=0x0 *bufferp=[00000000] }
-> *optimizedp=0 *lvalp=2 *addrp=0x0 *bufferp=[00000000] }
-> *optimizedp=0 *lvalp=2 *addrp=0x0 *bufferp=[00000000] }
{ frame_register_unwind (frame=3,regnum="r1",...) { frame_register_unwind (frame=2,regnum="r1",...) { frame_register_unwind (frame=1,regnum="r1",...) { frame_register_unwind (frame=0,regnum="r1",...) { frame_register_unwind (frame=-1,regnum="r1",...) -> *optimizedp=0 *lvalp=2 *addrp=0x4 *bufferp=[00000000] }
-> *optimizedp=0 *lvalp=2 *addrp=0x4 *bufferp=[00000000] }
-> *optimizedp=0 *lvalp=2 *addrp=0x4 *bufferp=[00000000] }
-> *optimizedp=0 *lvalp=2 *addrp=0x4 *bufferp=[00000000] }
-> *optimizedp=0 *lvalp=2 *addrp=0x4 *bufferp=[00000000] }
{ frame_register_unwind (frame=3,regnum="r2",...) { frame_register_unwind (frame=2,regnum="r2",...) { frame_register_unwind (frame=1,regnum="r2",...) { frame_register_unwind (frame=0,regnum="r2",...) { frame_register_unwind (frame=-1,regnum="r2",...) -> *optimizedp=0 *lvalp=2 *addrp=0x8 *bufferp=[20a81810] }
-> *optimizedp=0 *lvalp=2 *addrp=0x8 *bufferp=[20a81810] }
-> *optimizedp=0 *lvalp=2 *addrp=0x8 *bufferp=[20a81810] }
-> *optimizedp=0 *lvalp=2 *addrp=0x8 *bufferp=[20a81810] }
-> *optimizedp=0 *lvalp=2 *addrp=0x8 *bufferp=[20a81810] }
#4  0x10067d78 in dsp_timer1intHandler (irq=0x0, dev_id=0x0, regs=0x1018a820) at dspdriver.c:169

Breakpoint 3, get_prev_frame (this_frame=0x82a9634)
    at ../../src/gdb/frame.c:1743
1743	  if (frame_debug)
(top-gdb) c
Continuing.
{ get_prev_frame (this_frame=4) { get_frame_id (fi=4) { frame_register_unwind (frame=3,regnum="r11",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf5c *bufferp=[7cdfff10] }
{ frame_func_unwind (fi=3) -> 0x10067d28 }
{ frame_id_eq (l={stack=0x10ffdf6c,code=0x100556e4,special=0x0},r={stack=0x10ffdf80,code=0x10067d28,special=0x0}) -> 0 }
-> {stack=0x10ffdf80,code=0x10067d28,special=0x0} }
{ frame_id_p (l={stack=0x10ffdf80,code=0x10067d28,special=0x0}) -> 1 }
{ frame_id_inner (l={stack=0x10ffdf80,code=0x10067d28,special=0x0},r={stack=0x10ffdf6c,code=0x100556e4,special=0x0}) -> 0 }
{ frame_id_eq (l={stack=0x10ffdf80,code=0x10067d28,special=0x0},r={stack=0x10ffdf6c,code=0x100556e4,special=0x0}) -> 0 }
{ frame_register_unwind (frame=4,regnum="pc",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf78 *bufferp=[04c00010] }
{ frame_pc_unwind (this_frame=4) -> 0x1000c004 }
-> {level=5,type=UNKNOWN_FRAME,unwind=<unknown>,pc=0x1000c004,id=<unknown>,func=<unknown>} }
{ frame_register_unwind (frame=4,regnum="r7",...) { frame_register_unwind (frame=3,regnum="r7",...) { frame_register_unwind (frame=2,regnum="r7",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf40 *bufferp=[01000000] }
-> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf40 *bufferp=[01000000] }
-> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf40 *bufferp=[01000000] }
{ frame_register_unwind (frame=4,regnum="r8",...) { frame_register_unwind (frame=3,regnum="r8",...) { frame_register_unwind (frame=2,regnum="r8",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf44 *bufferp=[a8dfff10] }
-> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf44 *bufferp=[a8dfff10] }
-> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf44 *bufferp=[a8dfff10] }
#5  0x1000c004 in do_IRQ (irq=0x1, regs=0x10ffdfa8) at irq.c:247

Breakpoint 3, get_prev_frame (this_frame=0x82a97d0)
    at ../../src/gdb/frame.c:1743
1743	  if (frame_debug)
(top-gdb) c
Continuing.
{ get_prev_frame (this_frame=5) { get_frame_id (fi=5) { frame_register_unwind (frame=4,regnum="r11",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdf70 *bufferp=[a4dfff10] }
{ frame_func_unwind (fi=4) -> 0x1000bf68 }
{ frame_id_eq (l={stack=0x10ffdf80,code=0x10067d28,special=0x0},r={stack=0x10ffdfa8,code=0x1000bf68,special=0x0}) -> 0 }
-> {stack=0x10ffdfa8,code=0x1000bf68,special=0x0} }
{ frame_id_p (l={stack=0x10ffdfa8,code=0x1000bf68,special=0x0}) -> 1 }
{ frame_id_inner (l={stack=0x10ffdfa8,code=0x1000bf68,special=0x0},r={stack=0x10ffdf80,code=0x10067d28,special=0x0}) -> 0 }
{ frame_id_eq (l={stack=0x10ffdfa8,code=0x1000bf68,special=0x0},r={stack=0x10ffdf80,code=0x10067d28,special=0x0}) -> 0 }
{ frame_register_unwind (frame=5,regnum="pc",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdfa0 *bufferp=[00b20010] }
{ frame_pc_unwind (this_frame=5) -> 0x1000b200 }
-> {level=6,type=UNKNOWN_FRAME,unwind=<unknown>,pc=0x1000b200,id=<unknown>,func=<unknown>} }
#6  0x1000b200 in linux_VECTOR_IRQ ()

Breakpoint 3, get_prev_frame (this_frame=0x82a996c)
    at ../../src/gdb/frame.c:1743
1743	  if (frame_debug)
(top-gdb) c
Continuing.
{ get_prev_frame (this_frame=6) { get_frame_id (fi=6) { frame_register_unwind (frame=5,regnum="sp",...) -> *optimizedp=26 *lvalp=0 *addrp=0x82a99d0 *bufferp=[a8dfff10] }
{ frame_func_unwind (fi=5) -> 0x1000b1b8 }
{ frame_id_eq (l={stack=0x10ffdfa8,code=0x1000bf68,special=0x0},r={stack=0x10ffdff0,code=0x1000b1b8,special=0x0}) -> 0 }
-> {stack=0x10ffdff0,code=0x1000b1b8,special=0x0} }
{ frame_id_p (l={stack=0x10ffdff0,code=0x1000b1b8,special=0x0}) -> 1 }
{ frame_id_inner (l={stack=0x10ffdff0,code=0x1000b1b8,special=0x0},r={stack=0x10ffdfa8,code=0x1000bf68,special=0x0}) -> 0 }
{ frame_id_eq (l={stack=0x10ffdff0,code=0x1000b1b8,special=0x0},r={stack=0x10ffdfa8,code=0x1000bf68,special=0x0}) -> 0 }
{ frame_register_unwind (frame=6,regnum="pc",...) { frame_register_unwind (frame=5,regnum="lr",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdfa0 *bufferp=[00b20010] }
-> *optimizedp=0 *lvalp=1 *addrp=0x10ffdfa0 *bufferp=[00b20010] }
{ frame_pc_unwind (this_frame=6) -> 0x1000b200 }
-> {level=7,type=UNKNOWN_FRAME,unwind=<unknown>,pc=0x1000b200,id=<unknown>,func=<unknown>} }
#7  0x1000b200 in linux_VECTOR_IRQ ()

Breakpoint 3, get_prev_frame (this_frame=0x82a9b08)
    at ../../src/gdb/frame.c:1743
1743	  if (frame_debug)
(top-gdb) c
Continuing.
{ get_prev_frame (this_frame=7) { get_frame_id (fi=7) { frame_register_unwind (frame=6,regnum="sp",...) -> *optimizedp=26 *lvalp=0 *addrp=0x82a9b6c *bufferp=[f0dfff10] }
{ frame_func_unwind (fi=6) -> 0x1000b1b8 }
{ frame_id_eq (l={stack=0x10ffdff0,code=0x1000b1b8,special=0x0},r={stack=0x10ffe038,code=0x1000b1b8,special=0x0}) -> 0 }
-> {stack=0x10ffe038,code=0x1000b1b8,special=0x0} }
{ frame_id_p (l={stack=0x10ffe038,code=0x1000b1b8,special=0x0}) -> 1 }
{ frame_id_inner (l={stack=0x10ffe038,code=0x1000b1b8,special=0x0},r={stack=0x10ffdff0,code=0x1000b1b8,special=0x0}) -> 0 }
{ frame_id_eq (l={stack=0x10ffe038,code=0x1000b1b8,special=0x0},r={stack=0x10ffdff0,code=0x1000b1b8,special=0x0}) -> 0 }
{ frame_register_unwind (frame=7,regnum="pc",...) { frame_register_unwind (frame=6,regnum="lr",...) { frame_register_unwind (frame=5,regnum="lr",...) -> *optimizedp=0 *lvalp=1 *addrp=0x10ffdfa0 *bufferp=[00b20010] }
-> *optimizedp=0 *lvalp=1 *addrp=0x10ffdfa0 *bufferp=[00b20010] }
-> *optimizedp=0 *lvalp=1 *addrp=0x10ffdfa0 *bufferp=[00b20010] }
{ frame_pc_unwind (this_frame=7) -> 0x1000b200 }
-> {level=8,type=UNKNOWN_FRAME,unwind=<unknown>,pc=0x1000b200,id=<unknown>,func=<unknown>} }
#8  0x1000b200 in linux_VECTOR_IRQ ()

Breakpoint 3, get_prev_frame (this_frame=0x82a9ca4)
    at ../../src/gdb/frame.c:1743
1743	  if (frame_debug)
(top-gdb) quit
The program is running.  Exit anyway? (y or n)