[RFA] Don't unwind past entry point

[Randolph Chung <randolph@tausq.org>]

Here's a proposed patch to fix the problem i mentioned in
http://sources.redhat.com/ml/gdb/2004-11/msg00020.html

Briefly, if a compiler does a tail-call optimization to main, a function
called from main will return directly to the caller of main, and we
never terminate a backtrace.

It doesn't address Joel's problem with threads though.

is this ok?

randolph


2004-11-09  Randolph Chung  <tausq@debian.org>

	* frame.c (backtrace_past_entry): New flag.
	(get_prev_frame): Stop backtrace at the entry function if enabled
	by flag.  Update comments.
	(_initialize_frame): Add command to set backtrace_past_entry flag.

	doc/
	* gdb.texinfo: Document set/show backtrace past-entry commands.
	Rearrange index entries for set/show backtrace past-main.

Index: frame.c
===================================================================
RCS file: /cvs/src/src/gdb/frame.c,v
retrieving revision 1.192
diff -u -p -r1.192 frame.c
--- frame.c	29 Oct 2004 20:23:06 -0000	1.192
+++ frame.c	9 Nov 2004 19:21:38 -0000
@@ -115,6 +115,7 @@ static int frame_debug;
 /* Flag to indicate whether backtraces should stop at main et.al.  */
 
 static int backtrace_past_main;
+static int backtrace_past_entry;
 static unsigned int backtrace_limit = UINT_MAX;
 
 static void
@@ -1213,8 +1214,6 @@ get_prev_frame (struct frame_info *this_
      dummy frame PCs typically land in the entry func.  Don't apply
      this test to the sentinel frame.  Sentinel frames should always
      be allowed to unwind.  */
-  /* NOTE: cagney/2003-02-25: Don't enable until someone has found
-     hard evidence that this is needed.  */
   /* NOTE: cagney/2003-07-07: Fixed a bug in inside_main_func() -
      wasn't checking for "main" in the minimal symbols.  With that
      fixed asm-source tests now stop in "main" instead of halting the
@@ -1227,13 +1226,12 @@ get_prev_frame (struct frame_info *this_
      I guess) to determine the address range of the start function.
      That should provide a far better stopper than the current
      heuristics.  */
-  /* NOTE: cagney/2003-07-15: Need to add a "set backtrace
-     beyond-entry-func" command so that this can be selectively
-     disabled.  */
-  if (0
-#if 0
-      && backtrace_beyond_entry_func
-#endif
+  /* NOTE: tausq/2004-10-09: this is needed if, for example, the compiler
+     applied tail-call optimizations to main so that a function called 
+     from main returns directly to the caller of main.  Since we don't
+     stop at main, we should at least stop at the entry point of the
+     application.  */
+  if (!backtrace_past_entry
       && this_frame->unwind->type != DUMMY_FRAME && this_frame->level >= 0
       && inside_entry_func (this_frame))
     {
@@ -1531,6 +1529,17 @@ Whether backtraces should continue past 
 			   NULL, NULL, &set_backtrace_cmdlist,
 			   &show_backtrace_cmdlist);
 
+  add_setshow_boolean_cmd ("past-entry", class_obscure,
+			   &backtrace_past_entry, "\
+Set whether backtraces should continue past the entry point of a program.", "\
+Show whether backtraces should continue past the entry point of a program.", "\
+Normally there are no callers beyond the entry point of a program, so GDB\n\
+will terminate the backtrace there.  Set this variable if you need to see \n\
+the rest of the stack trace.", "\
+Whether backtraces should continue past the entry point is %s.",
+			   NULL, NULL, &set_backtrace_cmdlist,
+			   &show_backtrace_cmdlist);
+
   add_setshow_uinteger_cmd ("limit", class_obscure,
 			    &backtrace_limit, "\
 Set an upper bound on the number of backtrace levels.", "\
Index: doc/gdb.texinfo
===================================================================
RCS file: /cvs/src/src/gdb/doc/gdb.texinfo,v
retrieving revision 1.223
diff -u -p -r1.223 gdb.texinfo
--- doc/gdb.texinfo	8 Nov 2004 17:25:49 -0000	1.223
+++ doc/gdb.texinfo	9 Nov 2004 19:21:47 -0000
@@ -4109,7 +4109,7 @@ in a backtrace, you can change this beha
 @table @code
 @item set backtrace past-main
 @itemx set backtrace past-main on
-@kindex set backtrace
+@kindex set backtrace past-main
 Backtraces will continue past the user entry point.
 
 @item set backtrace past-main off
@@ -4117,9 +4117,24 @@ Backtraces will stop when they encounter
 default.
 
 @item show backtrace past-main
-@kindex show backtrace
+@kindex show backtrace past-main
 Display the current user entry point backtrace policy.
 
+@item set backtrace past-entry
+@itemx set backtrace past-entry on
+@kindex set backtrace past-entry
+Backtraces will continue past the internal entry point of an application. 
+This entry point is encoded by the linker when the application is built,
+and is likely before the user entry point ``main'' (or equivalent) is called.
+
+@item set backtrace past-entry off
+Backtraces will stop when they encouter the internal entry point of an
+application.  This is the default.
+
+@item show backtrace past-entry
+@kindex show backtrace past-entry
+Display the current internal entry point backtrace policy.
+
 @item set backtrace limit @var{n}
 @itemx set backtrace limit 0
 @cindex backtrace limit