Don't allow switching on the TUI in some cases

Don't allow switching on the TUI in some cases

[Pedro Alves]

A slightly updated patch originally posted at:
<http://sourceware.org/ml/gdb-patches/2010-03/msg01085.html>

Tested on x86_64-linux and checked in.

-- 
Pedro Alves

2010-03-30  Pedro Alves  <pedro@codesourcery.com>

	* tui/tui-interp.c (tui_is_toplevel): New.
	(tui_init): Set it.
	(tui_allowed_p): New.
	* tui/tui.c (tui_enable): Check if the TUI is allowed before
	enabling it.
	* tui/tui.h (tui_allowed_p): Declare.

---
 gdb/tui/tui-interp.c |   17 +++++++++++++++++
 gdb/tui/tui.c        |    3 +++
 gdb/tui/tui.h        |    4 ++++
 3 files changed, 24 insertions(+)

Index: src/gdb/tui/tui-interp.c
===================================================================
--- src.orig/gdb/tui/tui-interp.c	2010-03-30 21:40:50.000000000 +0100
+++ src/gdb/tui/tui-interp.c	2010-03-30 22:44:05.000000000 +0100
@@ -45,11 +45,16 @@ tui_exit (void)
   tui_disable ();
 }
 
+/* True if TUI is the top-level interpreter.  */
+static int tui_is_toplevel = 0;
+
 /* These implement the TUI interpreter.  */
 
 static void *
 tui_init (int top_level)
 {
+  tui_is_toplevel = top_level;
+
   /* Install exit handler to leave the screen in a good shape.  */
   atexit (tui_exit);
 
@@ -63,6 +68,18 @@ tui_init (int top_level)
   return NULL;
 }
 
+/* True if enabling the TUI is allowed.  Example, if the top level
+   interpreter is MI, enabling curses will certainly lose.  */
+
+int
+tui_allowed_p (void)
+{
+  /* Only if TUI is the top level interpreter.  Also don't try to
+     setup curses (and print funny control characters) if we're not
+     outputting to a terminal.  */
+  return tui_is_toplevel && ui_file_isatty (gdb_stdout);
+}
+
 static int
 tui_resume (void *data)
 {
Index: src/gdb/tui/tui.c
===================================================================
--- src.orig/gdb/tui/tui.c	2010-03-30 21:40:50.000000000 +0100
+++ src/gdb/tui/tui.c	2010-03-30 22:39:28.000000000 +0100
@@ -364,6 +364,9 @@ tui_initialize_readline (void)
 void
 tui_enable (void)
 {
+  if (!tui_allowed_p ())
+    error (_("TUI mode not allowed"));
+
   if (tui_active)
     return;
 
Index: src/gdb/tui/tui.h
===================================================================
--- src.orig/gdb/tui/tui.h	2010-03-30 21:40:50.000000000 +0100
+++ src/gdb/tui/tui.h	2010-03-30 22:39:28.000000000 +0100
@@ -65,6 +65,10 @@ extern int tui_get_command_dimension (un
    key shortcut.  */
 extern void tui_initialize_readline (void);
 
+/* True if enabling the TUI is allowed.  Example, if the top level
+   interpreter is MI, enabling curses will certainly lose.  */
+extern int tui_allowed_p (void);
+
 /* Enter in the tui mode (curses).  */
 extern void tui_enable (void);
 

Re: Don't allow switching on the TUI in some cases

[Eli Zaretskii]

> From: Pedro Alves <pedro@codesourcery.com>
> Date: Tue, 30 Mar 2010 23:23:40 +0100
> 
>  void
>  tui_enable (void)
>  {
> +  if (!tui_allowed_p ())
> +    error (_("TUI mode not allowed"));
> +

Does this need a suitable change for the manual?  We currently say
this in the "TUI Commands" node:

    The TUI has specific commands to control the text windows.  These
    commands are always available, even when GDB is not in the TUI mode.
    When GDB is in the standard mode, most of these commands will
    automatically switch to the TUI mode.

Looks like this is no longer true with this patch, is that right?

Thanks.

Re: Don't allow switching on the TUI in some cases

[Pedro Alves]

On Wednesday 31 March 2010 08:30:02, Eli Zaretskii wrote:
> > From: Pedro Alves <pedro@codesourcery.com>
> > Date: Tue, 30 Mar 2010 23:23:40 +0100
> > 
> >  void
> >  tui_enable (void)
> >  {
> > +  if (!tui_allowed_p ())
> > +    error (_("TUI mode not allowed"));
> > +
> 
> Does this need a suitable change for the manual?  We currently say
> this in the "TUI Commands" node:
> 
>     The TUI has specific commands to control the text windows.  These
>     commands are always available, even when GDB is not in the TUI mode.
>     When GDB is in the standard mode, most of these commands will
>     automatically switch to the TUI mode.
> 
> Looks like this is no longer true with this patch, is that right?

I had considered it and thought it could be more noise
than signal to extend that, but, maybe if I keep it a separate
paragraph it's okay.  How about this?

-- 
Pedro Alves

2010-03-31  Pedro Alves  <pedro@codesourcery.com>

	gdb/doc/
	* gdb.texinfo (TUI Commands): Mention that in some cases, these
	commands error out.

---
 gdb/doc/gdb.texinfo |    6 ++++++
 1 file changed, 6 insertions(+)

Index: src/gdb/doc/gdb.texinfo
===================================================================
--- src.orig/gdb/doc/gdb.texinfo	2010-03-31 12:32:17.000000000 +0100
+++ src/gdb/doc/gdb.texinfo	2010-03-31 12:42:53.000000000 +0100
@@ -21653,6 +21653,12 @@ These commands are always available, eve
 the TUI mode.  When @value{GDBN} is in the standard mode, most
 of these commands will automatically switch to the TUI mode.
 
+Note that if @value{GDBN}'s @code{stdout} is not connected to a
+terminal, or @value{GDBN} has been started with the machine interface
+interpreter (@pxref{GDB/MI, ,The @sc{gdb/mi} Interface}), most of
+these commands will fail with an error, because it would not be
+possible or desirable to enable curses window management.
+
 @table @code
 @item info win
 @kindex info win

Re: Don't allow switching on the TUI in some cases

[Eli Zaretskii]

> From: Pedro Alves <pedro@codesourcery.com>
> Date: Wed, 31 Mar 2010 12:47:07 +0100
> 
> 2010-03-31  Pedro Alves  <pedro@codesourcery.com>
> 
> 	gdb/doc/
> 	* gdb.texinfo (TUI Commands): Mention that in some cases, these
> 	commands error out.
> 
> ---
>  gdb/doc/gdb.texinfo |    6 ++++++
>  1 file changed, 6 insertions(+)
> 
> Index: src/gdb/doc/gdb.texinfo
> ===================================================================
> --- src.orig/gdb/doc/gdb.texinfo	2010-03-31 12:32:17.000000000 +0100
> +++ src/gdb/doc/gdb.texinfo	2010-03-31 12:42:53.000000000 +0100
> @@ -21653,6 +21653,12 @@ These commands are always available, eve
>  the TUI mode.  When @value{GDBN} is in the standard mode, most
>  of these commands will automatically switch to the TUI mode.
>  
> +Note that if @value{GDBN}'s @code{stdout} is not connected to a
> +terminal, or @value{GDBN} has been started with the machine interface
> +interpreter (@pxref{GDB/MI, ,The @sc{gdb/mi} Interface}), most of
> +these commands will fail with an error, because it would not be
> +possible or desirable to enable curses window management.

That's fine, thanks.