[PATCH] [gdb/tui] Handle error in tui_enable

[Tom de Vries <tdevries@suse.de>]

Say we simulate an error:
...
static void error_once () {
  static int v = 0;
  if (v == 1)
    return;

  v = 1;
  error (_("Oh no!!!"));
}
...
in the call to tui_set_initial_layout in tui_enable:
...
       tui_show_frame_info (deprecated_safe_get_selected_frame ());
+      error_once ();
       tui_set_initial_layout ();
...

After doing "tui enable"
...
$ gdb
(gdb) tui enable
...
the screen is cleared, and we have:
...
❌️ Oh no!!!
           (gdb) <blinking cursor>
...

After typing "apropos tui" (which is not echoed) and pressing enter, I run into
a segmentation violation here in tui_inject_newline_into_command_window:
...
    at /data/vries/gdb/leap-16-0/build/../../src/gdb/tui/tui-io.c:1084
1084	  WINDOW *w = tui_cmd_win ()->handle.get ();
...
because:
...
$2 = (tui_cmd_window *) 0x0
(gdb) p tui_cmd_win ()
...

The problem is that tui_active is true, and so
tui_inject_newline_into_command_window get called:
...
static void
tui_command_line_handler (gdb::unique_xmalloc_ptr<char> &&rl)
{
 ...
  if (tui_active)
    tui_inject_newline_into_command_window ();
 ...
}
...

Fix this by catching the error in tui_enable, and resetting tui_active back to
false.

While this fixes the segmentation fault, and does allow "apropos tui" to run,
still the command is not echoed, and the output of the command is garbled by
runaway indentation.

Fix this by using endwin / delscreen, as borrowed from earlier in tui_enable:
...
      if (cap == NULL || cap == (char *) -1 || *cap == '\0')
        {
          endwin ();
          delscreen (s);
          error (_("Cannot enable the TUI: "
                   "terminal doesn't support cursor addressing [TERM=%s]"),
                 gdb_getenv_term ());
        }
...

Note that this doesn't allow a second attempt:
...
$ gdb -q
(gdb) tui enable
❌️ Oh no!!!
(gdb) tui enable
❌️ Cannot enable the TUI
(gdb)
...
because tui_finish_init is stuck at TRIBOOL_UNKNOWN.

IWBN to fix this in a way that allows the second "tui enable" to succeed.  I
tried this for a bit, but didn't get it to work.

Tested on x86_64-linux.

Suggested-By: Tom Tromey <tom@tromey.com> [1]

Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=34100

[1] https://sourceware.org/pipermail/gdb-patches/2025-May/217660.html
---
 gdb/tui/tui.c | 39 ++++++++++++++++++++++++---------------
 1 file changed, 24 insertions(+), 15 deletions(-)

diff --git a/gdb/tui/tui.c b/gdb/tui/tui.c
index 5e39eef71c6..9fef7feffe5 100644
--- a/gdb/tui/tui.c
+++ b/gdb/tui/tui.c
@@ -490,22 +490,31 @@ tui_enable (void)
 	 rely on this flag being true in order to know that the window
 	 they are creating is currently valid.  */
       tui_active = true;
+      try
+	{
+	  cbreak ();
+	  noecho ();
+	  /* timeout (1); */
+	  nodelay (w, FALSE);
+	  nl ();
+	  keypad (w, TRUE);
+	  tui_set_term_height_to (LINES);
+	  tui_set_term_width_to (COLS);
+	  def_prog_mode ();
+	  tui_show_frame_info (deprecated_safe_get_selected_frame ());
+	  tui_set_initial_layout ();
+	  tui_set_win_focus_to (tui_src_win ());
+	  keypad (tui_cmd_win ()->handle.get (), TRUE);
+	  wrefresh (tui_cmd_win ()->handle.get ());
+	}
+      catch (const gdb_exception &)
+	{
+	  endwin ();
+	  delscreen (s);
+	  tui_active = false;
+	  throw;
+	}
 
-      cbreak ();
-      noecho ();
-      /* timeout (1); */
-      nodelay(w, FALSE);
-      nl();
-      keypad (w, TRUE);
-      tui_set_term_height_to (LINES);
-      tui_set_term_width_to (COLS);
-      def_prog_mode ();
-
-      tui_show_frame_info (deprecated_safe_get_selected_frame ());
-      tui_set_initial_layout ();
-      tui_set_win_focus_to (tui_src_win ());
-      keypad (tui_cmd_win ()->handle.get (), TRUE);
-      wrefresh (tui_cmd_win ()->handle.get ());
       tui_finish_init = TRIBOOL_FALSE;
     }
   else

base-commit: 4308969628943757f972e766c70352c4828cb4d2
-- 
2.51.0