Re: [RFA/commit] configure: search waddstr only when building with TUI

[Joel Brobecker <brobecker@adacore.com>]

[-- Attachment #1: Type: text/plain, Size: 2749 bytes --]

New version of the patch resent, with the following modifications:

  - Remove the use of brobecker/<date> in the comment. Seems
    like not everyone thinks it's useful. It doesn't bring much
    value in specific this case anyways, so removed.

  - Remove the quoting of parts of a C header from Tru64, to avoid
    copyright issues.

The suggestions where made by Tom. Thank you, Tom, for the feedback.

> This one has been annoying us for a long time before we finally
> found the source of the problem.  It happens only on Tru64, and
> the symptoms are a screwed terminal after GDB has started (missing
> carriage returns).  The source of the problem is the following:
> 
>   * readline's configure determines that libtermcap is sufficient
>     for its purposes, and thus includes termcap headers, etc.
> 
>   * GDB's configure has the following search:
>      # For the TUI, we need enhanced curses functionality.
>      [...]
>      AC_SEARCH_LIBS(waddstr, [ncurses cursesX curses])
>     This search is done even when TUI is not requested, and so
>     causes GDB to be compiled with libcurses.
> 
> On Tru64, unfortunately, the two libraries are not compatible.
> I provide a little more details in the patch itself.
> 
> Ideally, we would want to adapt the build such that readline and
> GDB end up using the same library.  But I haven't seen any other
> system but Tru64 where this is a problem, and fixing this seems
> a little tricky to coordinate.
> 
> I side-stepped the issue and declared gdb with TUI broken on Tru64,
> and adjusted the configure to search for waddstr only when TUI was
> requested (enable_tui either "yes" or "auto").  On Tru64, we error
> out if TUI is requested, or disable it if "auto" was selected. The
> nice consequence of this change is that this should get rid of
> the dependency on libcurses on the platforms where its inclusion
> was useless...
> 
> 2008-10-02  Joel Brobecker  <brobecker@adacore.com>
> 
>         * configure.ac: On alpha-osf, error out if enable_tui is set to
>         "yes", and set enable_tui to "no" if previously set to "auto".
>         Check for waddstr only if TUI support was requested. Move the
>         part of the configure script that updates various Makefile
>         variables up, together with the check for waddstr.
>         * configure: Regenerate.
> 
> The patch looks big, but I'm just moving things around inside an if
> (except for the part were I error-out if TUI is requested on Tru64).
> Tested on x86-linux, no regression. Tested on tru64 as well, but I
> don't remember whether I ran the testsuite or not. I did verify
> that GDB is now linked with libtermcap rater than libcurses.
> 
> Any objection? I'd like to commit this in about a week.

-- 
Joel

[-- Attachment #2: osf-tui.diff --]
[-- Type: text/plain, Size: 4779 bytes --]

diff -r 45c21f56c590 -r 9ac7e94abc9f configure.ac
--- a/configure.ac	Thu Oct 02 16:18:27 2008 -0700
+++ b/configure.ac	Fri Oct 03 14:19:59 2008 -0700
@@ -429,19 +429,69 @@ AC_SEARCH_LIBS(socketpair, socket)
 # Link in zlib if we can.  This allows us to read compressed debug sections.
 AC_SEARCH_LIBS(zlibVersion, z, [AC_CHECK_HEADERS(zlib.h)])
 
-# For the TUI, we need enhanced curses functionality.
-#
-# FIXME: kettenis/20040905: We prefer ncurses over the vendor-supplied
-# curses library because the latter might not provide all the
-# functionality we need.  However, this leads to problems on systems
-# where the linker searches /usr/local/lib, but the compiler doesn't
-# search /usr/local/include, if ncurses is installed in /usr/local.  A
-# default installation of ncurses on alpha*-dec-osf* will lead to such
-# a situation.
-AC_SEARCH_LIBS(waddstr, [ncurses cursesX curses])
-
 # On HP/UX we may need libxpdl for dlgetmodinfo (used by solib-pa64.c).
 AC_SEARCH_LIBS(dlgetmodinfo, [dl xpdl])
+
+# On alpha-osf, it appears that libtermcap and libcurses are not compatible.
+# There is a very specific comment in /usr/include/curses.h explaining that
+# termcap routines built into libcurses must not be used.
+# 
+# The symptoms we observed so far is GDB unexpectedly changing
+# the terminal settings when tgetent is called - this is particularly
+# visible as the output is missing carriage returns, and so rapidly
+# becomes very hard to read.
+#
+# The readline configure script has already decided that libtermcap
+# was enough for its purposes, and so decided to build readline using
+# libtermcap.  Since the TUI mode requires curses, building GDB with
+# TUI enabled results in both libraries to be used at the same time,
+# which is not allowed.  This basically means that GDB with TUI is
+# broken on alpha-osf.
+
+case $host_os in
+  alpha*-*-osf* )
+    if "$enable_tui" = "yes"; then
+      AC_MSG_ERROR([Building GDB with TUI mode is not supported on this host])
+    fi
+    if "$enable_tui" = "auto"; then
+      enable_tui=no
+    fi
+    ;;
+esac
+
+# Check whether we should enable the TUI, but only do so if we really
+# can.
+if test x"$enable_tui" != xno; then
+  if test -d $srcdir/tui; then
+    # For the TUI, we need enhanced curses functionality.
+    #
+    # FIXME: kettenis/20040905: We prefer ncurses over the vendor-supplied
+    # curses library because the latter might not provide all the
+    # functionality we need.  However, this leads to problems on systems
+    # where the linker searches /usr/local/lib, but the compiler doesn't
+    # search /usr/local/include, if ncurses is installed in /usr/local.  A
+    # default installation of ncurses on alpha*-dec-osf* will lead to such
+    # a situation.
+    AC_SEARCH_LIBS(waddstr, [ncurses cursesX curses])
+
+    if test "$ac_cv_search_waddstr" != no; then
+      CONFIG_OBS="$CONFIG_OBS \$(SUBDIR_TUI_OBS)"
+      CONFIG_DEPS="$CONFIG_DEPS \$(SUBDIR_TUI_DEPS)"
+      CONFIG_SRCS="$CONFIG_SRCS \$(SUBDIR_TUI_SRCS)"
+      ENABLE_CFLAGS="$ENABLE_CFLAGS \$(SUBDIR_TUI_CFLAGS)"
+      CONFIG_ALL="${CONFIG_ALL} all-tui"
+      CONFIG_CLEAN="${CONFIG_CLEAN} clean-tui"
+      CONFIG_INSTALL="${CONFIG_INSTALL} install-tui"
+      CONFIG_UNINSTALL="${CONFIG_UNINSTALL} uninstall-tui"
+    else
+      if test x"$enable_tui" = xyes; then
+	AC_MSG_ERROR([no enhanced curses library found; disable TUI])
+      else
+	AC_MSG_WARN([no enhanced curses library found; disabling TUI])
+      fi
+    fi
+  fi
+fi
 
 # Since GDB uses Readline, we need termcap functionality.  In many
 # cases this will be provided by the curses library, but some systems
@@ -1748,29 +1798,6 @@ AC_SUBST(GDBTK_SRC_DIR)
 
 AC_PATH_X
 
-# Check whether we should enable the TUI, but only do so if we really
-# can.
-if test x"$enable_tui" != xno; then
-  if test -d $srcdir/tui; then
-    if test "$ac_cv_search_waddstr" != no; then
-      CONFIG_OBS="$CONFIG_OBS \$(SUBDIR_TUI_OBS)"
-      CONFIG_DEPS="$CONFIG_DEPS \$(SUBDIR_TUI_DEPS)"
-      CONFIG_SRCS="$CONFIG_SRCS \$(SUBDIR_TUI_SRCS)"
-      ENABLE_CFLAGS="$ENABLE_CFLAGS \$(SUBDIR_TUI_CFLAGS)"
-      CONFIG_ALL="${CONFIG_ALL} all-tui"
-      CONFIG_CLEAN="${CONFIG_CLEAN} clean-tui"
-      CONFIG_INSTALL="${CONFIG_INSTALL} install-tui"
-      CONFIG_UNINSTALL="${CONFIG_UNINSTALL} uninstall-tui"
-    else
-      if test x"$enable_tui" = xyes; then
-	AC_MSG_ERROR([no enhanced curses library found; disable TUI])
-      else
-	AC_MSG_WARN([no enhanced curses library found; disabling TUI])
-      fi
-    fi
-  fi
-fi
-
 # Unlike the sim directory, whether a simulator is linked is controlled by 
 # presence of a gdb_sim definition in the target configure.tgt entry.  
 # This code just checks for a few cases where we'd like to ignore those