Re: Make the "python" command resemble the standard Python interpreter

[Khoo Yit Phang <khooyp@cs.umd.edu>]

[-- Attachment #1: Type: text/plain, Size: 1739 bytes --]

Hi,

On Jan 11, 2012, at 2:03 PM, Kevin Pouget wrote:

> On Wed, Jan 11, 2012 at 7:48 PM, Khoo Yit Phang <khooyp@cs.umd.edu> wrote:
>> 
>> It also hooks GDB's readline wrappers (command_line_input) to Python so that line editing > and history editing works. Additionally, Python's standard readline module is stubbed out
>> because it conflicts with GDB's use of readline.
> 
> that's interesting! I noticed that there's conflict between Python and
> GDB readline; do you thing that this hook could also be used to fix
> PDB's commandline prompt? (when you run "python import
> pdb;pdb.set_trace()", history and left/right arrow don't work
> properly)

Unfortunately no, and actually, my previous patch causes pdb.set_trace() to not work at all since since my dummy readline methods just throws exceptions.

Attached is a new patch that uses another way to disable the readline module, which works with pdb.set_trace(). However, readline support still doesn't work with pdb. The reason is because pdb uses raw_input, which in turn uses sys.stdin/sys.stdout to determine whether to use readline, but GDB replaces sys.stdin/sys.stdout with it's own file-like objects that isn't recognized as a tty by Python.

It should be possible to replace raw_input with one that works with GDB, but that can be another patch.

Making Python's readline module work under GDB is not possible, since it re-initializes libreadline and generally assumes that libreadline is completely under Python's control (and libreadline itself has no support for nesting/reentrancy).

It should also be possible to write a wrapper module around readline that avoids conflicts with GDB, but that'll take some more work.

Yit
January 11, 2012


[-- Attachment #2: python-interactive.diff --]
[-- Type: application/octet-stream, Size: 9747 bytes --]

# HG changeset patch
# Parent 9f7d0a74b37f443edf5a7446fcadc806aea32f21
Make the "python" command resemble the standard Python interpreter.
- Use Py_single_input mode to interpret "python" with arguments, so that results of expressions are printed.
- Use Python's built-in interactive loop for "python" without arguments.
- Hook PyOS_ReadlineFunctionPointer to command_line_input to provide readline support.
- Block importing of the standard Python readline module using a sys.meta_path loader to prevent conflicts with GDB.

gdb/ChangeLog:

2012-01-11  Khoo Yit Phang <khooyp@cs.umd.edu>

	Make the "python" command resemble the standard Python interpreter.
	* Makefile.in (SUBDIR_PYTHON_OBS): Add py-gdb-readline.o.
	(SUBDIR_PYTHON_SRCS): Add python/py-gdb-readline.c.
	(py-gdb-readline.o): Add rule to compile python/py-gdb-readline.c.
	* python/py-gdb-readline.c: New file.
	* python/python-internal.h (gdbpy_initialize_gdb_readline): New
	prototype.
	* python/python.c (eval_python_command): New function.
	(eval_python_from_control_command): Call eval_python_command instead
	of PyRun_SimpleString.
	(python_command): For "python" with arguments, call
	eval_python_command.  For "python" without arguments, call
	PyRun_InteractiveLoop if from_tty is true, other call
	execute_control_command_untraced.

diff --git a/gdb/Makefile.in b/gdb/Makefile.in
--- a/gdb/Makefile.in
+++ b/gdb/Makefile.in
@@ -283,6 +283,7 @@
 	py-finishbreakpoint.o \
 	py-frame.o \
 	py-function.o \
+	py-gdb-readline.o \
 	py-inferior.o \
 	py-infthread.o \
 	py-lazy-string.o \
@@ -315,6 +316,7 @@
 	python/py-finishbreakpoint.c \
 	python/py-frame.c \
 	python/py-function.c \
+	python/py-gdb-readline.c \
 	python/py-inferior.c \
 	python/py-infthread.c \
 	python/py-lazy-string.c \
@@ -2071,6 +2073,10 @@
 	$(COMPILE) $(PYTHON_CFLAGS) $(srcdir)/python/py-function.c
 	$(POSTCOMPILE)
 
+py-gdb-readline.o: $(srcdir)/python/py-gdb-readline.c
+	$(COMPILE) $(PYTHON_CFLAGS) $(srcdir)/python/py-gdb-readline.c
+	$(POSTCOMPILE)
+
 py-inferior.o: $(srcdir)/python/py-inferior.c
 	$(COMPILE) $(PYTHON_CFLAGS) $(srcdir)/python/py-inferior.c
 	$(POSTCOMPILE)
diff --git a/gdb/python/py-gdb-readline.c b/gdb/python/py-gdb-readline.c
new file mode 100644
--- /dev/null
+++ b/gdb/python/py-gdb-readline.c
@@ -0,0 +1,133 @@
+/* Readline support for Python.
+
+   Copyright (C) 2012 Free Software Foundation, Inc.
+
+   This file is part of GDB.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include "defs.h"
+#include "python-internal.h"
+#include "exceptions.h"
+#include "top.h"
+#include "cli/cli-utils.h"
+#include "gdb_string.h"
+
+#include <stddef.h>
+
+/* Readline function suitable for PyOS_ReadlineFunctionPointer, which
+   is used for Python's interactive parser and raw_input.  In both
+   cases, sys_stdin and sys_stdout are always stdin and stdout
+   respectively, as far as I can tell; they are ignored and
+   command_line_input is used instead.  */
+
+static char *
+gdbpy_readline_wrapper (FILE *sys_stdin, FILE *sys_stdout,
+			char *prompt)
+{
+  int n;
+  char *p = NULL, *p_start, *p_end, *q;
+  volatile struct gdb_exception except;
+
+  TRY_CATCH (except, RETURN_MASK_ALL)
+    {
+      struct cleanup *cleanup = gdbpy_suspend_sigint_handler ();
+
+      p = command_line_input (prompt, 0, "python");
+      do_cleanups (cleanup);
+    }
+
+  /* Detect Ctrl-C and treat as KeyboardInterrupt.  */
+  if (except.reason == RETURN_QUIT)
+    return NULL;
+
+  /* Handle errors by raising Python exceptions.  */
+  if (except.reason < 0)
+    {
+      /* The thread state is nulled during gdbpy_readline_wrapper,
+	 with the original value saved in the following undocumented
+	 variable (see Python's Parser/myreadline.c and
+	 Modules/readline.c).  */
+      PyEval_RestoreThread (_PyOS_ReadlineTState);
+      gdbpy_convert_exception (except);
+      PyEval_SaveThread ();
+      return NULL;
+    }
+
+  /* Detect EOF (Ctrl-D).  */
+  if (p == NULL)
+    {
+      q = PyMem_Malloc (1);
+      if (q != NULL)
+	q[0] = '\0';
+      return q;
+    }
+
+  n = strlen (p);
+
+  /* Detect "end" like process_next_line in cli/cli-script.c.  */
+  p_start = remove_trailing_whitespace (p, p + n);
+  p_end = skip_spaces (p_start);
+  while (p_start < p_end && (*p_start == ' ' || *p_start == '\t'))
+    p_start++;
+
+  /* Treat "end" as EOF.  */
+  if (p_end - p_start == 3 && !strncmp (p_start, "end", 3))
+    {
+      q = PyMem_Malloc (1);
+      if (q != NULL)
+	q[0] = '\0';
+      return q;
+    }
+
+  /* Copy the line to Python and return.  */
+  q = PyMem_Malloc (n + 2);
+  if (q != NULL)
+    {
+      strncpy (q, p, n);
+      q[n] = '\n';
+      q[n + 1] = '\0';
+    }
+  return q;
+}
+
+/* Initialize Python readline support.  */
+
+void
+gdbpy_initialize_gdb_readline (void)
+{
+  /* Python's readline module conflicts with GDB's use of readline
+     since readline is not reentrant.  Ideally, a reentrant wrapper to
+     GDB's readline should be implemented to replace Python's readline
+     and prevent conflicts.  For now, this file implements a
+     sys.meta_path finder that simply fails to import the readline
+     module.  */
+  PyRun_SimpleString ("\
+import sys\n\
+\n\
+class GdbRemoveReadlineFinder:\n\
+  def find_module(self, fullname, path=None):\n\
+    if fullname == 'readline' and path is None:\n\
+      return self\n\
+    return None\n\
+\n\
+  def load_module(self, fullname):\n\
+    raise ImportError('readline module disabled under GDB')\n\
+\n\
+sys.meta_path.append(GdbRemoveReadlineFinder())\n\
+");
+
+  PyOS_ReadlineFunctionPointer = gdbpy_readline_wrapper;
+}
+
diff --git a/gdb/python/python-internal.h b/gdb/python/python-internal.h
--- a/gdb/python/python-internal.h
+++ b/gdb/python/python-internal.h
@@ -234,6 +234,7 @@
 struct symtab_and_line *sal_object_to_symtab_and_line (PyObject *obj);
 struct frame_info *frame_object_to_frame_info (PyObject *frame_obj);
 
+void gdbpy_initialize_gdb_readline (void);
 void gdbpy_initialize_auto_load (void);
 void gdbpy_initialize_values (void);
 void gdbpy_initialize_frames (void);
diff --git a/gdb/python/python.c b/gdb/python/python.c
--- a/gdb/python/python.c
+++ b/gdb/python/python.c
@@ -279,6 +279,44 @@
   return script;
 }
 
+/* Evaluate a Python command like PyRun_SimpleString, but uses
+   Py_single_input which prints the result of expressions if the input
+   is from tty, and Py_file_input otherwise.  */
+
+static void
+eval_python_command (const char *command, int from_tty)
+{
+  struct cleanup *cleanup;
+  PyObject *m, *d, *v;
+
+  cleanup = ensure_python_env (get_current_arch (), current_language);
+
+  m = PyImport_AddModule ("__main__");
+  if (m == NULL)
+    error (_("Error while executing Python code."));
+
+  d = PyModule_GetDict (m);
+  v = PyRun_StringFlags (command,
+			 from_tty ? Py_single_input : Py_file_input,
+			 d, d, NULL);
+  if (v == NULL)
+    {
+      int interrupt = PyErr_ExceptionMatches (PyExc_KeyboardInterrupt);
+
+      PyErr_Print ();
+      if (! interrupt)
+	error (_("Error while executing Python code."));
+    }
+  else
+    {
+      Py_DECREF (v);
+      if (Py_FlushLine ())
+	PyErr_Clear ();
+    }
+
+  do_cleanups (cleanup);
+}
+
 /* Take a command line structure representing a 'python' command, and
    evaluate its body using the Python interpreter.  */
 
@@ -292,13 +330,10 @@
   if (cmd->body_count != 1)
     error (_("Invalid \"python\" block structure."));
 
-  cleanup = ensure_python_env (get_current_arch (), current_language);
+  script = compute_python_string (cmd->body_list[0]);
+  cleanup = make_cleanup (xfree, script);
 
-  script = compute_python_string (cmd->body_list[0]);
-  ret = PyRun_SimpleString (script);
-  xfree (script);
-  if (ret)
-    error (_("Error while executing Python code."));
+  eval_python_command (script, 0);
 
   do_cleanups (cleanup);
 }
@@ -316,18 +351,26 @@
   while (arg && *arg && isspace (*arg))
     ++arg;
   if (arg && *arg)
-    {
-      ensure_python_env (get_current_arch (), current_language);
-
-      if (PyRun_SimpleString (arg))
-	error (_("Error while executing Python code."));
-    }
+    eval_python_command (arg, from_tty);
   else
     {
-      struct command_line *l = get_command_line (python_control, "");
+      if (from_tty)
+	{
+	  int err;
 
-      make_cleanup_free_command_lines (&l);
-      execute_control_command_untraced (l);
+	  ensure_python_env (get_current_arch (), current_language);
+	  err = PyRun_InteractiveLoop(instream, "<stdin>");
+	  dont_repeat ();
+	  if (err && ! PyErr_ExceptionMatches (PyExc_KeyboardInterrupt))
+	    error(_("Error while executing Python code."));
+	}
+      else
+	{
+	  struct command_line *l = get_command_line (python_control, "");
+
+	  make_cleanup_free_command_lines (&l);
+	  execute_control_command_untraced (l);
+	}
     }
 
   do_cleanups (cleanup);
@@ -1280,6 +1323,7 @@
   gdbpy_gdberror_exc = PyErr_NewException ("gdb.GdbError", NULL, NULL);
   PyModule_AddObject (gdb_module, "GdbError", gdbpy_gdberror_exc);
 
+  gdbpy_initialize_gdb_readline ();
   gdbpy_initialize_auto_load ();
   gdbpy_initialize_values ();
   gdbpy_initialize_frames ();