[PATCH 4/4]: Enable readline under "python"

[PATCH 4/4]: Enable readline under "python"

[Khoo Yit Phang]

[-- Attachment #1: Type: text/plain, Size: 817 bytes --]

Hi,

Here's the last Python patch: this enables readline support under "python" using a GDB-specific mechanism, and disables the standard Python readline module.

This is somewhat of a workaround: unfortunately, there is no easy way to make GDB and Python use libreadline without conflicting with each other, since libreadline is configured essentially using global variables. Using the standard Python readline module will lead to strange conflicts, e.g., changing key bindings or tab-completion. I've thought of two possible solutions which are both rather complicated: 1) either reimplement the readline module that carefully swaps configurations between GDB and Python; 2) or run Python's readline in a separate controlling terminal with a proxy readline module in GDB.

Thank you,

Yit
July 16, 2012


[-- Attachment #2: python-enable-readline --]
[-- Type: application/octet-stream, Size: 6127 bytes --]

# HG changeset patch
# Parent a7e89526afbe05cf717e55ca3a4bf3492efa3001
Provide a GDB-specific readline in Python, blocking the standard Python readline module using a sys.meta_path loader to prevent conflicts with GDB.

gdb/ChangeLog:

2012-07-16  Khoo Yit Phang <khooyp@cs.umd.edu>

	Enable readline in Python in a GDB-specific way and block the
	standard Python readline module to prevent conflicts with GDB.
	* Makefile.in (SUBDIR_PYTHON_OBS): Add py-gdb-readline.o.
	(SUBDIR_PYTHON_SRCS): Add python/py-gdb-readline.c.
	(py-gdb-readline.o): Add rule to compile python/py-gdb-readline.c.
	* python/py-gdb-readline.c: New file.
	* python/python-internal.h (gdbpy_initialize_gdb_readline): New
	prototype.
	* python/python.c (_initialize_python): Call
	gdbpy_initialize_gdb_readline.

diff --git a/gdb/Makefile.in b/gdb/Makefile.in
--- a/gdb/Makefile.in
+++ b/gdb/Makefile.in
@@ -280,6 +280,7 @@
 	py-finishbreakpoint.o \
 	py-frame.o \
 	py-function.o \
+	py-gdb-readline.o \
 	py-inferior.o \
 	py-infthread.o \
 	py-lazy-string.o \
@@ -312,6 +313,7 @@
 	python/py-finishbreakpoint.c \
 	python/py-frame.c \
 	python/py-function.c \
+	python/py-gdb-readline.c \
 	python/py-inferior.c \
 	python/py-infthread.c \
 	python/py-lazy-string.c \
@@ -2082,6 +2084,10 @@
 	$(COMPILE) $(PYTHON_CFLAGS) $(srcdir)/python/py-function.c
 	$(POSTCOMPILE)
 
+py-gdb-readline.o: $(srcdir)/python/py-gdb-readline.c
+	$(COMPILE) $(PYTHON_CFLAGS) $(srcdir)/python/py-gdb-readline.c
+	$(POSTCOMPILE)
+
 py-inferior.o: $(srcdir)/python/py-inferior.c
 	$(COMPILE) $(PYTHON_CFLAGS) $(srcdir)/python/py-inferior.c
 	$(POSTCOMPILE)
diff --git a/gdb/python/py-gdb-readline.c b/gdb/python/py-gdb-readline.c
new file mode 100644
--- /dev/null
+++ b/gdb/python/py-gdb-readline.c
@@ -0,0 +1,118 @@
+/* Readline support for Python.
+
+   Copyright (C) 2012 Free Software Foundation, Inc.
+
+   This file is part of GDB.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include "defs.h"
+#include "python-internal.h"
+#include "exceptions.h"
+#include "top.h"
+#include "cli/cli-utils.h"
+#include "gdb_string.h"
+
+#include <stddef.h>
+
+/* Readline function suitable for PyOS_ReadlineFunctionPointer, which
+   is used for Python's interactive parser and raw_input.  In both
+   cases, sys_stdin and sys_stdout are always stdin and stdout
+   respectively, as far as I can tell; they are ignored and
+   command_line_input is used instead.  */
+
+static char *
+gdbpy_readline_wrapper (FILE *sys_stdin, FILE *sys_stdout,
+			char *prompt)
+{
+  int n;
+  char *p = NULL, *p_start, *p_end, *q;
+  volatile struct gdb_exception except;
+
+  TRY_CATCH (except, RETURN_MASK_ALL)
+    {
+      struct cleanup *cleanup = gdbpy_suspend_sigint_handler ();
+
+      p = command_line_input (prompt, 0, "python");
+      do_cleanups (cleanup);
+    }
+
+  /* Detect user interrupt (Ctrl-C).  */
+  if (except.reason == RETURN_QUIT)
+    return NULL;
+
+  /* Handle errors by raising Python exceptions.  */
+  if (except.reason < 0)
+    {
+      /* The thread state is nulled during gdbpy_readline_wrapper,
+	 with the original value saved in the following undocumented
+	 variable (see Python's Parser/myreadline.c and
+	 Modules/readline.c).  */
+      PyEval_RestoreThread (_PyOS_ReadlineTState);
+      gdbpy_convert_exception (except);
+      PyEval_SaveThread ();
+      return NULL;
+    }
+
+  /* Detect EOF (Ctrl-D).  */
+  if (p == NULL)
+    {
+      q = PyMem_Malloc (1);
+      if (q != NULL)
+	q[0] = '\0';
+      return q;
+    }
+
+  n = strlen (p);
+
+  /* Copy the line to Python and return.  */
+  q = PyMem_Malloc (n + 2);
+  if (q != NULL)
+    {
+      strncpy (q, p, n);
+      q[n] = '\n';
+      q[n + 1] = '\0';
+    }
+  return q;
+}
+
+/* Initialize Python readline support.  */
+
+void
+gdbpy_initialize_gdb_readline (void)
+{
+  /* Python's readline module conflicts with GDB's use of readline
+     since readline is not reentrant.  Ideally, a reentrant wrapper to
+     GDB's readline should be implemented to replace Python's readline
+     and prevent conflicts.  For now, this file implements a
+     sys.meta_path finder that simply fails to import the readline
+     module.  */
+  PyRun_SimpleString ("\
+import sys\n\
+\n\
+class GdbRemoveReadlineFinder:\n\
+  def find_module(self, fullname, path=None):\n\
+    if fullname == 'readline' and path is None:\n\
+      return self\n\
+    return None\n\
+\n\
+  def load_module(self, fullname):\n\
+    raise ImportError('readline module disabled under GDB')\n\
+\n\
+sys.meta_path.append(GdbRemoveReadlineFinder())\n\
+");
+
+  PyOS_ReadlineFunctionPointer = gdbpy_readline_wrapper;
+}
+
diff --git a/gdb/python/python-internal.h b/gdb/python/python-internal.h
--- a/gdb/python/python-internal.h
+++ b/gdb/python/python-internal.h
@@ -235,6 +235,7 @@
 struct symtab_and_line *sal_object_to_symtab_and_line (PyObject *obj);
 struct frame_info *frame_object_to_frame_info (PyObject *frame_obj);
 
+void gdbpy_initialize_gdb_readline (void);
 void gdbpy_initialize_auto_load (void);
 void gdbpy_initialize_values (void);
 void gdbpy_initialize_frames (void);
diff --git a/gdb/python/python.c b/gdb/python/python.c
--- a/gdb/python/python.c
+++ b/gdb/python/python.c
@@ -1403,6 +1403,7 @@
   gdbpy_gdberror_exc = PyErr_NewException ("gdb.GdbError", NULL, NULL);
   PyModule_AddObject (gdb_module, "GdbError", gdbpy_gdberror_exc);
 
+  gdbpy_initialize_gdb_readline ();
   gdbpy_initialize_auto_load ();
   gdbpy_initialize_values ();
   gdbpy_initialize_frames ();

Re: [PATCH 4/4]: Enable readline under "python"

[Tom Tromey]

>>>>> "Yit" == Khoo Yit Phang <khooyp@cs.umd.edu> writes:

Yit> 1) either reimplement the readline module that carefully swaps
Yit> configurations between GDB and Python; 2) or run Python's readline in
Yit> a separate controlling terminal with a proxy readline module in GDB.

#2 seems extremely hard.
Maybe #1 is possible, I don't know.

Yit> 2012-07-16  Khoo Yit Phang <khooyp@cs.umd.edu>
Yit> 	Enable readline in Python in a GDB-specific way and block the
Yit> 	standard Python readline module to prevent conflicts with GDB.
Yit> 	* Makefile.in (SUBDIR_PYTHON_OBS): Add py-gdb-readline.o.
Yit> 	(SUBDIR_PYTHON_SRCS): Add python/py-gdb-readline.c.
Yit> 	(py-gdb-readline.o): Add rule to compile python/py-gdb-readline.c.
Yit> 	* python/py-gdb-readline.c: New file.
Yit> 	* python/python-internal.h (gdbpy_initialize_gdb_readline): New
Yit> 	prototype.
Yit> 	* python/python.c (_initialize_python): Call
Yit> 	gdbpy_initialize_gdb_readline.

This one is ok; but of course it depends on an earlier patch so it can't
go in until that is resolved.

Tom

Re: [PATCH 4/4]: Enable readline under "python"

[Khoo Yit Phang]

[-- Attachment #1: Type: text/plain, Size: 255 bytes --]

Hi,

I've updated my Python/GDB readline patch for the new SIGINT handling patch that you've just checked in, Tom. The only change is that there's no longer any need to swap SIGINT handlers. If it's alright, I'll check it in.

Yit
August 22, 2012


[-- Attachment #2: python-enable-readline.txt --]
[-- Type: text/plain, Size: 6202 bytes --]

# HG changeset patch
# Parent 9cf27242c2eadcf990d1dde46c20b21111d837fc
Provide a GDB-specific readline in Python, blocking the standard Python readline module using a sys.meta_path loader to prevent conflicts with GDB.

gdb/ChangeLog:

2012-08-22  Khoo Yit Phang <khooyp@cs.umd.edu>

	Enable readline in Python in a GDB-specific way and block the
	standard Python readline module to prevent conflicts with GDB.
	* Makefile.in (SUBDIR_PYTHON_OBS): Add py-gdb-readline.o.
	(SUBDIR_PYTHON_SRCS): Add python/py-gdb-readline.c.
	(py-gdb-readline.o): Add rule to compile python/py-gdb-readline.c.
	* python/py-gdb-readline.c: New file.
	* python/python-internal.h (gdbpy_initialize_gdb_readline): New
	prototype.
	* python/python.c (_initialize_python): Call
	gdbpy_initialize_gdb_readline.

diff --git a/gdb/Makefile.in b/gdb/Makefile.in
--- a/gdb/Makefile.in
+++ b/gdb/Makefile.in
@@ -280,6 +280,7 @@
 	py-finishbreakpoint.o \
 	py-frame.o \
 	py-function.o \
+	py-gdb-readline.o \
 	py-inferior.o \
 	py-infthread.o \
 	py-lazy-string.o \
@@ -312,6 +313,7 @@
 	python/py-finishbreakpoint.c \
 	python/py-frame.c \
 	python/py-function.c \
+	python/py-gdb-readline.c \
 	python/py-inferior.c \
 	python/py-infthread.c \
 	python/py-lazy-string.c \
@@ -2083,6 +2085,10 @@
 	$(COMPILE) $(PYTHON_CFLAGS) $(srcdir)/python/py-function.c
 	$(POSTCOMPILE)
 
+py-gdb-readline.o: $(srcdir)/python/py-gdb-readline.c
+	$(COMPILE) $(PYTHON_CFLAGS) $(srcdir)/python/py-gdb-readline.c
+	$(POSTCOMPILE)
+
 py-inferior.o: $(srcdir)/python/py-inferior.c
 	$(COMPILE) $(PYTHON_CFLAGS) $(srcdir)/python/py-inferior.c
 	$(POSTCOMPILE)
diff --git a/gdb/python/py-gdb-readline.c b/gdb/python/py-gdb-readline.c
new file mode 100644
--- /dev/null
+++ b/gdb/python/py-gdb-readline.c
@@ -0,0 +1,113 @@
+/* Readline support for Python.
+
+   Copyright (C) 2012 Free Software Foundation, Inc.
+
+   This file is part of GDB.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include "defs.h"
+#include "python-internal.h"
+#include "exceptions.h"
+#include "top.h"
+#include "cli/cli-utils.h"
+#include "gdb_string.h"
+
+#include <stddef.h>
+
+/* Readline function suitable for PyOS_ReadlineFunctionPointer, which
+   is used for Python's interactive parser and raw_input.  In both
+   cases, sys_stdin and sys_stdout are always stdin and stdout
+   respectively, as far as I can tell; they are ignored and
+   command_line_input is used instead.  */
+
+static char *
+gdbpy_readline_wrapper (FILE *sys_stdin, FILE *sys_stdout,
+			char *prompt)
+{
+  int n;
+  char *p = NULL, *p_start, *p_end, *q;
+  volatile struct gdb_exception except;
+
+  TRY_CATCH (except, RETURN_MASK_ALL)
+    p = command_line_input (prompt, 0, "python");
+
+  /* Detect user interrupt (Ctrl-C).  */
+  if (except.reason == RETURN_QUIT)
+    return NULL;
+
+  /* Handle errors by raising Python exceptions.  */
+  if (except.reason < 0)
+    {
+      /* The thread state is nulled during gdbpy_readline_wrapper,
+	 with the original value saved in the following undocumented
+	 variable (see Python's Parser/myreadline.c and
+	 Modules/readline.c).  */
+      PyEval_RestoreThread (_PyOS_ReadlineTState);
+      gdbpy_convert_exception (except);
+      PyEval_SaveThread ();
+      return NULL;
+    }
+
+  /* Detect EOF (Ctrl-D).  */
+  if (p == NULL)
+    {
+      q = PyMem_Malloc (1);
+      if (q != NULL)
+	q[0] = '\0';
+      return q;
+    }
+
+  n = strlen (p);
+
+  /* Copy the line to Python and return.  */
+  q = PyMem_Malloc (n + 2);
+  if (q != NULL)
+    {
+      strncpy (q, p, n);
+      q[n] = '\n';
+      q[n + 1] = '\0';
+    }
+  return q;
+}
+
+/* Initialize Python readline support.  */
+
+void
+gdbpy_initialize_gdb_readline (void)
+{
+  /* Python's readline module conflicts with GDB's use of readline
+     since readline is not reentrant.  Ideally, a reentrant wrapper to
+     GDB's readline should be implemented to replace Python's readline
+     and prevent conflicts.  For now, this file implements a
+     sys.meta_path finder that simply fails to import the readline
+     module.  */
+  PyRun_SimpleString ("\
+import sys\n\
+\n\
+class GdbRemoveReadlineFinder:\n\
+  def find_module(self, fullname, path=None):\n\
+    if fullname == 'readline' and path is None:\n\
+      return self\n\
+    return None\n\
+\n\
+  def load_module(self, fullname):\n\
+    raise ImportError('readline module disabled under GDB')\n\
+\n\
+sys.meta_path.append(GdbRemoveReadlineFinder())\n\
+");
+
+  PyOS_ReadlineFunctionPointer = gdbpy_readline_wrapper;
+}
+
diff --git a/gdb/python/python-internal.h b/gdb/python/python-internal.h
--- a/gdb/python/python-internal.h
+++ b/gdb/python/python-internal.h
@@ -232,6 +232,7 @@
 struct symtab_and_line *sal_object_to_symtab_and_line (PyObject *obj);
 struct frame_info *frame_object_to_frame_info (PyObject *frame_obj);
 
+void gdbpy_initialize_gdb_readline (void);
 void gdbpy_initialize_auto_load (void);
 void gdbpy_initialize_values (void);
 void gdbpy_initialize_frames (void);
diff --git a/gdb/python/python.c b/gdb/python/python.c
--- a/gdb/python/python.c
+++ b/gdb/python/python.c
@@ -1389,6 +1389,7 @@
   gdbpy_gdberror_exc = PyErr_NewException ("gdb.GdbError", NULL, NULL);
   PyModule_AddObject (gdb_module, "GdbError", gdbpy_gdberror_exc);
 
+  gdbpy_initialize_gdb_readline ();
   gdbpy_initialize_auto_load ();
   gdbpy_initialize_values ();
   gdbpy_initialize_frames ();

Re: [PATCH 4/4]: Enable readline under "python"

[Tom Tromey]

>>>>> "Yit" == Khoo Yit Phang <khooyp@cs.umd.edu> writes:

Yit> Hi,
Yit> I've updated my Python/GDB readline patch for the new SIGINT handling
Yit> patch that you've just checked in, Tom. The only change is that
Yit> there's no longer any need to swap SIGINT handlers. If it's alright,
Yit> I'll check it in.

It is ok.  Thanks!

Tom

Re: [PATCH 4/4]: Enable readline under "python"

[Kevin Pouget]

On Wed, Aug 22, 2012 at 9:55 PM, Tom Tromey <tromey@redhat.com> wrote:
>
> >>>>> "Yit" == Khoo Yit Phang <khooyp@cs.umd.edu> writes:
>
> Yit> Hi,
> Yit> I've updated my Python/GDB readline patch for the new SIGINT handling
> Yit> patch that you've just checked in, Tom. The only change is that
> Yit> there's no longer any need to swap SIGINT handlers. If it's alright,
> Yit> I'll check it in.
>
> It is ok.  Thanks!
>
> Tom

hello,

it might be a stupid question, and I didn't take time to read the
patch ... but how do we exit from this `python-interactive` command?
exit() and quit() just kill the entire GDB ... If I didn't miss
something obvious, it might worth adding a line in the documentation.


(and just a detail, in the gdb/ChangeLog of this patch, I think that this line:
> * doc/gdb.texinfo (Python Commands): Document the new commands.
is supposted to go to doc/ChangeLog


and by the way this readline and interactive prompt is a *very* cool
job, GDB/Python will become really more intuitive!


Thanks,

Kevin

Re: [PATCH 4/4]: Enable readline under "python"

[Khoo Yit Phang]

Hi,

On Aug 23, 2012, at 7:20 AM, Kevin Pouget wrote:

> it might be a stupid question, and I didn't take time to read the
> patch ... but how do we exit from this `python-interactive` command?
> exit() and quit() just kill the entire GDB ... If I didn't miss
> something obvious, it might worth adding a line in the documentation.

That's a good question: Ctrl-D will get you out of python-interactive. I'll add that to the documentation later today.

Yit
August 23, 2012

Re: [PATCH 4/4]: Enable readline under "python"

[Kevin Pouget]

On Thu, Aug 23, 2012 at 3:04 PM, Khoo Yit Phang <khooyp@cs.umd.edu> wrote:
>
> Hi,
>
> On Aug 23, 2012, at 7:20 AM, Kevin Pouget wrote:
>
> > it might be a stupid question, and I didn't take time to read the
> > patch ... but how do we exit from this `python-interactive` command?
> > exit() and quit() just kill the entire GDB ... If I didn't miss
> > something obvious, it might worth adding a line in the documentation.
>
> That's a good question: Ctrl-D will get you out of python-interactive.
> I'll add that to the documentation later today.
>
> Yit
> August 23, 2012

ok, thanks

maybe you can also somehow add that the line must be empty, otherwise
it doesn't work :)

-- 
Kevin

Re: [PATCH 4/4]: Enable readline under "python"

[Khoo Yit Phang]

Hi,

On Aug 23, 2012, at 7:20 AM, Kevin Pouget wrote:

> On Wed, Aug 22, 2012 at 9:55 PM, Tom Tromey <tromey@redhat.com> wrote:
> 
> (and just a detail, in the gdb/ChangeLog of this patch, I think that this line:
>> * doc/gdb.texinfo (Python Commands): Document the new commands.
> is supposted to go to doc/ChangeLog

Oh, I hadn't realized that there is another ChangeLog. What's the procedure for correcting this? Shall I check in a patch that just moves the entries to the correct ChangeLog?

Yit
August 23, 2012

Re: [PATCH 4/4]: Enable readline under "python"

[Pedro Alves]

On 08/24/2012 02:42 AM, Khoo Yit Phang wrote:
> Hi,
> 
> On Aug 23, 2012, at 7:20 AM, Kevin Pouget wrote:
> 
>> On Wed, Aug 22, 2012 at 9:55 PM, Tom Tromey <tromey@redhat.com> wrote:
>>
>> (and just a detail, in the gdb/ChangeLog of this patch, I think that this line:
>>> * doc/gdb.texinfo (Python Commands): Document the new commands.
>> is supposted to go to doc/ChangeLog
> 
> Oh, I hadn't realized that there is another ChangeLog. What's the procedure for correcting this? Shall I check in a patch that just moves the entries to the correct ChangeLog?

Yep.

-- 
Pedro Alves