Some feedback about the python scripting feature

Some feedback about the python scripting feature

[Marcelo Taube]

Dear gdb team,

I have been using the python scripting feature of gdb7

The feature is great it let me debug my code much better and in a much 
flexible way.

There are some features missing which would improve it even more, though.

I want to mention them in case noone did it before.

    * *calling functions*: From regular gdb command line the user can
      allways simply do " p foo()" to call function and print the
      result, it would be usefull to be able to call C functions (or the
      debugged lenguage) from python and get the result as a Value
      object in the same way a variable is accessed.
    * *equality of types: *i think there are problems with the equality
      of types in gdb. Look at the case that  two type objects were
      obtained and represent the same type, the expression
      'gdb.lookup_type("int") == gdb.lookup_type("int")' now returns
      false but it should return true.
    * *interactive python: *the current python command reads lines using
      gdb normal input instead of the regular python prompt line, making
      some nice features of python unavailable like
      completions-proposals (pressing the tab-key to get the name of a
      function completed) or auto indentation. Maybe running the regular
      python prompt line can be an option.
    * *regular members instead of dictionaries: *it would be more
      natural to have members of structs be accessed in their natural
      way like in c. For example if we have some struct "struct Type {
      int a; int b;}" and we have a value object in python called
      'value', Now to access 'value.a' we have to do 'value["a"]' but
      that feels preety unnatural, why not just support the syntax
      'value.a' ?

Thank you so much for your excelent work and for the time you spent 
reading this lines
Marcelo

Re: Some feedback about the python scripting feature

[Phil Muldoon]

On 11/09/2009 10:35 PM, Marcelo Taube wrote:
> Dear gdb team,
>
> I have been using the python scripting feature of gdb7
>
> The feature is great it let me debug my code much better and in a much 
> flexible way.
>
> There are some features missing which would improve it even more, though.
>
> I want to mention them in case noone did it before.


Thanks.  I'll comment on the two things I know about:


>
>    * *calling functions*: From regular gdb command line the user can
>      allways simply do " p foo()" to call function and print the
>      result, it would be usefull to be able to call C functions (or the
>      debugged lenguage) from python and get the result as a Value
>      object in the same way a variable is accessed.

There is a patch in the works:

http://sourceware.org/ml/archer/2009-q3/msg00206.html

But for right now you can use:

  foo = gdb.parse_and_eval("foo(bar)")


>    * *equality of types: *i think there are problems with the equality
>      of types in gdb. Look at the case that  two type objects were
>      obtained and represent the same type, the expression
>      'gdb.lookup_type("int") == gdb.lookup_type("int")' now returns
>      false but it should return true.

There was an enhancement filed for this:


http://sourceware.org/bugzilla/show_bug.cgi?id=10676


Regards

Phil

Re: Some feedback about the python scripting feature

[Tom Tromey]

>>>>> "Marcelo" == Marcelo Taube <mail.marcelo.taube@gmail.com> writes:

Marcelo> There are some features missing which would improve it even
Marcelo> more, though.

Feel free to file bug reports in GDB bugzilla.

Marcelo>    * *interactive python: *the current python command reads lines using
Marcelo>      gdb normal input instead of the regular python prompt line, making
Marcelo>      some nice features of python unavailable like
Marcelo>      completions-proposals (pressing the tab-key to get the name of a
Marcelo>      function completed) or auto indentation. Maybe running the regular
Marcelo>      python prompt line can be an option.

You should be able to easily enter the Python repl, though offhand I
don't know how.

Marcelo>    * *regular members instead of dictionaries: *it would be more
Marcelo>      natural to have members of structs be accessed in their natural
Marcelo>      way like in c. For example if we have some struct "struct Type {
Marcelo>      int a; int b;}" and we have a value object in python called
Marcelo>      'value', Now to access 'value.a' we have to do 'value["a"]' but
Marcelo>      that feels preety unnatural, why not just support the syntax
Marcelo>      'value.a' ?

We chose this approach to avoid conflict with Value's own members, like
`type'.

Tom

Re: Some feedback about the python scripting feature

[Daniel Jacobowitz]

On Tue, Nov 10, 2009 at 08:06:05AM -0700, Tom Tromey wrote:
> Marcelo>    * *interactive python: *the current python command reads lines using
> Marcelo>      gdb normal input instead of the regular python prompt line, making
> Marcelo>      some nice features of python unavailable like
> Marcelo>      completions-proposals (pressing the tab-key to get the name of a
> Marcelo>      function completed) or auto indentation. Maybe running the regular
> Marcelo>      python prompt line can be an option.
> 
> You should be able to easily enter the Python repl, though offhand I
> don't know how.

I thought it was easy to get the actual repl... but it isn't.  There
is a C function for it, but not a Python method.

However, there's a very good simulacrum:

  py import code
  py code.InteractiveConsole().interact()

Send EOF to exit.

I don't know if it has completion or automatic indentation, because no
version of Python I have installed does either :-)  Is this new?

-- 
Daniel Jacobowitz
CodeSourcery

Re: Some feedback about the python scripting feature

[Marcelo Taube]

Hello,
> I don't know if it has completion or automatic indentation, because no
> version of Python I have installed does either :-)  Is this new?
>   
I am sorry, i guess i was "slightly" wrong.
The completion is not automatic, someone added it to my working 
environment and i just thought it was part of the default behaiviour.
But it should be very easy to add to any other environment, i saw that 
the next three lines should do the job:

>>> import rlcompleter
>>> import readline
>>> readline.parse_and_bind("tab: complete")


Regarding the indentation, there is no support in the regular python 
shell... only in the 'ipython' shell or the 'idle' shell that are kind 
of improved CUIs for python.

In spite of that, i still think that the python shell has advantages, 
there is a difference between having direct response to the commands 
executed in python and having to imagine that the syntax is ok till you 
finish writing all the code. Also mixing the history of lines insterted 
in python context and lines inserted in gdb language is kind of 
bothering, they cannot really be reused.
|Thanks again for all the great work.
Marcelo

Re: Some feedback about the python scripting feature

[Daniel Jacobowitz]

On Tue, Nov 10, 2009 at 08:49:18PM +0200, Marcelo Taube wrote:
> Hello,
> >I don't know if it has completion or automatic indentation, because no
> >version of Python I have installed does either :-)  Is this new?
> I am sorry, i guess i was "slightly" wrong.
> The completion is not automatic, someone added it to my working
> environment and i just thought it was part of the default behaiviour.
> But it should be very easy to add to any other environment, i saw
> that the next three lines should do the job:
> 
> >>>import rlcompleter
> >>>import readline
> >>>readline.parse_and_bind("tab: complete")

This "works" in GDB... but it doesn't do anything you want it to do.
It changes the tab completion used by the GDB prompt!  Whoops.

> In spite of that, i still think that the python shell has advantages,
> there is a difference between having direct response to the commands
> executed in python and having to imagine that the syntax is ok till
> you finish writing all the code. Also mixing the history of lines
> insterted in python context and lines inserted in gdb language is
> kind of bothering, they cannot really be reused.

That should work with the command I showed you, at least to some
extent.

-- 
Daniel Jacobowitz
CodeSourcery

Re: Some feedback about the python scripting feature

[scott snyder]

>From: Daniel Jacobowitz <drow@false.org>
>  py import code
>  py code.InteractiveConsole().interact()

I tried this a while ago, and couldn't get readline to work.
This is because gdb set sys.stdout/stderr to objects which python
didn't recognize as a tty.  And if i hacked around that, i'd get crashes,
because gdb and python were each trying to configure readline differently.

I ended up adding a new function to the python api to read a line of input
using gdb's input reader:

=== modified file 'gdb/python/python.c'
--- gdb/python/python.c	2009-10-14 22:45:34 +0000
+++ gdb/python/python.c	2009-10-14 22:46:03 +0000
@@ -28,6 +28,7 @@
 #include "value.h"
 #include "language.h"
 #include "tui/tui-io.h"
+#include "readline/history.h"
 
 #include <ctype.h>
 
@@ -459,6 +460,37 @@
 
 \f
 
+/* Reading.  */
+
+static PyObject* 
+gdbpy_input (PyObject* self, PyObject* args)
+{
+  const char* prompt_str = "";
+  char* line = 0;
+  volatile struct gdb_exception e;
+
+  if (!PyArg_ParseTuple (args, "|s", &prompt_str))
+    return NULL;
+
+  TRY_CATCH (e, RETURN_MASK_QUIT) {
+    line = gdb_readline_wrapper ((char*)prompt_str);
+  }
+  if (e.reason < 0) {
+    PyErr_SetNone (PyExc_KeyboardInterrupt);
+    return NULL;
+  }
+
+  if (!line) {
+    PyErr_SetNone (PyExc_EOFError);
+    return NULL;
+  }
+
+  add_history (line);
+  return PyString_FromString (line);
+}
+
+\f
+
 /* The "current" objfile.  This is set when gdb detects that a new
    objfile has been loaded.  It is only set for the duration of a call
    to gdbpy_new_objfile; it is NULL at other times.  */
@@ -784,6 +816,8 @@
     "Write a string using gdb's filtered stream." },
   { "flush", gdbpy_flush, METH_NOARGS,
     "Flush gdb's filtered stdout stream." },
+  { "input", gdbpy_input, METH_VARARGS,
+    "Read a line of input from the user." },
 
   { "parse_and_eval", gdbpy_parse_and_eval, METH_VARARGS,
     "Parse a string as an expression, evaluate it, and return the result." },



Then this can be used with this command:


import gdb
import code
import sys

class PyInteract (gdb.Command):
    """Start a python read-eval-print loop from within gdb."""

    def __init__ (self):
        super (PyInteract, self).__init__ ("pyinteract", gdb.COMMAND_OBSCURE)
        return

    def invoke (self, arg, from_tty):
        self.dont_repeat()
        code.interact("Press Ctrl-D to return to gdb.",
                      gdb.input,
                      sys.modules['__main__'].__dict__)
        return


PyInteract()



This has a drawback in that Ctrl-C won't interrupt python code,
meaning it's possible to lose control of the debugger by executing
an infinite loop in python.

I'm currently using these patches to try to fix that.
This is rather uglier than i'd like, but it's worked
for me so far.

sss


=== modified file 'gdb/python/py-cmd.c'
--- gdb/python/py-cmd.c	2009-10-14 16:48:13 +0000
+++ gdb/python/py-cmd.c	2009-10-15 04:22:35 +0000
@@ -207,6 +207,8 @@
   Py_DECREF (wordobj);
   if (! resultobj)
     {
+      if (PyErr_Occurred() == PyExc_KeyboardInterrupt)
+        quit_flag = 1;
       /* Just swallow errors here.  */
       PyErr_Clear ();
       goto done;
@@ -228,6 +230,8 @@
 	  PyObject *elt = PySequence_GetItem (resultobj, i);
 	  if (elt == NULL || ! gdbpy_is_string (elt))
 	    {
+              if (PyErr_Occurred() == PyExc_KeyboardInterrupt)
+                quit_flag = 1;
 	      /* Skip problem elements.  */
 	      PyErr_Clear ();
 	      continue;

=== modified file 'gdb/python/py-prettyprint.c'
--- gdb/python/py-prettyprint.c	2009-10-14 16:48:13 +0000
+++ gdb/python/py-prettyprint.c	2009-10-15 04:22:35 +0000
@@ -49,8 +49,11 @@
 	return NULL;
 
       printer = PyObject_CallFunctionObjArgs (function, value, NULL);
-      if (! printer)
+      if (! printer) {
+        if (PyErr_Occurred() == PyExc_KeyboardInterrupt)
+          quit_flag = 1;
 	return NULL;
+      }
       else if (printer != Py_None)
 	return printer;
 
@@ -77,6 +80,8 @@
     PyObject *objf = objfile_to_objfile_object (obj);
     if (!objf)
       {
+        if (PyErr_Occurred() == PyExc_KeyboardInterrupt)
+          quit_flag = 1;
 	/* Ignore the error and continue.  */
 	PyErr_Clear ();
 	continue;
@@ -109,8 +114,11 @@
       goto done;
     }
   pp_list = PyObject_GetAttrString (gdb_module, "pretty_printers");
-  if (! pp_list)
+  if (! pp_list) {
+    if (PyErr_Occurred() == PyExc_KeyboardInterrupt)
+      quit_flag = 1;
     goto done;
+  }
   if (! PyList_Check (pp_list))
     goto done;
 
@@ -141,8 +149,11 @@
 	  if (! gdbpy_is_string (result))
 	    {
 	      *out_value = convert_value_from_python (result);
-	      if (PyErr_Occurred ())
+	      if (PyErr_Occurred ()) {
+                if (PyErr_Occurred() == PyExc_KeyboardInterrupt)
+                  quit_flag = 1;
 		*out_value = NULL;
+              }
 	      Py_DECREF (result);
 	      result = NULL;
 	    }

=== modified file 'gdb/python/python.c'
--- gdb/python/python.c	2009-10-14 22:46:40 +0000
+++ gdb/python/python.c	2009-10-15 04:22:35 +0000
@@ -69,6 +69,60 @@
 struct gdbarch *python_gdbarch;
 const struct language_defn *python_language;
 
+#include "event-loop.h"
+
+extern void* sigint_token;
+static void* py_sigint_token = 0;
+static void* old_sigint_token = 0;
+static int old_immediate_quit;
+
+static int
+py_checksignals(void* arg)
+{
+  return PyErr_CheckSignals();
+}
+
+static void
+async_pyinterrupt (gdb_client_data arg)
+{
+  PyErr_SetInterrupt();
+  Py_AddPendingCall (py_checksignals, NULL);
+}
+
+
+static void* 
+set_pyinterrupt()
+{
+  void* oldtoken = sigint_token;
+  old_sigint_token = oldtoken;
+  if (py_sigint_token == 0)
+    py_sigint_token = create_async_signal_handler (async_pyinterrupt, NULL);
+  sigint_token = py_sigint_token;
+  if (quit_flag) {
+    async_pyinterrupt (NULL);
+    quit_flag = 0;
+  }
+  old_immediate_quit = immediate_quit;
+  immediate_quit = 1;
+  return oldtoken;
+}
+
+void
+restore_pyinterrupt (void* token)
+{
+  if (!token) {
+    if (old_sigint_token) {
+      sigint_token = old_sigint_token;
+      immediate_quit = old_immediate_quit;
+    }
+  }
+  else
+    sigint_token = token;
+  old_sigint_token = 0;
+  if (PyOS_InterruptOccurred())
+    quit_flag = 1;
+}
+
 /* Restore global language and architecture and Python GIL state
    when leaving the Python interpreter.  */
 
@@ -77,6 +131,8 @@
   PyGILState_STATE state;
   struct gdbarch *gdbarch;
   const struct language_defn *language;
+  void* old_token;
+  int old_immediate;
 };
 
 static void
@@ -86,6 +142,10 @@
   PyGILState_Release (env->state);
   python_gdbarch = env->gdbarch;
   python_language = env->language;
+  if (env->old_token) {
+    restore_pyinterrupt (env->old_token);
+    immediate_quit = env->old_immediate;
+  }
   xfree (env);
 }
 
@@ -101,6 +161,8 @@
   env->state = PyGILState_Ensure ();
   env->gdbarch = python_gdbarch;
   env->language = python_language;
+  env->old_immediate = immediate_quit;
+  env->old_token = set_pyinterrupt();
 
   python_gdbarch = gdbarch;
   python_language = language;
@@ -171,17 +233,18 @@
 python_command (char *arg, int from_tty)
 {
   struct cleanup *cleanup;
-  cleanup = ensure_python_env (get_current_arch (), current_language);
 
   while (arg && *arg && isspace (*arg))
     ++arg;
   if (arg && *arg)
     {
+      cleanup = ensure_python_env (get_current_arch (), current_language);
       if (PyRun_SimpleString (arg))
 	{
 	  gdbpy_print_stack ();
 	  error (_("Error while executing Python code."));
 	}
+      do_cleanups (cleanup);
     }
   else
     {
@@ -190,7 +253,6 @@
       execute_control_command_untraced (l);
     }
 
-  do_cleanups (cleanup);
 }
 
 \f
@@ -311,8 +373,10 @@
 
   TRY_CATCH (except, RETURN_MASK_ALL)
     {
+      restore_pyinterrupt (0);
       execute_command (arg, from_tty);
     }
+  set_pyinterrupt();
   GDB_PY_HANDLE_EXCEPTION (except);
 
   /* Do any commands attached to breakpoint we stopped at.  */
@@ -382,8 +446,10 @@
 
   TRY_CATCH (except, RETURN_MASK_ALL)
     {
+      restore_pyinterrupt (0);
       execute_command (arg, from_tty);
     }
+  set_pyinterrupt();
   ui_out_redirect (uiout, 0);
   uiout = old_uiout;
   gdb_stdout = old_stdout;
@@ -414,8 +480,10 @@
 
   TRY_CATCH (except, RETURN_MASK_ALL)
     {
+      restore_pyinterrupt (0);
       result = parse_and_eval (expr_str);
     }
+  set_pyinterrupt();
   GDB_PY_HANDLE_EXCEPTION (except);
 
   return value_to_value_object (result);
@@ -452,6 +519,9 @@
 void
 gdbpy_print_stack (void)
 {
+  if (PyErr_Occurred() == PyExc_KeyboardInterrupt)
+    quit_flag = 1;
+
   if (gdbpy_should_print_stack)
     PyErr_Print ();
   else
@@ -473,8 +543,11 @@
     return NULL;
 
   TRY_CATCH (e, RETURN_MASK_QUIT) {
+    restore_pyinterrupt (0);
+    quit_flag = 0;
     line = gdb_readline_wrapper ((char*)prompt_str);
   }
+  set_pyinterrupt();
   if (e.reason < 0) {
     PyErr_SetNone (PyExc_KeyboardInterrupt);
     return NULL;