Python Scripting Question

Python Scripting Question

[Arjun Roy]

I have a question regarding the python scripting capabilities for GDB.
(running GDB on Fedora 11).

I'd like to be able to figure out the instruction mix for an arbitrary
program. I know I could step through it using nexti and disassemble, but
would like to script it out using python. So a couple of questions:

1. I'm getting some buggy behavior when I do the following:

(gdb) run
Starting program

Breakpoint 1, main () at test.c:3
3      long i = 0;
(gdb) python
>gdb.execute("disassemble $pc $pc+1")
>gdb.execute("disassemble $pc $pc+1")
>end
Dump of assembler code from 0x400478 to 0x400479:
0x0000000000400478 <main+4>:    movq   $0x0,-0x8(%rbp)
End of assembler dump.
Traceback (most recent call last):
  File "<string>", line 2, in <module>
TypeError: argument 1 must be string without null bytes, not str
Error while executing Python code.

Basically, running gdb.execute on the same disassemble statement (which
I assume should just print out the same asm twice, right?) has an issue.
But, if I execute this:

(gdb) python
>gdb.execute("disassemble $pc $pc+1")
>gdb.execute("disassemble $pc+1 $pc+2")
>end
Dump of assembler code from 0x400478 to 0x400479:
0x0000000000400478 <main+4>:    movq   $0x0,-0x8(%rbp)
End of assembler dump.
Dump of assembler code from 0x400479 to 0x40047a:
0x0000000000400479 <main+5>:    movl   $0x0,-0x8(%rbp)
End of assembler dump.

Then it works.

Is this a bug, am I misunderstanding the right way to do it, or what?

2. Ideally, I'd like to write a really simple script that:
- starts running a program
- steps through each instruction one by one, recording the opcode for
each instruction performed
- and stops when the program is over.

Does gdb-python support that? The issues I'm having are the one I
described above, and also not knowing how to check if the program is
done (I know it returns an exception when one tries to call nexti on a
finished program, but I'd rather do something like a "while
programNotDone()" ).

Thanks for the read.

-Arjun Roy

Re: Python Scripting Question

[Tom Tromey]

>>>>> "Arjun" == Arjun Roy <roy.arjun@gmail.com> writes:

Arjun> Traceback (most recent call last):
Arjun>   File "<string>", line 2, in <module>
Arjun> TypeError: argument 1 must be string without null bytes, not str
Arjun> Error while executing Python code.

Arjun> Is this a bug, am I misunderstanding the right way to do it, or what?

It is a bug.

Arjun> 2. Ideally, I'd like to write a really simple script that:
Arjun> - starts running a program
Arjun> - steps through each instruction one by one, recording the opcode for
Arjun> each instruction performed
Arjun> - and stops when the program is over.

Arjun> Does gdb-python support that?

Not yet.  Better execution control is on our wish-list.  There was a SoC
project to implement some of this; the patches need a bit of reworking
though.

Tom

Re: Python Scripting Question

[Tom Tromey]

>>>>> "Tom" == Tom Tromey <tromey@redhat.com> writes:

Arjun> Traceback (most recent call last):
Arjun> File "<string>", line 2, in <module>
Arjun> TypeError: argument 1 must be string without null bytes, not str
Arjun> Error while executing Python code.

Arjun> Is this a bug, am I misunderstanding the right way to do it, or what?

Tom> It is a bug.

I looked into this a little.

The fix is to make python.c:execute_gdb_command copy the command string
before passing it to execute_command.  This avoids gdb commands trashing
Python-owned strings.

It isn't clear to me that gdb commands are allowed to overwrite their
argument strings, though.  Are they?  disassemble_command does do this,
but it seems wrong to me -- won't it permanently modify the command
stored in a command script?

I'll check in the python fix after testing.

Tom

Re: Python Scripting Question

[Daniel Jacobowitz]

On Mon, Nov 09, 2009 at 09:41:54AM -0700, Tom Tromey wrote:
> It isn't clear to me that gdb commands are allowed to overwrite their
> argument strings, though.  Are they?  disassemble_command does do this,
> but it seems wrong to me -- won't it permanently modify the command
> stored in a command script?

Is it this bit?

      /* Two arguments.  */
      *space_index = '\0';

I don't think we should let commands modify their argument string.

-- 
Daniel Jacobowitz
CodeSourcery

Re: Python Scripting Question

[Tom Tromey]

>>>>> "Daniel" == Daniel Jacobowitz <drow@false.org> writes:

Daniel> On Mon, Nov 09, 2009 at 09:41:54AM -0700, Tom Tromey wrote:
>> It isn't clear to me that gdb commands are allowed to overwrite their
>> argument strings, though.  Are they?  disassemble_command does do this,
>> but it seems wrong to me -- won't it permanently modify the command
>> stored in a command script?

Daniel> Is it this bit?
Daniel>       /* Two arguments.  */
Daniel>       *space_index = '\0';

Yeah.

Daniel> I don't think we should let commands modify their argument string.

Me neither.  For the Python case I think we still have to be defensive,
because it seems that a command might temporarily modify its argument (I
am pretty sure this happens but I don't know how often), but even this
sort of modification could cause problems, if the command itself can
cause Python code to be called.

Tom

Re: Python Scripting Question

[Arjun Roy]

On 11/09/2009 08:25 AM, Tom Tromey wrote:
> Arjun> Traceback (most recent call last):
> Arjun>   File "<string>", line 2, in <module>
> Arjun> TypeError: argument 1 must be string without null bytes, not str
> Arjun> Error while executing Python code.
>
> Arjun> Is this a bug, am I misunderstanding the right way to do it, or what?
>
> It is a bug.
>
> Arjun> 2. Ideally, I'd like to write a really simple script that:
> Arjun> - starts running a program
> Arjun> - steps through each instruction one by one, recording the opcode for
> Arjun> each instruction performed
> Arjun> - and stops when the program is over.
>
> Arjun> Does gdb-python support that?
>
> Not yet.  Better execution control is on our wish-list.  There was a SoC
> project to implement some of this; the patches need a bit of reworking
> though.
>
> Tom
>   
It looks as though the two main things this would need would be to:

1. Be able to access the output one gets from executing a GDB command.
Ideally one would get an object with various fields filled in based on
the type of command, but for my purposes even a string would do - as
long as the results can be accessed somehow.

2. Some python objects for representing the current execution state of
the program, where one could call methods for determining what's going on.

Then in my case, a script would be conceptually as simple as:

whileProgramStillRunning()
   nexti
   op = getopcode from (disassemble at $pc, $pc+1)
   // do whatever i want with the result

How easy to hack in would this be?

-Arjun

Re: Python Scripting Question

[Tom Tromey]

>>>>> "Arjun" == Arjun Roy <roy.arjun@gmail.com> writes:

Arjun> It looks as though the two main things this would need would be to:

Arjun> 1. Be able to access the output one gets from executing a GDB command.
Arjun> Ideally one would get an object with various fields filled in based on
Arjun> the type of command, but for my purposes even a string would do - as
Arjun> long as the results can be accessed somehow.

Yeah, this has been on our wish-list for a while.

Basically we need to let python create two kinds of new ui-out objects:
one that writes the output to a python string, and another that is
mi-like and creates new python objects in response to output.

I have a tiny bit of code working toward this direction, but I never
finished it.

Arjun> 2. Some python objects for representing the current execution
Arjun> state of the program, where one could call methods for
Arjun> determining what's going on.

The SoC project addressed part of this.  See the archer list archives
for information.

Arjun> How easy to hack in would this be?

I don't think anything here is too difficult; it just requires someone
to actually do it.

Tom

Re: Python Scripting Question

[scott snyder]

>From: Tom Tromey <tromey@redhat.com>
>>>>>> "Arjun" == Arjun Roy <roy.arjun@gmail.com> writes:
>Arjun> It looks as though the two main things this would need would be to:
>Arjun> 1. Be able to access the output one gets from executing a GDB command.
>Arjun> Ideally one would get an object with various fields filled in based on
>Arjun> the type of command, but for my purposes even a string would do - as
>Arjun> long as the results can be accessed somehow.
>Yeah, this has been on our wish-list for a while.

FWIW, i've been using this patch.  This adds a function
`gdb.execute_getoutput', which returns the output as a string.

This probably isn't really the right way to do this ---
I don't think this does the right thing with MI, and it has an ugly
special case for TUI, but it's been working for me so far.

sss


=== modified file 'gdb/python/python.c'
--- gdb/python/python.c	2009-10-14 16:48:13 +0000
+++ gdb/python/python.c	2009-10-14 22:44:59 +0000
@@ -27,6 +27,7 @@
 #include "observer.h"
 #include "value.h"
 #include "language.h"
+#include "tui/tui-io.h"
 
 #include <ctype.h>
 
@@ -321,6 +322,86 @@
 
 \f
 
+/* A Python function which evaluates a string using the gdb CLI
+   and returns the output as a string.  */
+
+static
+void pyexecute_file_delete (struct ui_file* stream)
+{
+}
+
+static
+void pyexecute_file_write (struct ui_file* stream,
+                           const char* buf,
+                           long length_buf)
+{
+  PyObject* out = (PyObject*)ui_file_data (stream);
+  PyObject* snew = PyString_FromStringAndSize (buf, length_buf);
+  PyString_ConcatAndDel (&out, snew);
+  set_ui_file_data (stream, out, pyexecute_file_delete);
+}
+
+static PyObject *
+execute_gdb_command_getoutput (PyObject *self, PyObject *args)
+{
+  struct cmd_list_element *alias, *prefix, *cmd;
+  char *arg, *newarg;
+  PyObject *from_tty_obj = NULL;
+  int from_tty;
+  int cmp;
+  volatile struct gdb_exception except;
+  struct ui_file* fout;
+  PyObject* out;
+  struct ui_file* old_stdout;
+  struct ui_out* old_uiout;
+
+  if (! PyArg_ParseTuple (args, "s|O!", &arg, &PyBool_Type, &from_tty_obj))
+    return NULL;
+
+  from_tty = 0;
+  if (from_tty_obj)
+    {
+      cmp = PyObject_IsTrue (from_tty_obj);
+      if (cmp < 0)
+	  return NULL;
+      from_tty = cmp;
+    }
+
+  fout = ui_file_new();
+  out = PyString_FromString ("");
+  set_ui_file_data (fout, out, pyexecute_file_delete);
+  set_ui_file_write (fout, pyexecute_file_write);
+  old_stdout = gdb_stdout;
+  gdb_stdout = fout;
+  old_uiout = uiout;
+  if (uiout == tui_out)
+    uiout = tui_old_uiout;
+  if (ui_out_redirect (uiout, gdb_stdout) < 0)
+    warning (_("Current output protocol does not support redirection"));
+
+  TRY_CATCH (except, RETURN_MASK_ALL)
+    {
+      execute_command (arg, from_tty);
+    }
+  ui_out_redirect (uiout, 0);
+  uiout = old_uiout;
+  gdb_stdout = old_stdout;
+  out = ui_file_data (fout);
+  ui_file_delete (fout);
+  if (except.reason < 0) {
+    Py_DECREF (out);
+    out = 0;
+  }
+  GDB_PY_HANDLE_EXCEPTION (except);
+
+  /* Do any commands attached to breakpoint we stopped at.  */
+  bpstat_do_actions ();
+
+  return out;
+}
+
+\f
+
 /* Printing.  */
 
 /* A python function to write a single string using gdb's filtered
@@ -653,6 +734,8 @@
     "Get a value from history" },
   { "execute", execute_gdb_command, METH_VARARGS,
     "Execute a gdb command" },
+  { "execute_getoutput", execute_gdb_command_getoutput, METH_VARARGS,
+    "Execute a gdb command, returning the output as a string" },
   { "parameter", gdbpy_parameter, METH_VARARGS,
     "Return a gdb parameter's value" },
 

Re: Python Scripting Question

[Arjun Roy]

Thanks, I'll check it out. In a pinch that should meet what I need.

-Arjun


On 11/12/2009 12:53 PM, scott snyder wrote:
> FWIW, i've been using this patch.  This adds a function
> `gdb.execute_getoutput', which returns the output as a string.
>
> This probably isn't really the right way to do this ---
> I don't think this does the right thing with MI, and it has an ugly
> special case for TUI, but it's been working for me so far.
>
> sss
>
>
> === modified file 'gdb/python/python.c'
> --- gdb/python/python.c	2009-10-14 16:48:13 +0000
> +++ gdb/python/python.c	2009-10-14 22:44:59 +0000
> @@ -27,6 +27,7 @@
>  #include "observer.h"
>  #include "value.h"
>  #include "language.h"
> +#include "tui/tui-io.h"
>  
>  #include <ctype.h>
>  
> @@ -321,6 +322,86 @@
>  
>  \f
>  
> +/* A Python function which evaluates a string using the gdb CLI
> +   and returns the output as a string.  */
> +
> +static
> +void pyexecute_file_delete (struct ui_file* stream)
> +{
> +}
> +
> +static
> +void pyexecute_file_write (struct ui_file* stream,
> +                           const char* buf,
> +                           long length_buf)
> +{
> +  PyObject* out = (PyObject*)ui_file_data (stream);
> +  PyObject* snew = PyString_FromStringAndSize (buf, length_buf);
> +  PyString_ConcatAndDel (&out, snew);
> +  set_ui_file_data (stream, out, pyexecute_file_delete);
> +}
> +
> +static PyObject *
> +execute_gdb_command_getoutput (PyObject *self, PyObject *args)
> +{
> +  struct cmd_list_element *alias, *prefix, *cmd;
> +  char *arg, *newarg;
> +  PyObject *from_tty_obj = NULL;
> +  int from_tty;
> +  int cmp;
> +  volatile struct gdb_exception except;
> +  struct ui_file* fout;
> +  PyObject* out;
> +  struct ui_file* old_stdout;
> +  struct ui_out* old_uiout;
> +
> +  if (! PyArg_ParseTuple (args, "s|O!", &arg, &PyBool_Type, &from_tty_obj))
> +    return NULL;
> +
> +  from_tty = 0;
> +  if (from_tty_obj)
> +    {
> +      cmp = PyObject_IsTrue (from_tty_obj);
> +      if (cmp < 0)
> +	  return NULL;
> +      from_tty = cmp;
> +    }
> +
> +  fout = ui_file_new();
> +  out = PyString_FromString ("");
> +  set_ui_file_data (fout, out, pyexecute_file_delete);
> +  set_ui_file_write (fout, pyexecute_file_write);
> +  old_stdout = gdb_stdout;
> +  gdb_stdout = fout;
> +  old_uiout = uiout;
> +  if (uiout == tui_out)
> +    uiout = tui_old_uiout;
> +  if (ui_out_redirect (uiout, gdb_stdout) < 0)
> +    warning (_("Current output protocol does not support redirection"));
> +
> +  TRY_CATCH (except, RETURN_MASK_ALL)
> +    {
> +      execute_command (arg, from_tty);
> +    }
> +  ui_out_redirect (uiout, 0);
> +  uiout = old_uiout;
> +  gdb_stdout = old_stdout;
> +  out = ui_file_data (fout);
> +  ui_file_delete (fout);
> +  if (except.reason < 0) {
> +    Py_DECREF (out);
> +    out = 0;
> +  }
> +  GDB_PY_HANDLE_EXCEPTION (except);
> +
> +  /* Do any commands attached to breakpoint we stopped at.  */
> +  bpstat_do_actions ();
> +
> +  return out;
> +}
> +
> +\f
> +
>  /* Printing.  */
>  
>  /* A python function to write a single string using gdb's filtered
> @@ -653,6 +734,8 @@
>      "Get a value from history" },
>    { "execute", execute_gdb_command, METH_VARARGS,
>      "Execute a gdb command" },
> +  { "execute_getoutput", execute_gdb_command_getoutput, METH_VARARGS,
> +    "Execute a gdb command, returning the output as a string" },
>    { "parameter", gdbpy_parameter, METH_VARARGS,
>      "Return a gdb parameter's value" },
>  
>
>   

Re: Python Scripting Question

[Tom Tromey]

>>>>> "scott" == scott snyder <snyder@fnal.gov> writes:

scott> FWIW, i've been using this patch.  This adds a function
scott> `gdb.execute_getoutput', which returns the output as a string.

scott> This probably isn't really the right way to do this ---
scott> I don't think this does the right thing with MI, and it has an ugly
scott> special case for TUI, but it's been working for me so far.

This looks reasonable.  However, I was planning to do it in a different
way.

In particular what I've been thinking is that we'd have a way to let the
user pass any pythonic output stream to gdb; either with explicit
commands to override the current output stream, or as an argument to
gdb.execute.  Handling MI-like output would work in a similar way, just
with a way to indicate that the new output stream is MI-like.

What do you think?  If that is too vague I can try to write up something
more documentation-like.

scott> +  if (uiout == tui_out)
scott> +    uiout = tui_old_uiout;

This sort of redirection probably belongs in a helper function in ui-out.

scott> +  /* Do any commands attached to breakpoint we stopped at.  */
scott> +  bpstat_do_actions ();

This could produce output.  Arguably it should be redirected as well.

Tom