Python API problems

Python API problems

[Joseph Garvin]

I was quite excited to see that gdb 7.0 added python scripting and so
tried to implement a few ideas. Unfortunately, under the current API,
none of them are possible AFAICT. It's possible others have pointed
out these issues already but I thought I'd share them here in case the
developers were interested in knowing what people are trying with the
API and what issues they're running into.

I wanted to be able to wildcard break on all the functions in a
particular C++ scope, so if I had a namespace called 'foo' I could do
this:

break foo::*

And any functions in the foo namespace would receive breakpoints. So I
attempted to code an alternative version of break called 'starbreak'
that would do this. Issues I ran into:

-None of gdb's standard commands are exposed directly. You have to
create strings and pass them in to gdb.execute. gdb.execute doesn't
have a return value, so you can't get back any information from
commands. If break were exposed as a python function for example, it
could return a breakpoint object on which you could later in the
python code call enable/disable, or inspect to see what address or
line number was broken at (if for example you broke by function name).

-There's a lookup_type but not a more general lookup_symbol. It'd be
nice if such a function existed. Also, search_type and search_symbol
for matches to a regex.

-That could be worked around if gdb.execute returned a string
containing what information would otherwise have gone to the console.
Then we could at least parse the output. This would be nice in general
for working around API holes until they're filled. Because of this I
can't even manually search the output from "info functions" (though
searching that would be slooooooow).

-You can't call the builtin completion functions. Basically I'd like
to call complete_symbols("foo::", "") to get a list of symbols in the
foo namespace, then iterate through them and break on each one. If in
my constructor I pass in gdb.COMPLETE_SYMBOL, I would expect then if I
call the base class version of complete, e.g.
gdb.Command.complete(self, "foo::", "") I would get back the results
of the builtin version of complete. But I don't, complete is not
defined. This doesn't feel very pythonic.

-Needing to pass the name of the class to the constructor is silly.
Python has reflection facilities (see the 'inspect' module), gdb
should be able to inspect the class and figure out its name itself.

-Needing to manually instantiate an instance of the class *may be*
silly. Python's reflection could be used to look at the imported
classes and make instantiations of any classes defined to inherit from
gdb.Command. On the other hand, real pythonistas might be able to
conjure some hack that depends on commands being instantiated in an
order they define, so this isn't necessarily a good idea, but I can't
think of why someone would want to do that off the top of my head at
least.

Re: Python API problems

[Joseph Garvin]

Also, being able to know the thread number for the current frame would
be useful for writing commands that keep state associated with each
thread.

On Wed, Jan 27, 2010 at 1:27 PM, Joseph Garvin
<joseph.h.garvin@gmail.com> wrote:
> I was quite excited to see that gdb 7.0 added python scripting and so
> tried to implement a few ideas. Unfortunately, under the current API,
> none of them are possible AFAICT. It's possible others have pointed
> out these issues already but I thought I'd share them here in case the
> developers were interested in knowing what people are trying with the
> API and what issues they're running into.
>
> I wanted to be able to wildcard break on all the functions in a
> particular C++ scope, so if I had a namespace called 'foo' I could do
> this:
>
> break foo::*
>
> And any functions in the foo namespace would receive breakpoints. So I
> attempted to code an alternative version of break called 'starbreak'
> that would do this. Issues I ran into:
>
> -None of gdb's standard commands are exposed directly. You have to
> create strings and pass them in to gdb.execute. gdb.execute doesn't
> have a return value, so you can't get back any information from
> commands. If break were exposed as a python function for example, it
> could return a breakpoint object on which you could later in the
> python code call enable/disable, or inspect to see what address or
> line number was broken at (if for example you broke by function name).
>
> -There's a lookup_type but not a more general lookup_symbol. It'd be
> nice if such a function existed. Also, search_type and search_symbol
> for matches to a regex.
>
> -That could be worked around if gdb.execute returned a string
> containing what information would otherwise have gone to the console.
> Then we could at least parse the output. This would be nice in general
> for working around API holes until they're filled. Because of this I
> can't even manually search the output from "info functions" (though
> searching that would be slooooooow).
>
> -You can't call the builtin completion functions. Basically I'd like
> to call complete_symbols("foo::", "") to get a list of symbols in the
> foo namespace, then iterate through them and break on each one. If in
> my constructor I pass in gdb.COMPLETE_SYMBOL, I would expect then if I
> call the base class version of complete, e.g.
> gdb.Command.complete(self, "foo::", "") I would get back the results
> of the builtin version of complete. But I don't, complete is not
> defined. This doesn't feel very pythonic.
>
> -Needing to pass the name of the class to the constructor is silly.
> Python has reflection facilities (see the 'inspect' module), gdb
> should be able to inspect the class and figure out its name itself.
>
> -Needing to manually instantiate an instance of the class *may be*
> silly. Python's reflection could be used to look at the imported
> classes and make instantiations of any classes defined to inherit from
> gdb.Command. On the other hand, real pythonistas might be able to
> conjure some hack that depends on commands being instantiated in an
> order they define, so this isn't necessarily a good idea, but I can't
> think of why someone would want to do that off the top of my head at
> least.
>

Re: Python API problems

[Rich Lane]

Excerpts from Joseph Garvin's message of 2010-01-27 14:27:05 -0500:
> -That could be worked around if gdb.execute returned a string
> containing what information would otherwise have gone to the console.
> Then we could at least parse the output. This would be nice in general
> for working around API holes until they're filled. Because of this I
> can't even manually search the output from "info functions" (though
> searching that would be slooooooow).

I worked around this with a terrible hack:

def gdbx(cmd):
   fd, filename = tempfile.mkstemp('.gdb-out')
   gdb.execute("set logging off")
   gdb.execute("set logging file" + filename)
   gdb.execute("set logging redirect on")
   gdb.execute("set logging on")
   try:
      gdb.execute(cmd)
   finally:
      gdb.execute("set logging off")
      gdb.execute("set logging redirect off")
      str = os.fdopen(fd).read()
      os.remove(filename)
      return str.strip()

Unfortunately, this doesn't work if you try to use "interpreter-exec mi"
to get easily parseable output.

There's an open bug: http://sourceware.org/bugzilla/show_bug.cgi?id=10808

I'm also very much looking forward to the python API becoming more
complete. It would be useful to have the function parameters in a Frame
object, for example.

Re: Python API problems

[Tom Tromey]

>>>>> "Joseph" == Joseph Garvin <joseph.h.garvin@gmail.com> writes:

Sorry for the delay in this reply...

Joseph> Unfortunately, under the current API, none of them are possible
Joseph> AFAICT.

Yeah.  In gdb CVS the support is mostly for pretty-printing and nothing
else; there is more on the appropriate Archer branch (and we're slowly
pushing that upstream), but even there, there is a lot missing.

Joseph> I wanted to be able to wildcard break on all the functions in a
Joseph> particular C++ scope, so if I had a namespace called 'foo' I could do
Joseph> this:

Yeah, better symbol table stuff is on the wish-list.
Feel free to file bugs for this or any other missing features, btw.

Joseph> If break were exposed as a python function for example, it
Joseph> could return a breakpoint object on which you could later in the
Joseph> python code call enable/disable, or inspect to see what address or
Joseph> line number was broken at (if for example you broke by function name).

There is some breakpoint support on the branch.

Joseph> -There's a lookup_type but not a more general lookup_symbol. It'd be
Joseph> nice if such a function existed. Also, search_type and search_symbol
Joseph> for matches to a regex.

There is some symbol stuff on the branch, though I think not searching.
Phil should have the symbol patch ready for submission soon, he's been
working on that recently.

Joseph> -That could be worked around if gdb.execute returned a string
Joseph> containing what information would otherwise have gone to the console.

This is http://sourceware.org/bugzilla/show_bug.cgi?id=10808

Also I have been thinking that we could make a new "MI-like" ui-out
object that creates Python objects, so you could get structured output
from at least some commands.

Even if we did this the output would be fairly primitive objects -- ints
and strings and whatnot, not Breakpoints or the like.  The latter would
require even more work.  Instead I think we'll just expose the various
interest APIs directly, as we've already done for value and types.

Joseph> -You can't call the builtin completion functions. Basically I'd like
Joseph> to call complete_symbols("foo::", "") to get a list of symbols in the
Joseph> foo namespace, then iterate through them and break on each one. If in
Joseph> my constructor I pass in gdb.COMPLETE_SYMBOL, I would expect then if I
Joseph> call the base class version of complete, e.g.
Joseph> gdb.Command.complete(self, "foo::", "") I would get back the results
Joseph> of the builtin version of complete. But I don't, complete is not
Joseph> defined. This doesn't feel very pythonic.

I wouldn't mind a bug report for this.

Joseph> -Needing to pass the name of the class to the constructor is silly.
Joseph> Python has reflection facilities (see the 'inspect' module), gdb
Joseph> should be able to inspect the class and figure out its name itself.

You may want a multi-word name for your command.

You may like this:
http://sourceware.org/ml/gdb-patches/2010-01/msg00221.html

Joseph> -Needing to manually instantiate an instance of the class *may be*
Joseph> silly. Python's reflection could be used to look at the imported
Joseph> classes and make instantiations of any classes defined to inherit from
Joseph> gdb.Command. On the other hand, real pythonistas might be able to
Joseph> conjure some hack that depends on commands being instantiated in an
Joseph> order they define, so this isn't necessarily a good idea, but I can't
Joseph> think of why someone would want to do that off the top of my head at
Joseph> least.

I tend to think that future APIs akin to this one will be a little more
"duck-type-y" and a little less inheritance based.  The inheritance
thing is my java background showing through, oops.

We're very interested in pushing the Python effort forward.  It has been
a little slow lately; right now Phil is working on Python stuff
full-time, but he has been busy with fixing bugs and now patch
wrangling.

Tom