container_of equivalent in gdb-python script

container_of equivalent in gdb-python script

[Jan Kiszka]

Hi,

I'm currently getting my feet wet with gdb's great python extension. My
first exercise is to walk a linked list of objects in a C inferior
(namely 'modules' of the Linux kernel) and evaluate the elements. I'm at
the point where I would apply a standard container_of() macro in C:
convert the list entry into a gdb.Value that describes the containing
object.

Of course, I can implement container_of as a gdb expression, print the
result and drag it in via gdb.history(). Done that already, basically
works. But it is fairly ugly as the print output will flood the screen.

I'm probably overseeing some obvious path in the python extension, any
pointers welcome!

Thanks,
Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

Re: container_of equivalent in gdb-python script

[Tom Tromey]

>>>>> "Jan" == Jan Kiszka <jan.kiszka@siemens.com> writes:

Jan> I'm at the point where I would apply a standard container_of()
Jan> macro in C: convert the list entry into a gdb.Value that
Jan> describes the containing object.

Jan> Of course, I can implement container_of as a gdb expression, print the
Jan> result and drag it in via gdb.history(). Done that already, basically
Jan> works. But it is fairly ugly as the print output will flood the screen.

I would say that the usual approach would be to reimplement the macro
in Python.  The Python Value API is reasonably robust and can usually
be used for this.  If you post the macro definition maybe I could help
with that.  More details wouldn't hurt, either... are you writing a
pretty-printer?  A new command?  A convenience function?

Tom

Re: container_of equivalent in gdb-python script

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2104 bytes --]

Tom Tromey wrote:
>>>>>> "Jan" == Jan Kiszka <jan.kiszka@siemens.com> writes:
> 
> Jan> I'm at the point where I would apply a standard container_of()
> Jan> macro in C: convert the list entry into a gdb.Value that
> Jan> describes the containing object.
> 
> Jan> Of course, I can implement container_of as a gdb expression, print the
> Jan> result and drag it in via gdb.history(). Done that already, basically
> Jan> works. But it is fairly ugly as the print output will flood the screen.
> 
> I would say that the usual approach would be to reimplement the macro
> in Python.  The Python Value API is reasonably robust and can usually
> be used for this.  If you post the macro definition maybe I could help
> with that.  More details wouldn't hurt, either... are you writing a
> pretty-printer?  A new command?  A convenience function?

I want to automate 'add-symbol-file linux_module.ko 0xff...' that you
have to run for loading the symbols of dynamically loaded kernel
modules. Before that you also have to look up the module base address,
typically by cat'ing /proc/modules on the target. With a proper python
script, this will be trivial to do automatically. You just have to walk
the module list of the kernel you are attached to, extract names and
base addresses, search for the corresponding module binaries (also easy
with python) and issue the proper add-symbol-file commands.

But now back to the core problems, starting with the exercise to
implement offset_of(type, field):

def offset_of(type, field):
	container_type = gdb.lookup_type(type)
	dummy_obj = gdb.selected_frame().read_var('modules')
	container_obj = dummy_obj.cast(container_type)
	field_obj = container_obj[field]
	return int(str(field_obj.address), 16) - \
	       int(str(container_obj.address), 16)

I meanwhile discovered (reading testcases and python-*.c)
gdb.lookup_type() and Value.cast() to make this real. But you see, I
still need an ugly synthetic Value object which must have a non-'None'
address to do this calculation. Is there a cleaner, more generic way?

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: container_of equivalent in gdb-python script

[Tom Tromey]

>>>>> "Jan" == Jan Kiszka <jan.kiszka@web.de> writes:

Jan> I meanwhile discovered (reading testcases and python-*.c)
Jan> gdb.lookup_type() and Value.cast() to make this real. But you see, I
Jan> still need an ugly synthetic Value object which must have a non-'None'
Jan> address to do this calculation. Is there a cleaner, more generic way?

Yeah, I think the missing piece is "gdb.parse_and_eval", which is on
the archer python branch, but which we haven't pushed upstream into
gdb CVS yet.  This takes an expression (a string), parses it,
evaluates it, and returns a Value.  It is very convenient when
implementing your own gdb commands.

I don't think there's any particular problem with submitting this,
just somebody finding the time.

Tom

Re: container_of equivalent in gdb-python script

[Daniel Jacobowitz]

On Wed, Jun 10, 2009 at 10:10:39PM +0200, Jan Kiszka wrote:
> I want to automate 'add-symbol-file linux_module.ko 0xff...' that you
> have to run for loading the symbols of dynamically loaded kernel
> modules. Before that you also have to look up the module base address,
> typically by cat'ing /proc/modules on the target. With a proper python
> script, this will be trivial to do automatically. You just have to walk
> the module list of the kernel you are attached to, extract names and
> base addresses, search for the corresponding module binaries (also easy
> with python) and issue the proper add-symbol-file commands.

FYI, I am hoping to work on this soon (in the next month).  The right
representation is not a set of add-symbol-file commands, which doesn't
play nicely with repeated unload/reload, but instead a synthetic list
of shared libraries.  That calls for some extensions to the current
set of Python hooks and shared library infrastructure, though!  And it
will require basically everything you're doing now :-)

> But now back to the core problems, starting with the exercise to
> implement offset_of(type, field):
> 
> def offset_of(type, field):
> 	container_type = gdb.lookup_type(type)
> 	dummy_obj = gdb.selected_frame().read_var('modules')
> 	container_obj = dummy_obj.cast(container_type)
> 	field_obj = container_obj[field]
> 	return int(str(field_obj.address), 16) - \
> 	       int(str(container_obj.address), 16)
> 
> I meanwhile discovered (reading testcases and python-*.c)
> gdb.lookup_type() and Value.cast() to make this real. But you see, I
> still need an ugly synthetic Value object which must have a non-'None'
> address to do this calculation. Is there a cleaner, more generic way?

It's typical to do this the same way folks do in C: Create a null
pointer of the right type.

-- 
Daniel Jacobowitz
CodeSourcery

Re: container_of equivalent in gdb-python script

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 994 bytes --]

Tom Tromey wrote:
>>>>>> "Jan" == Jan Kiszka <jan.kiszka@web.de> writes:
> 
> Jan> I meanwhile discovered (reading testcases and python-*.c)
> Jan> gdb.lookup_type() and Value.cast() to make this real. But you see, I
> Jan> still need an ugly synthetic Value object which must have a non-'None'
> Jan> address to do this calculation. Is there a cleaner, more generic way?
> 
> Yeah, I think the missing piece is "gdb.parse_and_eval", which is on
> the archer python branch, but which we haven't pushed upstream into
> gdb CVS yet.  This takes an expression (a string), parses it,
> evaluates it, and returns a Value.  It is very convenient when
> implementing your own gdb commands.

Perfect, that solves the problem! Just checked out and tried successfully.

> 
> I don't think there's any particular problem with submitting this,
> just somebody finding the time.
> 

Hopefully before the next release. I would try to help, but I don't have
an FSF assignment.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: container_of equivalent in gdb-python script

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2482 bytes --]

Daniel Jacobowitz wrote:
> On Wed, Jun 10, 2009 at 10:10:39PM +0200, Jan Kiszka wrote:
>> I want to automate 'add-symbol-file linux_module.ko 0xff...' that you
>> have to run for loading the symbols of dynamically loaded kernel
>> modules. Before that you also have to look up the module base address,
>> typically by cat'ing /proc/modules on the target. With a proper python
>> script, this will be trivial to do automatically. You just have to walk
>> the module list of the kernel you are attached to, extract names and
>> base addresses, search for the corresponding module binaries (also easy
>> with python) and issue the proper add-symbol-file commands.
> 
> FYI, I am hoping to work on this soon (in the next month).  The right
> representation is not a set of add-symbol-file commands, which doesn't
> play nicely with repeated unload/reload, but instead a synthetic list

Currently, I does. I simply reload the whole set of symbols (vmlinux +
modules) when requested by the user. Of course, one could additionally
register a triggering breakpoint on return of load_module(), but having
a simple helper that can be called on demand will already be a great
improvement.

> of shared libraries.  That calls for some extensions to the current
> set of Python hooks and shared library infrastructure, though!  And it
> will require basically everything you're doing now :-)

Looking forward nevertheless. What ever makes kernel debugging more
handy is welcome. Once finished, I will post my script here.

> 
>> But now back to the core problems, starting with the exercise to
>> implement offset_of(type, field):
>>
>> def offset_of(type, field):
>> 	container_type = gdb.lookup_type(type)
>> 	dummy_obj = gdb.selected_frame().read_var('modules')
>> 	container_obj = dummy_obj.cast(container_type)
>> 	field_obj = container_obj[field]
>> 	return int(str(field_obj.address), 16) - \
>> 	       int(str(container_obj.address), 16)
>>
>> I meanwhile discovered (reading testcases and python-*.c)
>> gdb.lookup_type() and Value.cast() to make this real. But you see, I
>> still need an ugly synthetic Value object which must have a non-'None'
>> address to do this calculation. Is there a cleaner, more generic way?
> 
> It's typical to do this the same way folks do in C: Create a null
> pointer of the right type.

You can't do that with the current upstream python interface, but the
parse_and_eval solves this nicely.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: container_of equivalent in gdb-python script

[Tom Tromey]

>>>>> "Jan" == Jan Kiszka <jan.kiszka@web.de> writes:

Daniel> It's typical to do this the same way folks do in C: Create a null
Daniel> pointer of the right type.

Jan> You can't do that with the current upstream python interface, but the
Jan> parse_and_eval solves this nicely.

Actually, for constants you can:

(gdb) python print gdb.Value(0)
0
(gdb) python print type(gdb.Value(0))
<type 'gdb.Value'>
(gdb) python print gdb.Value(0).cast(gdb.lookup_type('char').pointer())
0x0

Whoops, this seems to be undocumented.

Tom

Re: container_of equivalent in gdb-python script

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 776 bytes --]

Tom Tromey wrote:
>>>>>> "Jan" == Jan Kiszka <jan.kiszka@web.de> writes:
> 
> Daniel> It's typical to do this the same way folks do in C: Create a null
> Daniel> pointer of the right type.
> 
> Jan> You can't do that with the current upstream python interface, but the
> Jan> parse_and_eval solves this nicely.
> 
> Actually, for constants you can:
> 
> (gdb) python print gdb.Value(0)
> 0
> (gdb) python print type(gdb.Value(0))
> <type 'gdb.Value'>
> (gdb) python print gdb.Value(0).cast(gdb.lookup_type('char').pointer())
> 0x0

Yeah, seen this (in the testsuite). But such objects neither have
addresses, nor do they help with non-constant objects I'm interested in.

> 
> Whoops, this seems to be undocumented.

Not the only piece... :)

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: container_of equivalent in gdb-python script

[Tom Tromey]

>>>>> "Jan" == Jan Kiszka <jan.kiszka@web.de> writes:

Jan> Yeah, seen this (in the testsuite). But such objects neither have
Jan> addresses, nor do they help with non-constant objects I'm interested in.

Yeah.  You can use this for "offsetof" but it sounds like you wanted
more.

Tom> Whoops, this seems to be undocumented.

Jan> Not the only piece... :)

Please report anything you think is missing.  We try very hard to
document the entirety of the Python API; anything missing is due to an
oversight.  I will write doc patches for anything you "don't" find ;)

Tom

Re: container_of equivalent in gdb-python script

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 902 bytes --]

Tom Tromey wrote:
>>>>>> "Jan" == Jan Kiszka <jan.kiszka@web.de> writes:
> 
> Jan> Yeah, seen this (in the testsuite). But such objects neither have
> Jan> addresses, nor do they help with non-constant objects I'm interested in.
> 
> Yeah.  You can use this for "offsetof" but it sounds like you wanted
> more.

Yep, essentially instantiate a gdb.Value object of arbitrary type at an
arbitrary address.

> 
> Tom> Whoops, this seems to be undocumented.
> 
> Jan> Not the only piece... :)
> 
> Please report anything you think is missing.  We try very hard to
> document the entirety of the Python API; anything missing is due to an
> oversight.  I will write doc patches for anything you "don't" find ;)
> 

I have to correct myself: Actually looking at the right documentation, I
find all the latest features described (I was probably still starring at
6.8 docs...).

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]