[help] Calling malloc() from a Python pretty-printer

[help] Calling malloc() from a Python pretty-printer

[Marc Mezzarobba]

Dear gdb gurus,

(This is a repost of a question that I sent to the gdb@gnu mailing list 
a few days ago. My apologies to people who read both lists!)

Is it supposed to be okay to call an inferior's function, and 
specifically malloc(), from a Python pretty-printer?

My understanding of the documentation was that it should work. But when 
my pretty-printer that calls malloc() is invoked while the selected 
stack frame is not the innermost one, gdb complains that it detected an 
internal problem or just crashes. What am I doing wrong? Is there a fine 
print I missed? Or is that a bug?

Here is a complete example:


==> foo.c <==
struct foo { int val; };

int bar(struct foo x) {
    struct foo y = x;
    --y.val;
    if (!y.val) return 0;
    return bar(y);
}

int main(void) {
    struct foo x = { .val = 42 };
    return bar(x);
}

==> foo-gdb.py <==
class Printer(object):
    def __init__(self):
        pass
    def to_string(self):
        gdb.lookup_symbol("malloc")[0].value()(256)
        return "tada"

def lookup_type(val):
    return Printer()

gdb.printing.register_pretty_printer(gdb, lookup_type)

==> transcript <==
~/docs/vrac/pygdb$ gdb foo
GNU gdb (Debian 7.7.1+dfsg-3) 7.7.1
[...]
Reading symbols from foo...done.
(gdb) break bar
Breakpoint 1 at 0x4004c1: file foo.c, line 4.
(gdb) r
Starting program: /home/marc/docs/vrac/pygdb/foo 

Breakpoint 1, bar (x=tada) at foo.c:4
4           struct foo y = x;
(gdb) c 10
Will ignore next 9 crossings of breakpoint 1.  Continuing.

Breakpoint 1, bar (x=tada) at foo.c:4
4           struct foo y = x;
(gdb) up
#1  0x00000000004004e8 in bar (x=tada) at foo.c:7
7           return bar(y);
/home/zumbi/gdb-7.7.1+dfsg/gdb/frame.c:2139: internal-error: 
get_frame_pc_if_available: Assertion `frame->next != NULL' failed.             
A problem internal to GDB has been detected,                          
further debugging may prove unreliable.                               
Quit this debugging session? (y or n) y

/home/zumbi/gdb-7.7.1+dfsg/gdb/frame.c:2139: internal-error: 
get_frame_pc_if_available: Assertion `frame->next != NULL' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.


(In case someone has a better approach to suggest, here is what I am 
trying to achieve. I am working with a library that provides a version 
of sprintf() for its custom data structures, and I would like to write a 
lightweight pretty-printer that reuses this sprintf(). Given my use 
cases, I don't think it is much of a problem if the pretty-printer needs 
to be disabled to debug some issues where the additional allocations are 
likely to interact with the actual problem. And if possible I would like 
to avoid writing a separate Python interface for the library...)

Can someone help me?


Thanks a lot,

-- 
Marc Mezzarobba

Re: [help] Calling malloc() from a Python pretty-printer

[Phil Muldoon]

On 23/09/14 09:10, Marc Mezzarobba wrote:
> Dear gdb gurus,
>
> (This is a repost of a question that I sent to the gdb@gnu mailing list
> a few days ago. My apologies to people who read both lists!)
>
> Is it supposed to be okay to call an inferior's function, and
> specifically malloc(), from a Python pretty-printer?

Though not expressly forbidden it is highly discouraged to call (in
GDB parlance, an "inferior function call") in a pretty-printer.  The
results are too unpredictable.  The inferior call might generate a
signal (which in GDB will cause an error and the inferior will be
stopped at the signal emission phase), or hit a breakpoint (similar I
think to the signal), or a whole host of things where the pretty
printer looses control of the inferior.  An inferior function call
generates a temporary "dummy" frame to execute the call, and there are
several specialized conditions and limitations to be aware of (some of
which I detailed earlier.)


> My understanding of the documentation was that it should work.

It should, but see above.

> But when 
> my pretty-printer that calls malloc() is invoked while the selected
> stack frame is not the innermost one, gdb complains that it detected an
> internal problem or just crashes. What am I doing wrong? Is there a fine
> print I missed? Or is that a bug?

That is a bug.  Your call stack is smashed now, and GDB is saying it
cannot find the next frame.  This should never happen. I would be
curious to see what the result of:

(gdb) call malloc (1024)

are from your inferior, or in your pretty-printer the equivalent:

foo = gdb.parse_and_eval (malloc (1024))

Also a backtrace from the crashing GDB would be optimal if you can
generate one.

> (In case someone has a better approach to suggest, here is what I am
> trying to achieve. I am working with a library that provides a version
> of sprintf() for its custom data structures, and I would like to write a
> lightweight pretty-printer that reuses this sprintf(). Given my use
> cases, I don't think it is much of a problem if the pretty-printer needs
> to be disabled to debug some issues where the additional allocations are
> likely to interact with the actual problem. And if possible I would like
> to avoid writing a separate Python interface for the library...)

I am curious why you need to call malloc? Generally if you need to
store information about an inferior during pretty-printing, it is
often preferable to just allocate that storage in python.  So say you
needed to store an array of integers during you call, instead of doing
something like:

foo = gdb.parse_and_eval (calloc (100, sizeof(int)))

Just instead create a python list in the pretty printer:

foo = []

Then add to the list in Python.  You can make such lists last the
duration of the pretty printing call, or global.  Usual Python syntax
applies.

Cheers,

Phil

Re: [help] Calling malloc() from a Python pretty-printer

[Marc Mezzarobba]

Phil Muldoon wrote:
> That is a bug.  Your call stack is smashed now, and GDB is saying it
> cannot find the next frame.  This should never happen. I would be
> curious to see what the result of:
> 
> (gdb) call malloc (1024)
> 
> are from your inferior,

Thank you very much for your reply.

Here is what I get in various situations (after removing the call to
malloc from the pretty-printer):

(gdb) r
Starting program: /home/marc/docs/vrac/pygdb/foo 

Breakpoint 1, bar (x=tada) at foo.c:4
4           struct foo y = x;
(gdb) call malloc (1024)
$1 = (void *) 0x601010
(gdb) c 10
Will ignore next 9 crossings of breakpoint 1.  Continuing.

Breakpoint 1, bar (x=tada) at foo.c:4
4           struct foo y = x;
(gdb) call malloc (1024)
$2 = (void *) 0x601420
(gdb) up
#1  0x00000000004004e8 in bar (x=tada) at foo.c:7
7           return bar(y);
(gdb) call malloc (1024)
$3 = (void *) 0x601830

> or in your pretty-printer the equivalent:
> 
> foo = gdb.parse_and_eval (malloc (1024))

The return values of malloc() are exactly the same as above, but (like
in my original example) gdb fails after printing

#1  0x00000000004004e8 in bar (x=tada) at foo.c:7
7           return bar(y);

> Also a backtrace from the crashing GDB would be optimal if you can
> generate one.

Here is what I managed to get. Note that this is not with foo.c but
with a significantly more complicated inferior (involving a dlopened
object that calls the Maple C API, with both Maple and the master
program using an LD_PRELOADed libgmp...). Also, crashes apparently
tend to occur when the inferior was itself executing memory
management functions when it was interrupted. I'm also including a
partial backtrace of the inferior in case this is relevant.

GDB:

#0  frame_unwind_pc (this_frame=0x170002)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/frame.c:761
#1  0x000000000065d48c in get_frame_address_in_block (
    this_frame=this_frame@entry=0x2409e00)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/frame.c:2162
#2  0x000000000065d592 in get_frame_address_in_block_if_available (
    this_frame=this_frame@entry=0x2409e00, pc=pc@entry=0x7fff3f753040)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/frame.c:2226
#3  0x000000000051b514 in get_frame_block (frame=0x2409e00, 
    addr_in_block=addr_in_block@entry=0x0)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/blockframe.c:62
#4  0x0000000000607e6d in dwarf_expr_frame_base (baton=0x7fff3f7531c0, 
    start=0x7fff3f7530f8, length=0x7fff3f753100)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/dwarf2loc.c:360
#5  0x00000000006027b9 in execute_stack_op (ctx=ctx@entry=0x1748db0, 
    op_ptr=0x7fd7f749db02 "\"h", op_ptr@entry=0x7fd7f749daff "\221\340|\"h", 
    op_end=op_end@entry=0x7fd7f749db02 "\"h")
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/dwarf2expr.c:954
#6  0x0000000000603a24 in dwarf_expr_eval (ctx=ctx@entry=0x1748db0, 
    addr=addr@entry=0x7fd7f749daff "\221\340|\"h", len=len@entry=3)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/dwarf2expr.c:364
#7  0x0000000000604994 in dwarf2_evaluate_loc_desc_full (type=0x24977f0, 
    frame=0x2409e00, data=0x7fd7f749daff "\221\340|\"h", size=3, 
    per_cu=0xfd3490, byte_offset=0)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/dwarf2loc.c:2251
#8  0x000000000053576a in default_read_var_value (var=0x24c29e0, 
    frame=0x2409e00) at /home/zumbi/gdb-7.7.1+dfsg/gdb/findvar.c:439
#9  0x000000000058b264 in read_frame_arg (sym=sym@entry=0x24c29e0, 
    frame=frame@entry=0x2409e00, argp=argp@entry=0x7fff3f753370, 
    entryargp=entryargp@entry=0x7fff3f753390)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/stack.c:345
#10 0x000000000058bbfd in print_frame_args (func=<optimized out>, 
    frame=frame@entry=0x2409e00, num=num@entry=-1, stream=0xf4a410)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/stack.c:672
#11 0x000000000058c59f in print_frame (frame=0x2409e00, 
    print_level=<optimized out>, print_what=SRC_AND_LOC, print_args=1, sal=...)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/stack.c:1203
#12 0x000000000058ca8f in print_frame_info (frame=0x2409e00, print_level=1, 
    print_what=SRC_AND_LOC, print_args=-1029483744, print_args@entry=1, 
    set_current_sal=1) at /home/zumbi/gdb-7.7.1+dfsg/gdb/stack.c:855
#13 0x000000000058cc7f in print_stack_frame (frame=0x2409e00, print_level=1, 
    print_what=SRC_AND_LOC, set_current_sal=1)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/stack.c:170
#14 0x0000000000653acf in execute_command (p=<optimized out>, 
    p@entry=0xdbc230 "up", from_tty=1)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/top.c:468
#15 0x000000000059c335 in command_handler (command=0xdbc230 "up")
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/event-top.c:435
#16 0x000000000059c8f4 in command_line_handler (rl=<optimized out>)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/event-top.c:632
#17 0x0000003fca22d52e in rl_callback_read_char ()
   from /lib/x86_64-linux-gnu/libreadline.so.6
#18 0x000000000059c399 in rl_callback_read_char_wrapper (
    client_data=<optimized out>)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/event-top.c:164
#19 0x000000000059ae71 in process_event ()
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/event-loop.c:342
#20 0x000000000059b2b7 in gdb_do_one_event ()
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/event-loop.c:406
#21 0x000000000059b4de in start_event_loop ()
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/event-loop.c:431
#22 0x00000000005949d3 in captured_command_loop (data=data@entry=0x0)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/main.c:267
#23 0x0000000000592a8a in catch_errors (
    func=func@entry=0x5949c0 <captured_command_loop>, 
    func_args=func_args@entry=0x0, errstring=errstring@entry=0x73fcf5 "", 
    mask=mask@entry=RETURN_MASK_ALL)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/exceptions.c:524
#24 0x0000000000595896 in captured_main (data=data@entry=0x7fff3f7539f0)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/main.c:1067
#25 0x0000000000592a8a in catch_errors (
    func=func@entry=0x594d70 <captured_main>, 
    func_args=func_args@entry=0x7fff3f7539f0, 
    errstring=errstring@entry=0x73fcf5 "", mask=mask@entry=RETURN_MASK_ALL)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/exceptions.c:524
#26 0x0000000000595d64 in gdb_main (args=args@entry=0x7fff3f7539f0)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/main.c:1076
#27 0x000000000045484e in main (argc=<optimized out>, argv=<optimized out>)
    at /home/zumbi/gdb-7.7.1+dfsg/gdb/gdb.c:34

Inferior (frame #7 is the first one that triggers the display hook in
this case, and gdb crashes after displaying the opening parenthesis):

#1  0x000000000050a487 in safeRealloc (ptr=0xa593a0, size=16) at general.c:429
#2  0x000000000050a50f in wrapSafeRealloc (ptr=0xa593a0, old_size=8, 
    new_size=16) at general.c:444
#3  0x00007ffff7ed9544 in __gmpz_realloc ()
   from /home/marc/opt/maple/17/bin.X86_64_LINUX/libgmp.so
#4  0x00007ffff7ed6638 in __gmpz_mul_2exp ()
   from /home/marc/opt/maple/17/bin.X86_64_LINUX/libgmp.so
#5  0x0000003429039c4f in mpfr_get_z ()
   from /usr/lib/x86_64-linux-gnu/libmpfr.so.4
#6  0x000000342901c0f5 in mpfr_pow ()
   from /usr/lib/x86_64-linux-gnu/libmpfr.so.4
#7  0x000000000055e5fc in __tryFaithEvaluationOptimizedPow (


>> (In case someone has a better approach to suggest, here is what I am
>> trying to achieve. I am working with a library that provides a
>> version of sprintf() for its custom data structures, and I would like
>> to write a lightweight pretty-printer that reuses this sprintf().
>> Given my use cases, I don't think it is much of a problem if the
>> pretty-printer needs to be disabled to debug some issues where the
>> additional allocations are likely to interact with the actual
>> problem. And if possible I would like to avoid writing a separate
>> Python interface for the library...)
> 
> I am curious why you need to call malloc?

This was just the first thing I tried in order to obtain a buffer for
sprintf when I started experimenting with pretty-printers...

Thanks again,

-- 
Marc