Re: question: python gc doesn't collect buffer allocated by read_memory()

[HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com>]

Hello Tom,

From: Tom Tromey <tromey@redhat.com>
Subject: Re: question: python gc doesn't collect buffer allocated by read_memory()
Date: Mon, 26 Mar 2012 10:58:22 -0600

>>>>>> ">" == HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com> writes:
> 
>>> Sorry for missing. I first found this on gdb-7.2-48.el6.x86_64. I used
>>> 7.4 in the presentation of the first mail.
> 
> Thanks.
> 
>>> On the other hand, it appears to me that buffer objects returned by
>>> inferior.read_memory() is never collected by gc.collect().
> 
> I think I found the problem.
> 
> The issue is that PyBuffer_FromReadWriteObject acquires a reference to
> the base object -- but the code in gdb assumed that it stole a reference.
> 
> Could you try the appended patch?
> It works for me; if it works for you, I will put it in.
> If you can't try it, I'll just assume it is ok and go ahead...
> 
> Tom
> 

Thanks! I was also looking into py-inferior.c but I've first started
with studying python gc and it must have been taken more time.

I tried your patch using gdb of today's daily update and the same
script below.

import gdb
import gc

i = gdb.inferiors()[0]
buf = gdb.parse_and_eval('buf')

count = 100000
while count >= 0:
    i.read_memory(buf.address, buf.type.sizeof)
    count -= 1
gc.collect()

I typed ps aux command to see VSZ/RSS each time I executed the script
above that reads 4KB buffer 100000 times so about 390MB total.

[Before]

$ ps aux | head -1
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
$ ps aux | grep gdb | grep -v grep
hat      28438  1.0  0.3  81936 12340 pts/1    S+   10:15   0:00 /media/pub/repos/gdb/gdb/gdb ./testpro ./core.27403
$ ps aux | grep gdb | grep -v grep
hat      28438  9.3 10.7 491876 422296 pts/1   S+   10:15   0:01 /media/pub/repos/gdb/gdb/gdb ./testpro ./core.27403

400 MB increased.

[After]

$ ps aux | head -1
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
$ ps aux | grep gdb | grep -v grep
hat      28446  8.6  0.4  86164 16756 pts/1    S+   10:15   0:00 /media/pub/repos/gdb/gdb.fixed ./testpro ./core.27403
$ ps aux | grep gdb | grep -v grep
hat      28446  1.5  0.5  90928 21444 pts/1    S+   10:15   0:01 /media/pub/repos/gdb/gdb.fixed ./testpro ./core.27403

4 MB increased.

So, it seems to me that buffers allocated in infpy_read_memory() are
now collected sanely. I confirm the issue I reported has been fixed.


BTW, there is still about 4MB increment. I tried two more times, then
still saw constant increase.

$ ps aux | grep gdb | grep -v grep
hat      28446  2.1  0.6  95548 26132 pts/1    S+   10:15   0:02 /media/pub/repos/gdb/gdb.fixed ./testpro ./core.27403
$ ps aux | grep gdb | grep -v grep
hat      28446  0.7  0.7 100304 30820 pts/1    S+   10:15   0:02 /media/pub/repos/gdb/gdb.fixed ./testpro ./core.27403

And the increment can be reproduced by simplifying the script to
i.read_memory() only.

(gdb) python
>count=100000
>while count >= 0:
>  i.read_memory(addr, size)
>  count -= 1
>gc.collect()
>end

I suspect another objects allocated remain while not collected.

Thanks.
HATAYAMA, Daisuke