Python/MI/STL visualization

Python/MI/STL visualization

[Vladimir Prus]

[-- Attachment #1: Type: text/plain, Size: 2750 bytes --]


Hi,
I've just implement the logic for computing the children of MI varobj
using Python scripts. Using the attached .gdbinit, I can do the following:

  -var-create V * v
  ^done,name="V",numchild="1",value="{...}",type="std::vector<int,std::allocator<int> >"
  (gdb)
  -var-set-visualizer V VectorVisualizer
  ^done
  (gdb)
  -var-list-children --all-values V
  ^done,numchild="2",children=[
  child={name="V.0",exp="0",numchild="0",value="1",type="int"},
  child={name="V.1",exp="1",numchild="0",value="2",type="int"}]

Previously, we discussed how to best report the case where the number of children of
varobj changes. The approach I've implemented is for -var-update to report the varobj
that had the number of children changed, and include the new 'children' attribute
for that varobj. So, if I had a vector of 1 element and push another other, I get
this:

  -var-update V
  ^done,changelist=[{name="V",in_scope="true",type_changed="false",
     children=[{name="V.0",exp="0",numchild="0",type="int"},
               {name="V.1",exp="1",numchild="0",type="int"}]}]

I like this approach because it does not assume that children are added or removed
at back -- if the list of children change, we report the entire new list, and can
put new varobj in the middle.

On Python level, visualization is handled by a Python class instance -- one instance
per varobj. This approach, as opposed to a function, allows Python code to do
whatever caching it sees fit.

Of course, there are quite some issues and questions:

1. Vectors can get large, and getting them can get slow. Do we want to have
incremental fetch of some kind? On UI level, I'm thinking of something
like KDevelop's incremental fetch of frames, see 
http://vladimir_prus.blogspot.com/2007/02/debugger-stories-stack-widget.html
But we also need MI level support.

2. Presumably, it's better to automatically assign visualizers to varobjs
of specific types. What's the best way to specify 'all vectors'. Does
using regexps seem good enough?

3. One can have vector of vectors. However, present code requires the visualizers
be explicitly set for each element of the outer vector. Should be have some way to
set visualizers on all future children of a given varobj? I'm not quite sure
how std::map will be presented, but probably we will have children of different
types. Then, should we have a way to set visualizers on all future children of 
specific type.

4. We still have the problem that GCC debug information will say that variable
exists even before the constructor for that variable has run. So, creating 
robust visualizer is rather hard. Anybody knows if we can workaround this?
I think even comparing declaration line with the current line is better than
nothing.

- Volodya

[-- Attachment #2: .gdbinit --]
[-- Type: text/x-java, Size: 646 bytes --]

python
class StringVisualizer:
    def to_string (self, v):
        # FIXME: catch any exceptions accessing bogus memory
        data = v.element ("_M_dataplus")
        return str (data.element ("_M_p"))
class VectorVisualizer:
    def children (self, v):
        result = []
        impl = v.element ('_M_impl')
        start = impl.element ('_M_start')
        finish = impl.element ('_M_finish')
        current = start
        index = 0
        while not current.equals(finish):
            result.append((str(index), current.dereference()))
            current = current.increment(1)
            index = index + 1
        return result
end

Re: Python/MI/STL visualization

[Vladimir Prus]

Vladimir Prus wrote:

> 
> Hi,
> I've just implement the logic for computing the children of MI varobj
> using Python scripts. 

Oh, and then code itself can be found at:

  http://gitorious.org/projects/gdb-python/repos/mainline/logs/python-revisited

- Volodya

Re: Python/MI/STL visualization

[André Pönitz]

On Tuesday 22 April 2008 22:06:54 Vladimir Prus wrote:
> Hi,

Hi Vladimir.

> I've just implement the logic for computing the children of MI varobj
> using Python scripts. Using the attached .gdbinit, I can do the following:
> 
>   -var-create V * v
>   ^done,name="V",numchild="1",value="{...}",type="std::vector<int,std::allocator<int> >"
>   (gdb)
>   -var-set-visualizer V VectorVisualizer
>   ^done
>   (gdb)
>   -var-list-children --all-values V
>   ^done,numchild="2",children=[
>   child={name="V.0",exp="0",numchild="0",value="1",type="int"},
>   child={name="V.1",exp="1",numchild="0",value="2",type="int"}]

Nice.

[Btw, one thing I do when dumping large MI-style data is to 

   ^done,numchild="2",childtyp="int",childnumchild="0",children=[
   child={name="V.0",exp="0",value="1"},
   child={name="V.1",exp="1",value="2"}]

Not exactly good syntax but reduces communication effort which
shows when dumping large amount of data over a slow connection
especially if the 'childtype' is something lengthy ;-). I'd argue it makes
even the output better readable for human inspectors.]

> Previously, we discussed how to best report the case where the number of children of
> varobj changes. The approach I've implemented is for -var-update to report the varobj
> that had the number of children changed, and include the new 'children' attribute
> for that varobj. So, if I had a vector of 1 element and push another other, I get
> this:
> 
>   -var-update V
>   ^done,changelist=[{name="V",in_scope="true",type_changed="false",
>      children=[{name="V.0",exp="0",numchild="0",type="int"},
>                {name="V.1",exp="1",numchild="0",type="int"}]}]
>
> I like this approach because it does not assume that children are added or removed
> at back -- if the list of children change, we report the entire new list, and can
> put new varobj in the middle.
> 
> On Python level, visualization is handled by a Python class instance -- one instance
> per varobj. This approach, as opposed to a function, allows Python code to do
> whatever caching it sees fit.
> 
> Of course, there are quite some issues and questions:
> 
> 1. Vectors can get large, and getting them can get slow. Do we want to have
> incremental fetch of some kind? On UI level, I'm thinking of something
> like KDevelop's incremental fetch of frames, see 
> http://vladimir_prus.blogspot.com/2007/02/debugger-stories-stack-widget.html
> But we also need MI level support.

Specifying a 'child range' might do the trick. Maybe

  -var-set-child-range V <from> <to>
  -var-set-child-range V *      /* reset */

or such, and  whenever data is reported, only children in the range
 <from>...<to> are reported. 

> 2. Presumably, it's better to automatically assign visualizers to varobjs
> of specific types. What's the best way to specify 'all vectors'. Does
> using regexps seem good enough?

I would think so. It is nice when one can distinguish three cases
(simple type, pointers, "the rest"), and this should be possible with
regexps.
 
> 3. One can have vector of vectors. However, present code requires the visualizers 
> be explicitly set for each element of the outer vector. Should be have some way to 
> set visualizers on all future children of a given varobj? I'm not quite sure
> how std::map will be presented, but probably we will have children of different
> types. Then, should we have a way to set visualizers on all future children of 
> specific type.

I think visualizers are best set _by type_, not by var object. It's hard to
imagine a scenario where this is not wanted.

> 4. We still have the problem that GCC debug information will say that variable
> exists even before the constructor for that variable has run. So, creating 
> robust visualizer is rather hard. Anybody knows if we can workaround this?

Well, I run "visualizer code" injected into the inferior and trigger it with
gdb's "call". I do not even try to make them robust after noticing that 
there is not enough information available. But simply letting them crash
and handling the segfault works rather nicely as far as I can tell ;-)

> I think even comparing declaration line with the current line is better than
> nothing.

"Nothing" might be ok ;-)

Andre'

Re: Python/MI/STL visualization

[Vladimir Prus]

André Pönitz wrote:

> On Tuesday 22 April 2008 22:06:54 Vladimir Prus wrote:
>> Hi,
> 
> Hi Vladimir.
> 
>> I've just implement the logic for computing the children of MI varobj
>> using Python scripts. Using the attached .gdbinit, I can do the following:
>> 
>>   -var-create V * v
>>   ^done,name="V",numchild="1",value="{...}",type="std::vector<int,std::allocator<int> >"
>>   (gdb)
>>   -var-set-visualizer V VectorVisualizer
>>   ^done
>>   (gdb)
>>   -var-list-children --all-values V
>>   ^done,numchild="2",children=[
>>   child={name="V.0",exp="0",numchild="0",value="1",type="int"},
>>   child={name="V.1",exp="1",numchild="0",value="2",type="int"}]
> 
> Nice.
> 
> [Btw, one thing I do when dumping large MI-style data is to
> 
>    ^done,numchild="2",childtyp="int",childnumchild="0",children=[
>    child={name="V.0",exp="0",value="1"},
>    child={name="V.1",exp="1",value="2"}]
> 
> Not exactly good syntax but reduces communication effort which
> shows when dumping large amount of data over a slow connection
> especially if the 'childtype' is something lengthy ;-). I'd argue it makes
> even the output better readable for human inspectors.]

This might be a good change for future. 

>> Previously, we discussed how to best report the case where the number of children of
>> varobj changes. The approach I've implemented is for -var-update to report the varobj
>> that had the number of children changed, and include the new 'children' attribute
>> for that varobj. So, if I had a vector of 1 element and push another other, I get
>> this:
>> 
>>   -var-update V
>>   ^done,changelist=[{name="V",in_scope="true",type_changed="false",
>>      children=[{name="V.0",exp="0",numchild="0",type="int"},
>>                {name="V.1",exp="1",numchild="0",type="int"}]}]
>>
>> I like this approach because it does not assume that children are added or removed
>> at back -- if the list of children change, we report the entire new list, and can
>> put new varobj in the middle.
>> 
>> On Python level, visualization is handled by a Python class instance -- one instance
>> per varobj. This approach, as opposed to a function, allows Python code to do
>> whatever caching it sees fit.
>> 
>> Of course, there are quite some issues and questions:
>> 
>> 1. Vectors can get large, and getting them can get slow. Do we want to have
>> incremental fetch of some kind? On UI level, I'm thinking of something
>> like KDevelop's incremental fetch of frames, see
>> http://vladimir_prus.blogspot.com/2007/02/debugger-stories-stack-widget.html
>> But we also need MI level support.
> 
> Specifying a 'child range' might do the trick. Maybe
> 
>   -var-set-child-range V <from> <to>
>   -var-set-child-range V *      /* reset */
> 
> or such, and  whenever data is reported, only children in the range
>  <from>...<to> are reported.

Or adding range parameter to -var-list-children. In either case, I'm not yet sure 
how to make Python interface support this. Of course, we can always make python
side use iterators, which will work for the case where we fetch more children at
the end.

>> 2. Presumably, it's better to automatically assign visualizers to varobjs
>> of specific types. What's the best way to specify 'all vectors'. Does
>> using regexps seem good enough?
> 
> I would think so. It is nice when one can distinguish three cases
> (simple type, pointers, "the rest"), and this should be possible with
> regexps.

Okay.

>> 3. One can have vector of vectors. However, present code requires the visualizers
>> be explicitly set for each element of the outer vector. Should be have some way to
>> set visualizers on all future children of a given varobj? I'm not quite sure
>> how std::map will be presented, but probably we will have children of different
>> types. Then, should we have a way to set visualizers on all future children of
>> specific type.
> 
> I think visualizers are best set _by type_, not by var object. It's hard to
> imagine a scenario where this is not wanted.

Okay.
 
>> 4. We still have the problem that GCC debug information will say that variable
>> exists even before the constructor for that variable has run. So, creating
>> robust visualizer is rather hard. Anybody knows if we can workaround this?
> 
> Well, I run "visualizer code" injected into the inferior and trigger it with
> gdb's "call". I do not even try to make them robust after noticing that
> there is not enough information available. But simply letting them crash
> and handling the segfault works rather nicely as far as I can tell ;-)

Well, segfaul is not scary. What is scary is if due to uninitialized data,
the amount of the children the visualizer wants to fetch end up been 10000 :-)

- Volodya

Re: Python/MI/STL visualization

[André Pönitz]

On Wednesday 23 April 2008 09:46:07 Vladimir Prus wrote:
> >> 1. Vectors can get large, and getting them can get slow. Do we want to have
> >> incremental fetch of some kind? On UI level, I'm thinking of something
> >> like KDevelop's incremental fetch of frames, see
> >> http://vladimir_prus.blogspot.com/2007/02/debugger-stories-stack-widget.html
> >> But we also need MI level support.
> > 
> > Specifying a 'child range' might do the trick. Maybe
> > 
> >   -var-set-child-range V <from> <to>
> >   -var-set-child-range V *      /* reset */
> > 
> > or such, and  whenever data is reported, only children in the range
> >  <from>...<to> are reported.
> 
> Or adding range parameter to -var-list-children. In either case, I'm not yet sure 
> how to make Python interface support this. Of course, we can always make python
> side use iterators, which will work for the case where we fetch more children at
> the end.

Or that ;-)

>  [...]
> >> 4. We still have the problem that GCC debug information will say that variable
> >> exists even before the constructor for that variable has run. So, creating
> >> robust visualizer is rather hard. Anybody knows if we can workaround this?
> > 
> > Well, I run "visualizer code" injected into the inferior and trigger it with
> > gdb's "call". I do not even try to make them robust after noticing that
> > there is not enough information available. But simply letting them crash
> > and handling the segfault works rather nicely as far as I can tell ;-)
> 
> Well, segfaul is not scary. What is scary is if due to uninitialized data,
> the amount of the children the visualizer wants to fetch end up been 10000 :-)

Indeed.

My "solution" here is to wildly poke around in the structure hoping to 
trigger a segfault early and interpret such segfault as "uninitialized object".

So if there's a std::vector<> with size() reporting a non-0 value, try
to access the first and last element and one from the middle and 
hope that at least one of these addresses are "bad" ;-)

In practice, this works rather nicely. The bigger the reported size()
gets the more potential harm would actually listing the children do,
but the more likely it also is to recognize the object as uninitialized.

Of course, not having to play such "games" would be much nicer ;-}

Andre'

Re: Python/MI/STL visualization

[Vladimir Prus]

André Pönitz wrote:

 >> Well, segfaul is not scary. What is scary is if due to uninitialized data,
>> the amount of the children the visualizer wants to fetch end up been 10000 :-)
> 
> Indeed.
> 
> My "solution" here is to wildly poke around in the structure hoping to
> trigger a segfault early and interpret such segfault as "uninitialized object".
> 
> So if there's a std::vector<> with size() reporting a non-0 value, try
> to access the first and last element and one from the middle and
> hope that at least one of these addresses are "bad" ;-)
> 
> In practice, this works rather nicely. The bigger the reported size()
> gets the more potential harm would actually listing the children do,
> but the more likely it also is to recognize the object as uninitialized.

Oh, I think it's a nice trick :-)

- Volodya