[PATCH:MI] Return a subset of a variable object's children

[PATCH:MI] Return a subset of a variable object's children

[Nick Roberts]

This patch allows -var-list-children to create just some of a variable object's
children.  Following a discussion last year with Mark Khouzam, this is useful
for displaying large arrays (and structures) where it would take too long to
create all the children: it is only really necessary to create those that are
visible on the screen and update as further become visible.  It also allows a
step size (stride) other than one.

ISTR that it was said a long time ago that mi_getopt won't take long options
but that doesn't seem to be the case.  The difficulty is that 
-var-list-children takes 0, 1 and 2 as options so I've had to separate the
logic to keep it backward compatible.

Use of these new options means that the elements might not be printed in order
but it is always preserved if only the old options are used.

Example of use (Note subsequent -var-list-children _adds_ to list):

n
&"n\n"
~~"151\t  int i, n, m[10] = {0, 1, 4, 9, 16, 25, 36, 49, 64, 81};\n"
^done
(gdb) 
n
&"n\n"
~"152\t  int m1[2][3] = {{0, 1, 2}, {10, 11, 12}};\n"
^done
(gdb) 
-var-create - * m
^done,name="var1",numchild="10",value="[10]",type="int [10]",thread-id="1"
(gdb) 
-var-list-children -f 7 -n 2 --all-values var1
^done,numchild="2",children=[child={name="var1.7",exp="7",numchild="0",value="49",type="int",thread-id="1"},child={name="var1.8",exp="8",numchild="0",value="64",type="int",thread-id="1"}]
(gdb) 
-var-list-children -f 4 -n 3 -s 2 --all-values var1
^done,numchild="4",children=[child={name="var1.7",exp="7",numchild="0",value="49",type="int",thread-id="1"},child={name="var1.8",exp="8",numchild="0",value="64",type="int",thread-id="1"},child={name="var1.4",exp="4",numchild="0",value="16",type="int",thread-id="1"},child={name="var1.6",exp="6",numchild="0",value="36",type="int",thread-id="1"}]
(gdb) 


-- 
Nick                                           http://www.inet.net.nz/~nickrob


2008-04-27  Nick Roberts  <nickrob@snap.net.nz>

	* mi/mi-cmd-var.c (mi_cmd_var_list_children): Add options to select
	a subset of children.

	* varobj.h (varobj_list_children): Declare new parameters.

	* varobj.c (struct varobj): New member current_children.
	(varobj_list_children): Create any new varobjs for children
	specified by mi_cmd_var_list_children.
	(create_child): Add parameter real_index.  Use it.


*** mi-cmd-var.c	20 Apr 2008 10:20:39 +1200	1.50
--- mi-cmd-var.c	27 Apr 2008 21:34:58 +1200	
*************** mi_cmd_var_list_children (char *command,
*** 374,397 ****
    int numchild;
    enum print_values print_values;
    int ix;
  
!   if (argc != 1 && argc != 2)
!     error (_("mi_cmd_var_list_children: Usage: [PRINT_VALUES] NAME"));
  
!   /* Get varobj handle, if a valid var obj name was specified */
!   if (argc == 1)
!     var = varobj_get_handle (argv[0]);
    else
!     var = varobj_get_handle (argv[1]);
!   if (var == NULL)
!     error (_("Variable object not found"));
  
!   children = varobj_list_children (var);
    ui_out_field_int (uiout, "numchild", VEC_length (varobj_p, children));
-   if (argc == 2)
-     print_values = mi_parse_values_option (argv[0]);
-   else
-     print_values = PRINT_NO_VALUES;
  
    if (VEC_length (varobj_p, children) == 0)
      return MI_CMD_DONE;
--- 374,462 ----
    int numchild;
    enum print_values print_values;
    int ix;
+   int start, number, stride = 1;
  
!   enum opt
!     {
!       NO_VALUES, SIMPLE_VALUES, ALL_VALUES, FROM_OPT, NUMBER_OPT, STRIDE_OPT
!     };
  
!   static struct mi_opt opts[] =
!   {
!       { "-no-values", NO_VALUES, 0 },
!       { "-simple-values", SIMPLE_VALUES, 0 },
!       { "-all-values", ALL_VALUES, 0 },
!       { "f", FROM_OPT, 1 },
!       { "n", NUMBER_OPT, 1 },
!       { "s", STRIDE_OPT, 1 },
!       { 0, 0, 0 }
!     };
! 
!   if (argc == 1 || argc == 2)
!     {
!       /* Get varobj handle, if a valid var obj name was specified */
!       if (argc == 1)
! 	var = varobj_get_handle (argv[0]);
!       else
! 	var = varobj_get_handle (argv[1]);
!       if (var == NULL)
! 	error (_("Variable object not found"));
! 
!       if (argc == 2)
! 	print_values = mi_parse_values_option (argv[0]);
!       else
! 	print_values = PRINT_NO_VALUES;
! 
!       start = 0;
!       number = varobj_get_num_children (var);
!     }
    else
!     {
!       int optind = 0;
!       char *optarg;
!       while (1)
! 	{
! 	  int opt = mi_getopt ("mi_cmd_var_list_children",
! 			       argc, argv, opts, &optind, &optarg);
! 	  if (opt < 0)
! 	    break;
! 	  switch ((enum opt) opt)
! 	    {
! 	    case NO_VALUES:
! 	      print_values = PRINT_NO_VALUES;
! 	      break;
! 	    case SIMPLE_VALUES:
! 	      print_values = PRINT_SIMPLE_VALUES;
! 	      break;
! 	    case ALL_VALUES:
! 	      print_values = PRINT_ALL_VALUES;
! 	      break;
! 	    case FROM_OPT:
! 	      start = atol (optarg);
! 	      if (start < 0)
! 		start = 0;
! 	      break;
! 	    case NUMBER_OPT:
! 	      number = atol (optarg);
! 	      break;
! 	    case STRIDE_OPT:
! 	      stride = atol (optarg);
! 	      if (stride < 1)
! 	      	error (_("Positive stride values only"));
! 	      break;
! 	    }
! 	}
!       if (optind >= argc)
! 	error (_("Missing VARNAME"));
  
!       if (optind < argc - 1)
! 	error (_("Garbage at end of command"));
! 
!       var = varobj_get_handle (argv[optind]);
!     }
! 
!   children = varobj_list_children (var, start, number, stride);
    ui_out_field_int (uiout, "numchild", VEC_length (varobj_p, children));
  
    if (VEC_length (varobj_p, children) == 0)
      return MI_CMD_DONE;


*** varobj.h	10 Apr 2008 08:41:40 +1200	1.18
--- varobj.h	27 Apr 2008 15:14:51 +1200	
*************** extern int varobj_get_num_children (stru
*** 99,105 ****
  
  /* Return the list of children of VAR.  The returned vector
     should not be modified in any way.  */
! extern VEC (varobj_p)* varobj_list_children (struct varobj *var);
  
  extern char *varobj_get_type (struct varobj *var);
  
--- 99,106 ----
  
  /* Return the list of children of VAR.  The returned vector
     should not be modified in any way.  */
! extern VEC (varobj_p)* varobj_list_children (struct varobj *var, int start,
! 					     int number, int stride);
  
  extern char *varobj_get_type (struct varobj *var);
  


*** varobj.c	20 Apr 2008 10:20:39 +1200	1.111
--- varobj.c	27 Apr 2008 22:58:28 +1200	
*************** struct varobj
*** 131,136 ****
--- 131,140 ----
    /* The number of (immediate) children this variable has */
    int num_children;
  
+   /* The number of (immediate) children this variable that exist as varobjs.  
+      current_children < num_children  */
+   int current_children;
+ 
    /* If this object is a child, this points to its immediate parent. */
    struct varobj *parent;
  
*************** static int install_variable (struct varo
*** 187,193 ****
  
  static void uninstall_variable (struct varobj *);
  
! static struct varobj *create_child (struct varobj *, int, char *);
  
  /* Utility routines */
  
--- 191,197 ----
  
  static void uninstall_variable (struct varobj *);
  
! static struct varobj *create_child (struct varobj *, int, int, char *);
  
  /* Utility routines */
  
*************** varobj_get_num_children (struct varobj *
*** 752,762 ****
     the return code is the number of such children or -1 on error */
  
  VEC (varobj_p)*
! varobj_list_children (struct varobj *var)
  {
    struct varobj *child;
    char *name;
!   int i;
  
    if (var->num_children == -1)
      var->num_children = number_of_children (var);
--- 756,767 ----
     the return code is the number of such children or -1 on error */
  
  VEC (varobj_p)*
! varobj_list_children (struct varobj *var, int start, int number, int stride)
  {
    struct varobj *child;
    char *name;
!   int i, j, new_children = 0;
!   varobj_p varchild;
  
    if (var->num_children == -1)
      var->num_children = number_of_children (var);
*************** varobj_list_children (struct varobj *var
*** 765,790 ****
    if (var->num_children == -1)
      return var->children;
  
!   /* If we're called when the list of children is not yet initialized,
!      allocate enough elements in it.  */
!   while (VEC_length (varobj_p, var->children) < var->num_children)
!     VEC_safe_push (varobj_p, var->children, NULL);
! 
!   for (i = 0; i < var->num_children; i++)
!     {
!       varobj_p existing = VEC_index (varobj_p, var->children, i);
! 
!       if (existing == NULL)
! 	{
! 	  /* Either it's the first call to varobj_list_children for
! 	     this variable object, and the child was never created,
! 	     or it was explicitly deleted by the client.  */
! 	  name = name_of_child (var, i);
! 	  existing = create_child (var, i, name);
! 	  VEC_replace (varobj_p, var->children, i, existing);
  	}
      }
  
    return var->children;
  }
  
--- 770,800 ----
    if (var->num_children == -1)
      return var->children;
  
!   for (j = 0; j < number; j++)
!     {
!       int real_index = start + j * stride;
!       if (real_index >= var->num_children)
! 	break;
!       name = name_of_child (var, real_index);
! 
!       for (i = 0; i < var->current_children; i++)
! 	{
! 	  varchild = VEC_index (varobj_p, var->children, i);
! 
! 	  /* Child already exists */
! 	  if (strcmp (varchild->name, name) == 0)
! 	    break;
! 	}
!       /* No child found */
!       if (i == var->current_children)
! 	{
! 	  varchild = create_child (var, i + new_children, real_index, name);
! 	  VEC_safe_push (varobj_p, var->children, varchild);
! 	  new_children++;
  	}
      }
  
+   var->current_children += new_children;
    return var->children;
  }
  
*************** uninstall_variable (struct varobj *var)
*** 1442,1448 ****
  
  /* Create and install a child of the parent of the given name */
  static struct varobj *
! create_child (struct varobj *parent, int index, char *name)
  {
    struct varobj *child;
    char *childs_name;
--- 1452,1458 ----
  
  /* Create and install a child of the parent of the given name */
  static struct varobj *
! create_child (struct varobj *parent, int index, int real_index, char *name)
  {
    struct varobj *child;
    char *childs_name;
*************** create_child (struct varobj *parent, int
*** 1453,1459 ****
    /* name is allocated by name_of_child */
    child->name = name;
    child->index = index;
!   value = value_of_child (parent, index);
    child->parent = parent;
    child->root = parent->root;
    childs_name = xstrprintf ("%s.%s", parent->obj_name, name);
--- 1463,1469 ----
    /* name is allocated by name_of_child */
    child->name = name;
    child->index = index;
!   value = value_of_child (parent, real_index);
    child->parent = parent;
    child->root = parent->root;
    childs_name = xstrprintf ("%s.%s", parent->obj_name, name);
*************** new_variable (void)
*** 1494,1499 ****
--- 1504,1510 ----
    var->type = NULL;
    var->value = NULL;
    var->num_children = -1;
+   var->current_children = 0;
    var->parent = NULL;
    var->children = NULL;
    var->format = 0;

RE: [PATCH:MI] Return a subset of a variable object's children

[Marc Khouzam]

> From: gdb-patches-owner@sourceware.org on behalf of Nick Roberts
> Sent: Sunday, April 27, 2008 7:14 AM
> 
> This patch allows -var-list-children to create just some of a variable object's
> children.  Following a discussion last year with Mark Khouzam, this is useful
> for displaying large arrays (and structures) where it would take too long to
> create all the children: it is only really necessary to create those that are
> visible on the screen and update as further become visible.  It also allows a

Cool!
To be fully transparent though, DSF does not yet have a way to deal with different
behavior of different GDBs, so we won't be using such improvements right away.
We stick to common behaviors, until we can handle differences cleanly.
But such improvements may prove useful in the future.

> step size (stride) other than one.

I'm not sure what the stride would be used for.  Maybe something like printing
all even indexes of an array for example?
In any case, it is a pretty simple addition, and no one is forced to use it,
so I'm only asking to understand better.

> 2008-04-27  Nick Roberts  <nickrob@snap.net.nz>
> 
>       * mi/mi-cmd-var.c (mi_cmd_var_list_children): Add options to select
>       a subset of children.
> 
>       * varobj.h (varobj_list_children): Declare new parameters.
> 
>       * varobj.c (struct varobj): New member current_children.
>       (varobj_list_children): Create any new varobjs for children
>       specified by mi_cmd_var_list_children.
>       (create_child): Add parameter real_index.  Use it.
> 


I have a concern about the ordering of children.
I think not having a constant ordering for the children could prove a problem.
For example, I think the algorithm proposed will fail if a child
varObj is deleted by the user.  I believe deleting a varObj inserts NULL
in its current position, however, the algo always inserts at the end, so
it will miss the available deleted entry.

Also, the double loop may prove to be slow for large number of children.

I was thinking that we could keep order of children as they are defined
(current behaviour) but not fill them all, until requested.
We could create the full Vector of children as is done now by

  while (VEC_length (varobj_p, var->children) < var->num_children)
    VEC_safe_push (varobj_p, var->children, NULL);

but only actually create the children that have been requested by the user.
I'm not sure how much efficiency there is by allocating the memory before hand?
Also, is there no way to grow the vector by more than a single point at a time?

We can even improve on that by doing the following:
instead of allocating the vector for all children, we can allocate the vector
for the children up to start+number*stride.

I believe this will give a constant ordering (same as current) and avoid the
costly double loop, since we can simply check for NULL to know if a child
is there or not.  And the delete will work as is.

You can also probably use the vector size instead of the new current_children.


> 
> *** mi-cmd-var.c      20 Apr 2008 10:20:39 +1200      1.50
> --- mi-cmd-var.c      27 Apr 2008 21:34:58 +1200
> *************** mi_cmd_var_list_children (char *command,
[...]
> !   if (argc == 1 || argc == 2)
> !     {
> !       /* Get varobj handle, if a valid var obj name was specified */
> !       if (argc == 1)
> !     var = varobj_get_handle (argv[0]);
> !       else
> !     var = varobj_get_handle (argv[1]);
> !       if (var == NULL)
> !     error (_("Variable object not found"));
> ! 
> !       if (argc == 2)
> !     print_values = mi_parse_values_option (argv[0]);
> !       else
> !     print_values = PRINT_NO_VALUES;
> ! 
> !       start = 0;
> !       number = varobj_get_num_children (var);
> !     }
>     else
> !     {
> !       int optind = 0;
> !       char *optarg;
> !       while (1)
> !     {
> !       int opt = mi_getopt ("mi_cmd_var_list_children",
> !                            argc, argv, opts, &optind, &optarg);

It would be nice to use the string -var-list-children instead of
mi_cmd_var_list_children, since it may be used to print an error to the user.

> !       if (optind >= argc)
> !     error (_("Missing VARNAME"));

It would be nice to have the full Usage printed here.
  
> !       if (optind < argc - 1)
> !     error (_("Garbage at end of command"));
> ! 
> !       var = varobj_get_handle (argv[optind]);
 
Here, you need the var == NULL check again:
    
      if (var == NULL)
        error (_("Variable object not found"));

Or you can extract it from the if above and put it outside the if/else


Thanks for getting this started.

Marc

RE: [PATCH:MI] Return a subset of a variable object's children

[Nick Roberts]

 > > step size (stride) other than one.
 > 
 > I'm not sure what the stride would be used for.  Maybe something like
 > printing all even indexes of an array for example?  In any case, it is a
 > pretty simple addition, and no one is forced to use it, so I'm only asking
 > to understand better.

Yes, I think its just another way to sample a large array.  ISTR dbx allows
printing array slices in this way.

 > > 2008-04-27  Nick Roberts  <nickrob@snap.net.nz>
 > > 
 > >       * mi/mi-cmd-var.c (mi_cmd_var_list_children): Add options to select
 > >       a subset of children.
 > > 
 > >       * varobj.h (varobj_list_children): Declare new parameters.
 > > 
 > >       * varobj.c (struct varobj): New member current_children.
 > >       (varobj_list_children): Create any new varobjs for children
 > >       specified by mi_cmd_var_list_children.
 > >       (create_child): Add parameter real_index.  Use it.
 > > 
 > 
 > 
 > I have a concern about the ordering of children.  I think not having a
 > constant ordering for the children could prove a problem.  For example, I
 > think the algorithm proposed will fail if a child varObj is deleted by the
 > user.  I believe deleting a varObj inserts NULL in its current position,
 > however, the algo always inserts at the end, so it will miss the available
 > deleted entry.

You're probably right about ordering, and deletion does cause a segmentation
fault.

 > Also, the double loop may prove to be slow for large number of children.

I was thinking that only a small number of children would ever exist
simultaneously.  Scrolling might make that a larger number but maybe
it could be arranged to delete children that go out of view.

 > I was thinking that we could keep order of children as they are defined
 > (current behaviour) but not fill them all, until requested.
 > We could create the full Vector of children as is done now by
 > 
 >   while (VEC_length (varobj_p, var->children) < var->num_children)
 >     VEC_safe_push (varobj_p, var->children, NULL);

I guess this would remove the need for a second loop but it seems wasteful on
memory.  Previously children variable objects were stored as a linked list and,
as I have said before, I do think this is more suitable as objects can then be
inserted and removed at any point in the sequence of children.

 > but only actually create the children that have been requested by the user.
 > I'm not sure how much efficiency there is by allocating the memory before
 > hand?  Also, is there no way to grow the vector by more than a single point
 > at a time?

Like resize with STL vectors?  I'm not aware of one.

 > We can even improve on that by doing the following: instead of allocating
 > the vector for all children, we can allocate the vector for the children up
 > to start+number*stride.

The variables start, number and stride might be selectable by the user but I'm
thinking that "number" used by Gdb will be controlled by how many elements are
visible on the screen.  What happens with your approach when new elements
become visible and new children need to be created?

 > I believe this will give a constant ordering (same as current) and avoid the
 > costly double loop, since we can simply check for NULL to know if a child is
 > there or not.  And the delete will work as is.
 > 
 > You can also probably use the vector size instead of the new
 > current_children.

Thanks for the feedback.

-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: [PATCH:MI] Return a subset of a variable object's children

[Vladimir Prus]

Nick Roberts wrote:

> 
> This patch allows -var-list-children to create just some of a variable object's
> children.  Following a discussion last year with Mark Khouzam, this is useful
> for displaying large arrays (and structures) where it would take too long to
> create all the children: it is only really necessary to create those that are
> visible on the screen and update as further become visible.  It also allows a
> step size (stride) other than one.
> 
> ISTR that it was said a long time ago that mi_getopt won't take long options
> but that doesn't seem to be the case.  The difficulty is that
> -var-list-children takes 0, 1 and 2 as options so I've had to separate the
> logic to keep it backward compatible.
> 
> Use of these new options means that the elements might not be printed in order
> but it is always preserved if only the old options are used.

Thanks for working on this. Some concerns:

1. Can you please post MI patches in the unified diff format. I have troubles
groking context diff, so actual review of the code may be delayed.

2. Do we need the stride? One this feature is in, we no longer can remove it,
and if there's no pressing need we should not add it.

3. I'm not very comfortable with creating the children, and reporting them,
out-of-order. I think if I'm asking for childrem from 10 to 20, I should get
exactly the children I've asked for. If I can potentially also get children from
20 to 30, then followed by children from 10 to 20, it means the frontend will
have to do some smarts to find the new children and put them in the right place.
GDB surely know which children it created, so it can report just them.

Thanks,
Volodya

RE: [PATCH:MI] Return a subset of a variable object's children

[Vladimir Prus]

Nick Roberts wrote:

>  > > step size (stride) other than one.
>  > 
>  > I'm not sure what the stride would be used for.  Maybe something like
>  > printing all even indexes of an array for example?  In any case, it is a
>  > pretty simple addition, and no one is forced to use it, so I'm only asking
>  > to understand better.
> 
> Yes, I think its just another way to sample a large array.  ISTR dbx allows
> printing array slices in this way.

And is this behaviour particularly useful?

>  > > 2008-04-27  Nick Roberts  <nickrob@snap.net.nz>
>  > > 
>  > >       * mi/mi-cmd-var.c (mi_cmd_var_list_children): Add options to select
>  > >       a subset of children.
>  > > 
>  > >       * varobj.h (varobj_list_children): Declare new parameters.
>  > > 
>  > >       * varobj.c (struct varobj): New member current_children.
>  > >       (varobj_list_children): Create any new varobjs for children
>  > >       specified by mi_cmd_var_list_children.
>  > >       (create_child): Add parameter real_index.  Use it.
>  > > 
>  > 
>  > 
>  > I have a concern about the ordering of children.  I think not having a
>  > constant ordering for the children could prove a problem.  For example, I
>  > think the algorithm proposed will fail if a child varObj is deleted by the
>  > user.  I believe deleting a varObj inserts NULL in its current position,
>  > however, the algo always inserts at the end, so it will miss the available
>  > deleted entry.
> 
> You're probably right about ordering, and deletion does cause a segmentation
> fault.
> 
>  > Also, the double loop may prove to be slow for large number of children.
> 
> I was thinking that only a small number of children would ever exist
> simultaneously.  Scrolling might make that a larger number but maybe
> it could be arranged to delete children that go out of view.

I wonder if deleting children that are not visible is possible/desirable.
In Qt, item data is requested only when item is drawn. I think SWT's Tree can be
configured the same way. However, I don't think I saw any way, in either, to
detect than an item is no longer visible. Marc, can you tell if SWT allows that?

Even if technically possible, is this a desirable thing? I think the the primary
goal of incremental fetch is that if you happen to have std::vector with 200 children,
then display of it won't fill your entire screen with children of a single variable.
With incremental fetch, you can look at the children only if you're really interested.
On the other hand, I don't think keeping 200 varobjs in GDB is too expensive. And if
we talk about 10000 children, then well, I don't think standard variable display widget
is gonna be very good. Even if you delete varobjs that are not visible, it's too hard
to find anything interesting among 10000 elements.

 
>  > I was thinking that we could keep order of children as they are defined
>  > (current behaviour) but not fill them all, until requested.
>  > We could create the full Vector of children as is done now by
>  > 
>  >   while (VEC_length (varobj_p, var->children) < var->num_children)
>  >     VEC_safe_push (varobj_p, var->children, NULL);
> 
> I guess this would remove the need for a second loop but it seems wasteful on
> memory.  Previously children variable objects were stored as a linked list and,
> as I have said before, I do think this is more suitable as objects can then be
> inserted and removed at any point in the sequence of children.

Please feel free to implement generic list datastructure in C, or rewrite gdb in C++.
So far, using vector proved to be big convenience.

> 
>  > but only actually create the children that have been requested by the user.
>  > I'm not sure how much efficiency there is by allocating the memory before
>  > hand?  Also, is there no way to grow the vector by more than a single point
>  > at a time?
> 
> Like resize with STL vectors?  I'm not aware of one.

VEC_safe_grow

- Volodya

RE: [PATCH:MI] Return a subset of a variable object's children

[Nick Roberts]

 > >  > > step size (stride) other than one.
 > >  > 
 > >  > I'm not sure what the stride would be used for.  Maybe something like
 > >  > printing all even indexes of an array for example?  In any case, it is
 > >  > a pretty simple addition, and no one is forced to use it, so I'm only
 > >  > asking to understand better.
 > > 
 > > Yes, I think its just another way to sample a large array.  ISTR dbx allows
 > > printing array slices in this way.
 > 
 > And is this behaviour particularly useful?

It depends on your point of view.  In scientific computing (my background),
arrays are often used and sampling allows the data to be viewed at a lower
resolution for an overall picture of a value set.

 >...
 > > I was thinking that only a small number of children would ever exist
 > > simultaneously.  Scrolling might make that a larger number but maybe
 > > it could be arranged to delete children that go out of view.
 > 
 > I wonder if deleting children that are not visible is possible/desirable.
 > In Qt, item data is requested only when item is drawn. I think SWT's Tree
 > can be configured the same way. However, I don't think I saw any way, in
 > either, to detect than an item is no longer visible. Marc, can you tell if
 > SWT allows that?

In Emacs I would just find the first and last line numbers and work out
which elements were displayed from that.  

 > Even if technically possible, is this a desirable thing? 

I'm just floating ideas, I've not worked out the details.

 >                                                         I think the the
 > primary goal of incremental fetch is that if you happen to have std::vector
 > with 200 children, then display of it won't fill your entire screen with
 > children of a single variable.  With incremental fetch, you can look at the
 > children only if you're really interested.  On the other hand, I don't think
 > keeping 200 varobjs in GDB is too expensive. And if we talk about 10000
 > children, then well, I don't think standard variable display widget is gonna
 > be very good. Even if you delete varobjs that are not visible, it's too hard
 > to find anything interesting among 10000 elements.

Again in scientific computing, arrays often have many more than 10000 elements.
In image processing arrays are two dimensional 512x512 with over 250,0000
elements.  The user would have to identify the region of interest for the
display widget, e.g. [110:120][220:230] for a 10x10 square centred at
(115,225).


 > >  > I was thinking that we could keep order of children as they are defined
 > >  > (current behaviour) but not fill them all, until requested.
 > >  > We could create the full Vector of children as is done now by
 > >  > 
 > >  >   while (VEC_length (varobj_p, var->children) < var->num_children)
 > >  >     VEC_safe_push (varobj_p, var->children, NULL);
 > > 
 > > I guess this would remove the need for a second loop but it seems wasteful
 > > on memory.  Previously children variable objects were stored as a linked
 > > list and, as I have said before, I do think this is more suitable as
 > > objects can then be inserted and removed at any point in the sequence of
 > > children.
 > 
 > Please feel free to implement generic list datastructure in C, or rewrite
 > gdb in C++.  So far, using vector proved to be big convenience.

Clearly I'm not going to do either but we could simply go back to using the
linked list structures that were already in varobj.c.  It's a question of
whether the convenience outweighs the handicap of having to work with vectors
all the time or not.  IMHO it doesn't.


 > >  > but only actually create the children that have been requested by the
 > >  > user.  I'm not sure how much efficiency there is by allocating the
 > >  > memory before hand?  Also, is there no way to grow the vector by more
 > >  > than a single point at a time?
 > > 
 > > Like resize with STL vectors?  I'm not aware of one.
 > 
 > VEC_safe_grow

OK, I didn't know about that.  Why not use it instead of VEC_safe_push in the
construct above?

-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: [PATCH:MI] Return a subset of a variable object's children

[Vladimir Prus]

On Wednesday 30 April 2008 11:01:52 Nick Roberts wrote:
>  > >  > > step size (stride) other than one.
>  > >  > 
>  > >  > I'm not sure what the stride would be used for.  Maybe something like
>  > >  > printing all even indexes of an array for example?  In any case, it is
>  > >  > a pretty simple addition, and no one is forced to use it, so I'm only
>  > >  > asking to understand better.
>  > > 
>  > > Yes, I think its just another way to sample a large array.  ISTR dbx allows
>  > > printing array slices in this way.
>  > 
>  > And is this behaviour particularly useful?
> 
> It depends on your point of view.  In scientific computing (my background),
> arrays are often used and sampling allows the data to be viewed at a lower
> resolution for an overall picture of a value set.

...

>  > primary goal of incremental fetch is that if you happen to have std::vector
>  > with 200 children, then display of it won't fill your entire screen with
>  > children of a single variable.  With incremental fetch, you can look at the
>  > children only if you're really interested.  On the other hand, I don't think
>  > keeping 200 varobjs in GDB is too expensive. And if we talk about 10000
>  > children, then well, I don't think standard variable display widget is gonna
>  > be very good. Even if you delete varobjs that are not visible, it's too hard
>  > to find anything interesting among 10000 elements.
> 
> Again in scientific computing, arrays often have many more than 10000 elements.
> In image processing arrays are two dimensional 512x512 with over 250,0000
> elements.  The user would have to identify the region of interest for the
> display widget, e.g. [110:120][220:230] for a 10x10 square centred at
> (115,225).

It seems to me that specialized widgets are more suitable for this purpose, like
image viewer, or a charting component. Especially with image data, using varobjs
is probably not going to work. Creating varobjs per each item will be just too slow --
we need some high-bandwidth interaction way, like the memory-reading commands.
(And maybe those should have stride options).

>  > >  > I was thinking that we could keep order of children as they are defined
>  > >  > (current behaviour) but not fill them all, until requested.
>  > >  > We could create the full Vector of children as is done now by
>  > >  > 
>  > >  >   while (VEC_length (varobj_p, var->children) < var->num_children)
>  > >  >     VEC_safe_push (varobj_p, var->children, NULL);
>  > > 
>  > > I guess this would remove the need for a second loop but it seems wasteful
>  > > on memory.  Previously children variable objects were stored as a linked
>  > > list and, as I have said before, I do think this is more suitable as
>  > > objects can then be inserted and removed at any point in the sequence of
>  > > children.
>  > 
>  > Please feel free to implement generic list datastructure in C, or rewrite
>  > gdb in C++.  So far, using vector proved to be big convenience.
> 
> Clearly I'm not going to do either but we could simply go back to using the
> linked list structures that were already in varobj.c.  It's a question of 
> whether the convenience outweighs the handicap of having to work with vectors
> all the time or not.  IMHO it doesn't.

I disagree, and I haven't yet seen a practical "handicap". We're not going back
to hand-written data structures for MI implementation.

>  > >  > but only actually create the children that have been requested by the
>  > >  > user.  I'm not sure how much efficiency there is by allocating the
>  > >  > memory before hand?  Also, is there no way to grow the vector by more
>  > >  > than a single point at a time?
>  > > 
>  > > Like resize with STL vectors?  I'm not aware of one.
>  > 
>  > VEC_safe_grow
> 
> OK, I didn't know about that.  Why not use it instead of VEC_safe_push in the
> construct above?

Well, it happens not to initialize the data, so some changes in further logic will
be required. Until now, it was not a performance issue.

- Volodya

 

Re: [PATCH:MI] Return a subset of a variable object's children

[André Pönitz]

On Wednesday 30 April 2008 08:24:30 Vladimir Prus wrote:
> Nick Roberts wrote:
> > [...]
> > I was thinking that only a small number of children would ever exist
> > simultaneously.  Scrolling might make that a larger number but maybe
> > it could be arranged to delete children that go out of view.
> 
> I wonder if deleting children that are not visible is possible/desirable.

Well, I would still prefer a simple toggle that would allow me to switch off
any automatic creation of children and one-shot 'expression evaluation'
and one-shot 'children listing'.

I would expect this to be much simpler to implement on the gdb side and
gives all the flexibility needed (as far as I am concerned) on the
frontend side.

Complex containers may do all kind of funny stuff behind the debugger's
back (like reallocation, rebalancing, renumbering, moving stuff to and from
secondary memory) which a frontend might want to handle transparently.
I would not expect gdb to even be aware of that (let alone to handle(!) it).

Also, hard-wiring behaviour to certain types calls for trouble on the
frontend side. E.g. if there's a std::string member somewhere I might want
to see it as something "human readable" (-> 1 child), or a byte array
(->lots of children), or maybe a structured view if it is XML (-> some children).
So whatever behaviour is hard-wired will be wrong for  n - 1 use cases.

Regards,
Andre'

Re: [PATCH:MI] Return a subset of a variable object's children

[Vladimir Prus]

André Pönitz wrote:

> On Wednesday 30 April 2008 08:24:30 Vladimir Prus wrote:
>> Nick Roberts wrote:
>> > [...]
>> > I was thinking that only a small number of children would ever exist
>> > simultaneously.  Scrolling might make that a larger number but maybe
>> > it could be arranged to delete children that go out of view.
>> 
>> I wonder if deleting children that are not visible is possible/desirable.
> 
> Well, I would still prefer a simple toggle that would allow me to switch off
> any automatic creation of children

There's no automatic creation. Until you do -list-children, no child is created.

> and one-shot 'expression evaluation' and one-shot 'children listing'.

What is 'expression evaluation'. As for one-shot 'children listing' -- the
idea is that you can pass the number of children to create for -list-children,
and GDB won't do anything about children beyond this range.

> I would expect this to be much simpler to implement on the gdb side and
> gives all the flexibility needed (as far as I am concerned) on the
> frontend side.
> 
> Complex containers may do all kind of funny stuff behind the debugger's
> back (like reallocation, rebalancing, renumbering, moving stuff to and from
> secondary memory) which a frontend might want to handle transparently.
> I would not expect gdb to even be aware of that (let alone to handle(!) it).
> 
> Also, hard-wiring behaviour to certain types calls for trouble on the
> frontend side. E.g. if there's a std::string member somewhere I might want
> to see it as something "human readable" (-> 1 child), or a byte array
> (->lots of children), or maybe a structured view if it is XML (-> some children).
> So whatever behaviour is hard-wired will be wrong for  n - 1 use cases.

I'm afraid I don't get your point. If you want funny representations of
any type you can either:

1. Use Python visualizers (that can be switched on the fly)
2. Just get the raw data and show it as you see fit.

- Volodya


> 
> Regards,
> Andre'

Re: [PATCH:MI] Return a subset of a variable object's children

[Nick Roberts]

 > > > I was thinking that only a small number of children would ever exist
 > > > simultaneously.  Scrolling might make that a larger number but maybe
 > > > it could be arranged to delete children that go out of view.
 > > 
 > > I wonder if deleting children that are not visible is possible/desirable.
 > 
 > Well, I would still prefer a simple toggle that would allow me to switch off
 > any automatic creation of children and one-shot 'expression evaluation'
 > and one-shot 'children listing'.

Are you arguing against the general concept of variable objects?

 > I would expect this to be much simpler to implement on the gdb side and
 > gives all the flexibility needed (as far as I am concerned) on the
 > frontend side.

Presumably your one-shot 'children listing' requires all values to be printed
each time.  Variable objects work by just printing the values which have
changed.

 > Complex containers may do all kind of funny stuff behind the debugger's
 > back (like reallocation, rebalancing, renumbering, moving stuff to and from
 > secondary memory) which a frontend might want to handle transparently.
 > I would not expect gdb to even be aware of that (let alone to handle(!) it).
 >
 > Also, hard-wiring behaviour to certain types calls for trouble on the
 > frontend side. E.g...

Who's talking about complex containers or hardwiring?  The vectors and linked
lists discussed in this thread are internal to Gdb, not those of the program
being debugged.


-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: [PATCH:MI] Return a subset of a variable object's children

[André Pönitz]

On Wednesday 30 April 2008 11:19:59 Vladimir Prus wrote:
> André Pönitz wrote:
> >> I wonder if deleting children that are not visible is possible/desirable.
> > 
> > Well, I would still prefer a simple toggle that would allow me to switch off
> > any automatic creation of children
> 
> There's no automatic creation. Until you do -list-children, no child is created.

That's what I meant. Right now I can't get a list of children without
varobjects being created.

> > and one-shot 'expression evaluation' and one-shot 'children listing'.
> 
> What is 'expression evaluation'.

A shortcut for

  -var-create foo * exp
  -var-evaluate-expression foo
  -var-delete foo

optionally including a

  -var-list-children foo

Maybe its already there and I just did not see it...

> [...]
> I'm afraid I don't get your point. If you want funny representations of
> any type you can either:
> 
> 1. Use Python visualizers (that can be switched on the fly)

Right. I guess that's probably the route to go for me in the long run.
I am a bit scared by the additional dependency, though, so I wouldn't
mind to have a reasonably well working 'pure MI' solution as fallback,
too.

> 2. Just get the raw data and show it as you see fit.

This does not work too well with structures containing pointers
or such. It's fine for 'position independent' data like bitmaps or
so, though.

Andre'

Re: [PATCH:MI] Return a subset of a variable object's children

[Vladimir Prus]

André Pönitz wrote:

> On Wednesday 30 April 2008 11:19:59 Vladimir Prus wrote:
>> André Pönitz wrote:
>> >> I wonder if deleting children that are not visible is possible/desirable.
>> > 
>> > Well, I would still prefer a simple toggle that would allow me to switch off
>> > any automatic creation of children
>> 
>> There's no automatic creation. Until you do -list-children, no child is created.
> 
> That's what I meant. Right now I can't get a list of children without
> varobjects being created.

Yes, why is that a bad thing?

>> > and one-shot 'expression evaluation' and one-shot 'children listing'.
>> 
>> What is 'expression evaluation'.
> 
> A shortcut for
> 
>   -var-create foo * exp
>   -var-evaluate-expression foo
>   -var-delete foo
> 
> optionally including a
> 
>   -var-list-children foo
> 
> Maybe its already there and I just did not see it...

Well, there's data-evaluate-expression but it throws raw data at you, there's
no "children", just string.

I seems that doing the 3 or 4 commands above is just fine. Why do you want
to delete varobj immediately and what problem does sending 4 commands cause?

>> I'm afraid I don't get your point. If you want funny representations of
>> any type you can either:
>> 
>> 1. Use Python visualizers (that can be switched on the fly)
> 
> Right. I guess that's probably the route to go for me in the long run.
> I am a bit scared by the additional dependency, though, so I wouldn't
> mind to have a reasonably well working 'pure MI' solution as fallback,
> too.

Probably won't happen. 

>> 2. Just get the raw data and show it as you see fit.
> 
> This does not work too well with structures containing pointers
> or such. It's fine for 'position independent' data like bitmaps or
> so, though.

If a structure contains a pointer, and you want some custom visualization,
you can do something with the pointer value. I don't really understand
what you're trying to do :-(

- Volodya

Re: [PATCH:MI] Return a subset of a variable object's children

[André Pönitz]

[Sorry Nick for the private reply... I am still fighting the Mail client ;-}]

On Wednesday 30 April 2008 11:25:05 you wrote:
>  > > > I was thinking that only a small number of children would ever exist
>  > > > simultaneously.  Scrolling might make that a larger number but maybe
>  > > > it could be arranged to delete children that go out of view.
>  > > 
>  > > I wonder if deleting children that are not visible is possible/desirable.
>  > 
>  > Well, I would still prefer a simple toggle that would allow me to switch off
>  > any automatic creation of children and one-shot 'expression evaluation'
>  > >  > and one-shot 'children listing'. 
> 
> Are you arguing against the general concept of variable objects?

Erm... yes. Could be seen as such.

Variable objects are really nice and useful for displaying C-like structure if you
want a 1:1 display in the frontend when all (or at least most of) the state you
need can be kept in the debugger.

However, anything beyond that gets ugly and as soon as one arrives at 
the point where one basically duplicates a lot of the internal backend 
state in the frontend one develops the desire not to need to care for
the backend state at all...

>  > I would expect this to be much simpler to implement on the gdb side and
>  > gives all the flexibility needed (as far as I am concerned) on the
>  > frontend side.
> 
> Presumably your one-shot 'children listing' requires all values to be printed
> each time.  Variable objects work by just printing the values which have
> changed.
 
I am fully aware of that. It has a very limited utility, though...

I basically have to fill, say, 20 lines in the typical 'Locals & Watchers'
view.

With a one-shot approach I can fire off 20 simple request, getting back
20 simple results (say, a total of 1000 bytes of data) compare them with
the previous results myself, re-populate the view, mark changes etc.
No big deal as it happens at most once per user interaction. Easy and 
predictable. 

With the 'update notifications' I get an unpredictable amount of data
(there could be an array with a few thousand children after all, all
being changed, stalling my communication line for an unpredicatable
amount of time), _and_ the data I get is insufficient as it, for instance, 
does not tell me that a std::string has changed somewhere in the middle.
So I need to do a few 'one-shot evaluations' anyway to get the desired
results. All in all, less satisfactory.

Andre'

 

> 

RE: [PATCH:MI] Return a subset of a variable object's children

[Marc Khouzam]

> > Also, the double loop may prove to be slow for large number of children.
> 
> I was thinking that only a small number of children would ever exist
> simultaneously.  Scrolling might make that a larger number but maybe
> it could be arranged to delete children that go out of view.

You are right, we do that in DSF-GDB.  However, such a double loop can become
prohibitive relatively quickly, at numbers lower than when children start
being deleted.  For example, DSF-GDB allows for 1000 varObjs simultaneously,
before doing any deletion.  This may not prove too slow, but it creates
a requirement on the frontend to do proper management of varObjects, to
avoid any issues.  If we can avoid the double loop (and its string
comparisons), we would benefit, no?

> > I was thinking that we could keep order of children as they are defined
> > (current behaviour) but not fill them all, until requested.
> > We could create the full Vector of children as is done now by
> > 
> >   while (VEC_length (varobj_p, var->children) < var->num_children)
> >     VEC_safe_push (varobj_p, var->children, NULL);
> 
> I guess this would remove the need for a second loop but it seems wasteful on
> memory.  Previously children variable objects were stored as a linked list and,
> as I have said before, I do think this is more suitable as objects can then be
> inserted and removed at any point in the sequence of children.

Yes, linked list seem more suited for this usecase.  But I gather from Volodya's 
emails that vectors have benefits that we should not ignored, so let's continue 
the discussion with vectors.
To me, the advantage of your patch is far larger in the fact that we no longer
need to create all children varObj at once; my impression is that the memory allocation
is a small optimization in caparison.  Is that a correct impression?
Besides, currently, the memory is immediately allocated, so things wouldn't be 
any worse :-)

> > We can even improve on that by doing the following: instead of allocating
> > the vector for all children, we can allocate the vector for the children up
> > to start+number*stride.
> 
> The variables start, number and stride might be selectable by the user but I'm
> thinking that "number" used by Gdb will be controlled by how many elements are
> visible on the screen.  What happens with your approach when new elements
> become visible and new children need to be created?

Here is a simplified example.
Say you have an array of a 1000 integers and we can show 10 elements on the screen.
The user 'opens' the array, revealing the first 10 elements.  The user then jumps-scrolls
to position 500 revealing postions 500 to 509.  I imagine a set of commands to be
something like this:

-var-list-children -f 0 -n 10 var1
-var-list-children -f 500 -n 10 var1

The first call to var-list-children would allocate a vector of size (number+start = 0+10).
And create those 10 children varObjs.
The second call would expand that vector to a size of (number+start = 500+10).
And create those 10 children varObjs.
Positions 10 to 499 would remain NULL since we haven't create the children yet (and we may
never create them, if the user does not scroll there.)

Is that what you were asking?

Marc

RE: [PATCH:MI] Return a subset of a variable object's children

[Marc Khouzam]

> > > I was thinking that only a small number of children would ever exist
> > > simultaneously.  Scrolling might make that a larger number but maybe
> > > it could be arranged to delete children that go out of view.
> > 
> > I wonder if deleting children that are not visible is possible/desirable.
> > In Qt, item data is requested only when item is drawn. I think SWT's Tree
> > can be configured the same way. However, I don't think I saw any way, in
> > either, to detect than an item is no longer visible. Marc, can you tell if
> > SWT allows that?
> 
> In Emacs I would just find the first and last line numbers and work out
> which elements were displayed from that.  

I'm not sure what SWT allows...
In DSF-GDB, we have a LeastRecentlyUsed queue which allows us to know which
varObj is the oldest (which implies it is not visible), and we can deleted
once we have reached our limit of 1000 varObjs.

I still think that GDB should avoid having the requirement of 'deleting
old varObjs' toward the frontend.  It would be nicer if GDB would keep
its efficiency, not matter what the frontend did in this case.

RE: [PATCH:MI] Return a subset of a variable object's children

[Marc Khouzam]

> > > > > but only actually create the children that have been requested by the
> > > > > user.  I'm not sure how much efficiency there is by allocating the
> > > > > memory before hand?  Also, is there no way to grow the vector by more
> > > > > than a single point at a time?
> > > > 
> > > > Like resize with STL vectors?  I'm not aware of one.
> > > 
> > > VEC_safe_grow
> > 
> > OK, I didn't know about that.  Why not use it instead of VEC_safe_push in the
> > construct above?
> 
> Well, it happens not to initialize the data, so some changes in further logic will
> be required. Until now, it was not a performance issue.

My concern about

  while (VEC_length (varobj_p, var->children) < var->num_children)
    VEC_safe_push (varobj_p, var->children, NULL);

is that the vector may be reallocated multiple times.
How about something like this (assuming quick_push does what I think it does)

  var->children = VEC_alloc(varobj_p, var->num_children);
  for (i=0; i<var->num_children; i++)
    VEC_quick_push (varobj_p, var->children, NULL);

Same loop, but it avoids having to reallocate the vector.

Just a thought, not related to Nick's patch.

Marc

Re: [PATCH:MI] Return a subset of a variable object's children

[Nick Roberts]

 > > Are you arguing against the general concept of variable objects?
 > 
 > Erm... yes. Could be seen as such.

With respect, I'm not sure that you know what you are talking about.  The
concept of variable objects predates MI and was take from Insight.  It hasn't
been developed overnight but over a long time by experienced Gdb developers,
i.e, not the likes of me.

 > > ...
 > > Presumably your one-shot 'children listing' requires all values to be
 > > printed each time.  Variable objects work by just printing the values
 > > which have changed.
 >  
 > I am fully aware of that. It has a very limited utility, though...
 > 
 > I basically have to fill, say, 20 lines in the typical 'Locals & Watchers'
 > view.
 > 
 > With a one-shot approach I can fire off 20 simple request, getting back
 > 20 simple results (say, a total of 1000 bytes of data) compare them with
 > the previous results myself, re-populate the view, mark changes etc.
 > No big deal as it happens at most once per user interaction. Easy and 
 > predictable. 

How do you know if an expression is still in scope?  How do you know that
an original variable is not currently shadowed by another with the same name?
How do you know how many elements are in an array, members in a structure?
These are some of the questions that variable objects address.

 > With the 'update notifications' I get an unpredictable amount of data
 > (there could be an array with a few thousand children after all, all
 > being changed, stalling my communication line for an unpredicatable
 > amount of time),

This is what we're currently discussing: ways to ensure that thousands of
variable objects aren't created when viewing arrays with thousands of elements.

 >                 _and_ the data I get is insufficient as it, for instance, 
 > does not tell me that a std::string has changed somewhere in the middle.
 > So I need to do a few 'one-shot evaluations' anyway to get the desired
 > results. All in all, less satisfactory.

And maybe the Python interface can address this.

-- 
Nick                                           http://www.inet.net.nz/~nickrob

RE: [PATCH:MI] Return a subset of a variable object's children

[Nick Roberts]

 > > > Also, the double loop may prove to be slow for large number of children.
 > > 
 > > I was thinking that only a small number of children would ever exist
 > > simultaneously.  Scrolling might make that a larger number but maybe
 > > it could be arranged to delete children that go out of view.
 > 
 > You are right, we do that in DSF-GDB.  However, such a double loop can become
 > prohibitive relatively quickly, at numbers lower than when children start
 > being deleted.  For example, DSF-GDB allows for 1000 varObjs simultaneously,
 > before doing any deletion.  This may not prove too slow, but it creates
 > a requirement on the frontend to do proper management of varObjects, to
 > avoid any issues.  If we can avoid the double loop (and its string
 > comparisons), we would benefit, no?

The double loop is O(N*N).  I guess it could be reduced to O(N*log(N))
using some kind of sorting algorithm.

 > > > I was thinking that we could keep order of children as they are defined
 > > > (current behaviour) but not fill them all, until requested.
 > > > We could create the full Vector of children as is done now by
 > > > 
 > > >   while (VEC_length (varobj_p, var->children) < var->num_children)
 > > >     VEC_safe_push (varobj_p, var->children, NULL);
 > > 
 > > I guess this would remove the need for a second loop but it seems wasteful
 > > on memory.  Previously children variable objects were stored as a linked
 > > list and, as I have said before, I do think this is more suitable as
 > > objects can then be inserted and removed at any point in the sequence of
 > > children.
 > 
 > Yes, linked list seem more suited for this usecase.  But I gather from
 > Volodya's emails that vectors have benefits that we should not ignored, so
 > let's continue the discussion with vectors.  To me, the advantage of your
 > patch is far larger in the fact that we no longer need to create all
 > children varObj at once; my impression is that the memory allocation is a
 > small optimization in caparison.  Is that a correct impression?  Besides,
 > currently, the memory is immediately allocated, so things wouldn't be any
 > worse :-)

It's not just a question of memory allocation.  It's computationally
inefficient because Gdb has to traverse a large sparsely populated vector every
time it operates on it.  It wouldn't be any _worse_ than current Gdb, but I was
trying to make it significantly _better_.

 > > > We can even improve on that by doing the following: instead of
 > > > allocating the vector for all children, we can allocate the vector for
 > > > the children up to start+number*stride.
 > > 
 > > The variables start, number and stride might be selectable by the user but
 > > I'm thinking that "number" used by Gdb will be controlled by how many
 > > elements are visible on the screen.  What happens with your approach when
 > > new elements become visible and new children need to be created?
 > 
 > Here is a simplified example.  Say you have an array of a 1000 integers and
 > we can show 10 elements on the screen.  The user 'opens' the array,
 > revealing the first 10 elements.  The user then jumps-scrolls to position
 > 500 revealing postions 500 to 509.  I imagine a set of commands to be
 > something like this:
 > 
 > -var-list-children -f 0 -n 10 var1
 > -var-list-children -f 500 -n 10 var1
 > 
 > The first call to var-list-children would allocate a vector of size (number+start = 0+10).
 > And create those 10 children varObjs.
 > The second call would expand that vector to a size of (number+start = 500+10).
 > And create those 10 children varObjs.
 > Positions 10 to 499 would remain NULL since we haven't create the children yet (and we may
 > never create them, if the user does not scroll there.)
 > 
 > Is that what you were asking?

It looks like it's only a big saving if the user never moves far from the
start of the array.

-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: [PATCH:MI] Return a subset of a variable object's children

[Nick Roberts]

 > > Again in scientific computing, arrays often have many more than 10000
 > > elements.  In image processing arrays are two dimensional 512x512 with
 > > over 250,0000 elements.  The user would have to identify the region of
 > > interest for the display widget, e.g. [110:120][220:230] for a 10x10
 > > square centred at (115,225).
 > 
 > It seems to me that specialized widgets are more suitable for this purpose,
 > like image viewer, or a charting component. Especially with image data,
 > using varobjs is probably not going to work. Creating varobjs per each item
 > will be just too slow -- we need some high-bandwidth interaction way, like
 > the memory-reading commands.  (And maybe those should have stride options).

Other debuggers, e.g., Totalview have this capability and it's not slow.  In
England, I used it on a professional basis and found it useful.

 >...
 > >  > Please feel free to implement generic list datastructure in C, or
 > >  > rewrite gdb in C++.  So far, using vector proved to be big convenience.
 > > 
 > > Clearly I'm not going to do either but we could simply go back to using
 > > the linked list structures that were already in varobj.c.  It's a question
 > > of whether the convenience outweighs the handicap of having to work with
 > > vectors all the time or not.  IMHO it doesn't.
 > 
 > I disagree, and I haven't yet seen a practical "handicap". We're not going
 > back to hand-written data structures for MI implementation.

Well it's a shame that convenience should dictate the type of structures that
are used because I can get this patch to work with an old Gdb using linked
lists.

-- 
Nick                                           http://www.inet.net.nz/~nickrob

RE: [PATCH:MI] Return a subset of a variable object's children

[Marc Khouzam]

> The double loop is O(N*N).  I guess it could be reduced to O(N*log(N))
> using some kind of sorting algorithm.
>
> > To me, the advantage of your
> > patch is far larger in the fact that we no longer need to create all
> > children varObj at once; my impression is that the memory allocation is a
> > small optimization in caparison.  Is that a correct impression?  Besides,
> > currently, the memory is immediately allocated, so things wouldn't be any
> > worse :-)
>
>It's not just a question of memory allocation.  It's computationally
>inefficient because Gdb has to traverse a large sparsely populated vector every
>time it operates on it.  It wouldn't be any _worse_ than current Gdb, but I was
>trying to make it significantly _better_.

You are right.  I was focusing on making the proposed algorithm more efficient
but I didn't think about the other uses of the vector of children.

So, as you mention above, keeping the vector ordered (without empty spaces),
can reduce to O(N*log(N)).  If fact, I think it would be reduced to O(log(N))
since we would only need to find the location of the first new child; the position
of all other new children created in the same var-list-children can be determined
from that first position.
The expense would be in the insertion though, as the vector would have to be
mem-copied N times, one for each child that is not at the end of the vector.
I wonder if that is acceptable...

Marc

Re: [PATCH:MI] Return a subset of a variable object's children

[Vladimir Prus]

On Wednesday 30 April 2008 18:12:10 Marc Khouzam wrote:
> > > > I was thinking that only a small number of children would ever exist
> > > > simultaneously.  Scrolling might make that a larger number but maybe
> > > > it could be arranged to delete children that go out of view.
> > > 
> > > I wonder if deleting children that are not visible is possible/desirable.
> > > In Qt, item data is requested only when item is drawn. I think SWT's Tree
> > > can be configured the same way. However, I don't think I saw any way, in
> > > either, to detect than an item is no longer visible. Marc, can you tell if
> > > SWT allows that?
> > 
> > In Emacs I would just find the first and last line numbers and work out
> > which elements were displayed from that.  
> 
> I'm not sure what SWT allows...
> In DSF-GDB, we have a LeastRecentlyUsed queue which allows us to know which
> varObj is the oldest (which implies it is not visible), and we can deleted
> once we have reached our limit of 1000 varObjs.

What is the practical measured advantage of that?

> I still think that GDB should avoid having the requirement of 'deleting
> old varObjs' toward the frontend.  

There's no concept of 'old varobjs' in MI protocol. You should delete variable
objects as soon as whatever GUI thing you have no longer need to display the
date the varobj corresponds to.

> It would be nicer if GDB would keep 
> its efficiency, not matter what the frontend did in this case.

Let's clarify what varobjs are for, in my opinion. They are primarily for 
implementing the 'variables' widget -- that shows local variables of a
function, expressions that user wishes to be always visible, and some auxillary
values. There are natural scalability bounds in the UI -- if there are no
arrays involved, and you try to show 1000 items (backed by 1000 variable objects),
the UI is already useless. GDB is not the problem here.

Speaking about arrays, I also don't understand which use cases are been considered.
I can think about those:

1. "Oops". User, without much though, opens an item in variable tree. The
item happens to be array of 10000 elements. We don't want UI to block until those
elements are fetched -- we want to show a few (say 10) and allow the user to get
some more in case he really wanted to see something.

2. "Subset". Probably, there's some local subset of array elements that the user
wants to look at. I think in that case, we should have a way to create only children
in the range that user is interested in. And I think Nick's patch is just fine.

3. "Data analysis". User actually wants to make some sense of all 10000 element.
Maybe, well, draw a chart. Or just look at the data. The data is integer, or some
other atomic type. Then, either some specialized widget, or merely the memory view,
is quite adequate.

What I don't see as a valid use case is one where:

1. The size of array is too large to keep all varobj always created.
2. User would like to just scroll though all the array items, without
clear intentions.

Am I missing something, and this use case is actually valid?

- Volodya

Re: [PATCH:MI] Return a subset of a variable object's children

[Vladimir Prus]

On Wednesday 30 April 2008 18:21:03 Marc Khouzam wrote:
> > > > > > but only actually create the children that have been requested by the
> > > > > > user.  I'm not sure how much efficiency there is by allocating the
> > > > > > memory before hand?  Also, is there no way to grow the vector by more
> > > > > > than a single point at a time?
> > > > > 
> > > > > Like resize with STL vectors?  I'm not aware of one.
> > > > 
> > > > VEC_safe_grow
> > > 
> > > OK, I didn't know about that.  Why not use it instead of VEC_safe_push in the
> > > construct above?
> > 
> > Well, it happens not to initialize the data, so some changes in further logic will
> > be required. Until now, it was not a performance issue.
> 
> My concern about
> 
>   while (VEC_length (varobj_p, var->children) < var->num_children)
>     VEC_safe_push (varobj_p, var->children, NULL);
> 
> is that the vector may be reallocated multiple times.

As soon as we're into big-O here, we still get amortized linear time.

> How about something like this (assuming quick_push does what I think it does)
> 
>   var->children = VEC_alloc(varobj_p, var->num_children);
>   for (i=0; i<var->num_children; i++)
>     VEC_quick_push (varobj_p, var->children, NULL);
> 
> Same loop, but it avoids having to reallocate the vector.

Yes, it's a bit more efficient.

- Volodya

RE: [PATCH:MI] Return a subset of a variable object's children

[Marc Khouzam]

> > How about something like this (assuming quick_push does what I think it does)
> > 
> >   var->children = VEC_alloc(varobj_p, var->num_children);
> >   for (i=0; i<var->num_children; i++)
> >     VEC_quick_push (varobj_p, var->children, NULL);
> > 
> > Same loop, but it avoids having to reallocate the vector.
> 
> Yes, it's a bit more efficient.

In fact, I actually just read the vec.h header comment and that is pretty
much what is recommended.  In fact, the size value passed to VEC_alloc
should probably be negative here, to have even more efficiency, since
we know the vector won't grow.

But that is no longer the case with Nick's patch, as the vector will keep
on growing.

RE: [PATCH:MI] Return a subset of a variable object's children

[Marc Khouzam]

> > In DSF-GDB, we have a LeastRecentlyUsed queue which allows us to know which
> > varObj is the oldest (which implies it is not visible), and we can deleted
> > once we have reached our limit of 1000 varObjs.
> 
> What is the practical measured advantage of that?

I never took the time to measure the impact of this limit at different levels.
The idea is that we didn't know how many varObj GDB could handle without
loosing performance, so we chose 1000.

> > I still think that GDB should avoid having the requirement of 'deleting
> > old varObjs' toward the frontend.  
> 
> There's no concept of 'old varobjs' in MI protocol. You should delete variable
> objects as soon as whatever GUI thing you have no longer need to display the
> date the varobj corresponds to.

DSF-GDB tries to be as efficient as possible by avoiding request that go
to the backend.  Therefore, although a varObj is no longer being showed, we want
to keep it around in case it will be showed again.  Effectively, we are caching.
And our cache size is 1000 varObj.

I would appreciate your expertise on this.

> Let's clarify what varobjs are for, in my opinion. They are primarily for 
> implementing the 'variables' widget -- that shows local variables of a
> function, expressions that user wishes to be always visible, and some auxillary
> values. 

Yes, that is exactly when we use them: variables view, expressions view.

> There are natural scalability bounds in the UI -- if there are no
> arrays involved, and you try to show 1000 items (backed by 1000 variable objects),
> the UI is already useless. GDB is not the problem here.

Right, but for us the 1000 varObj are not meant to all be displayed at once
but are a caching system.

> Speaking about arrays, I also don't understand which use cases are been considered.
> I can think about those:
> 
> 1. "Oops". User, without much though, opens an item in variable tree. The
> item happens to be array of 10000 elements. We don't want UI to block until those
> elements are fetched -- we want to show a few (say 10) and allow the user to get
> some more in case he really wanted to see something.
> 
> 2. "Subset". Probably, there's some local subset of array elements that the user
> wants to look at. I think in that case, we should have a way to create only children
> in the range that user is interested in. And I think Nick's patch is just fine.

Now that I think about it, isn't there a mechanism to -var-create a slice of an array?
-var-create - * myArray@10 
The frontend would then manage multiple varObjs representing the slices of an array.
The CDT does that.
DSF does not do that because Pawel came up with a simpler idea (and because I wasn't
aware of the slice thing at the time; no doc on that slice thing, right?):
for arrays, we do not use -var-list-children, but instead, we -var-create each child
ourselves, when needed.

So, in our case, we have managed to work around large arrays.
However, I saw Nick's proposal as valuable because it applies to all structs and not
just arrays.  Also, it avoids the frontend having to have special logic for arrays.

> 3. "Data analysis". User actually wants to make some sense of all 10000 element.
> Maybe, well, draw a chart. Or just look at the data. The data is integer, or some
> other atomic type. Then, either some specialized widget, or merely the memory view,
> is quite adequate.
> 
> What I don't see as a valid use case is one where:
> 
> 1. The size of array is too large to keep all varobj always created.

I'm not sure what you mean here.  Before Nick's patch, issuing -var-list-children
on a large array would create a lot of varObjs.  In our case, it would have
filled up our varObj cache, although we would only be showing very few of the
array elements.

> 2. User would like to just scroll though all the array items, without
> clear intentions.

This is probably not a valid use case... although user behavior, even when dumb :-)
should not break the frontend.

But you got me thinking now... Maybe var-create using with an array slice is all we
need for dealing with a large array?

Marc

Re: [PATCH:MI] Return a subset of a variable object's children

[Vladimir Prus]

On Thursday 01 May 2008 22:14:12 Marc Khouzam wrote:
> 
> > > In DSF-GDB, we have a LeastRecentlyUsed queue which allows us to know which
> > > varObj is the oldest (which implies it is not visible), and we can deleted
> > > once we have reached our limit of 1000 varObjs.
> > 
> > What is the practical measured advantage of that?
> 
> I never took the time to measure the impact of this limit at different levels.
> The idea is that we didn't know how many varObj GDB could handle without
> loosing performance, so we chose 1000.
> 
> > > I still think that GDB should avoid having the requirement of 'deleting
> > > old varObjs' toward the frontend.  
> > 
> > There's no concept of 'old varobjs' in MI protocol. You should delete variable
> > objects as soon as whatever GUI thing you have no longer need to display the
> > date the varobj corresponds to.
> 
> DSF-GDB tries to be as efficient as possible by avoiding request that go
> to the backend.  Therefore, although a varObj is no longer being showed, we want
> to keep it around in case it will be showed again.  Effectively, we are caching.
> And our cache size is 1000 varObj.

What does it mean "no longer being showed". If you have a variables tree,
and some variable is in the tree, but is not visible on the screen right now,
there's no need whatsoever to delete the variable object for that variable.
If that variable is huge structure, then updating it (inside GDB) on each
step, can take some time, especially on a slow remote target. But, I think
you send -var-update once per each visible variable, so this is not an issue
for you.

If a variable is deleted from variables tree (because you left the scope
where that variable is defined), then you should delete variable object.
There's no reliable way to detect that exactly same variable is in scope
again and must be shown, so no reuse is possible.

Honestly, except for the fact that -var-update reads target memory, I'm sure
that GDB can create more variable objects that SWT can create TreeItems :-)

> I would appreciate your expertise on this.
> 
> > Let's clarify what varobjs are for, in my opinion. They are primarily for 
> > implementing the 'variables' widget -- that shows local variables of a
> > function, expressions that user wishes to be always visible, and some auxillary
> > values. 
> 
> Yes, that is exactly when we use them: variables view, expressions view.
> 
> > There are natural scalability bounds in the UI -- if there are no
> > arrays involved, and you try to show 1000 items (backed by 1000 variable objects),
> > the UI is already useless. GDB is not the problem here.
> 
> Right, but for us the 1000 varObj are not meant to all be displayed at once
> but are a caching system.

I'm still not sure what you mean -- you mean that a variable object is not shown
in any widget, but only is stored in the cache? What is the key of the cache?

> > Speaking about arrays, I also don't understand which use cases are been considered.
> > I can think about those:
> > 
> > 1. "Oops". User, without much though, opens an item in variable tree. The
> > item happens to be array of 10000 elements. We don't want UI to block until those
> > elements are fetched -- we want to show a few (say 10) and allow the user to get
> > some more in case he really wanted to see something.
> > 
> > 2. "Subset". Probably, there's some local subset of array elements that the user
> > wants to look at. I think in that case, we should have a way to create only children
> > in the range that user is interested in. And I think Nick's patch is just fine.
> 
> Now that I think about it, isn't there a mechanism to -var-create a slice of an array?
> -var-create - * myArray@10 

I don't know :-) I think that myArray@10 is gdb CLI trick; I never though it could
be used in MI, but thinking of it, I cannot see a reason why not.

> The frontend would then manage multiple varObjs representing the slices of an array.
> The CDT does that.
> DSF does not do that because Pawel came up with a simpler idea (and because I wasn't
> aware of the slice thing at the time; no doc on that slice thing, right?):
> for arrays, we do not use -var-list-children, but instead, we -var-create each child
> ourselves, when needed.
> 
> So, in our case, we have managed to work around large arrays.
> However, I saw Nick's proposal as valuable because it applies to all structs and not
> just arrays.  Also, it avoids the frontend having to have special logic for arrays.
> 
> > 3. "Data analysis". User actually wants to make some sense of all 10000 element.
> > Maybe, well, draw a chart. Or just look at the data. The data is integer, or some
> > other atomic type. Then, either some specialized widget, or merely the memory view,
> > is quite adequate.
> > 
> > What I don't see as a valid use case is one where:
> > 
> > 1. The size of array is too large to keep all varobj always created.
> 
> I'm not sure what you mean here.  Before Nick's patch, issuing -var-list-children
> on a large array would create a lot of varObjs.  In our case, it would have
> filled up our varObj cache, although we would only be showing very few of the
> array elements.

Well, then, the problem is in *your* code :-) If GDB has no problem creating
variables objects for all children, but your cache fills up, then you have wrong
size of a cache. Honestly, I suspect the best way would be just to remove the
caching, but I need to know exactly what is the key of the cache is.

> 
> > 2. User would like to just scroll though all the array items, without
> > clear intentions.
> 
> This is probably not a valid use case... although user behavior, even when dumb :-)
> should not break the frontend.

Yes, this is my "Oops" use case -- we should create some reasonable number of children
initially, and incrementally fetch more per user request, thereby giving the user a
chance to stop :-)

> But you got me thinking now... Maybe var-create using with an array slice is all we
> need for dealing with a large array?

I don't know. It means that if you want to get more children, you have to delete variable
object, and create new one and list children. It seems that -var-list-children V 10-20
is easier -- if -var-list-children can do this, then for example, incremental children
fetch in KDevelop will be implemented in a single line of code.

- Volodya

Re: [PATCH:MI] Return a subset of a variable object's children

[Daniel Jacobowitz]

Still reading the rest of the thread, but...

On Wed, Apr 30, 2008 at 10:29:39AM +1200, Nick Roberts wrote:
>  > but only actually create the children that have been requested by the user.
>  > I'm not sure how much efficiency there is by allocating the memory before
>  > hand?  Also, is there no way to grow the vector by more than a single point
>  > at a time?
> 
> Like resize with STL vectors?  I'm not aware of one.

VEC_reserve and VEC_quick_push.  There's no clean way to memset
the new elements but this will cut down on allocation anyway.

-- 
Daniel Jacobowitz
CodeSourcery

Re: [PATCH:MI] Return a subset of a variable object's children

[Daniel Jacobowitz]

On Thu, May 01, 2008 at 10:39:59PM +0400, Vladimir Prus wrote:
> I don't know :-) I think that myArray@10 is gdb CLI trick; I never though it could
> be used in MI, but thinking of it, I cannot see a reason why not.

Yes, it can, and it's documented - just not in the MI chapter.  The
bit on the end of -var-create is just a GDB source language
expression, so refer to Artificial Arrays in the manual.

If this is the same as slicing, then it'd be nice to mention the
word "slice" in that chapter somewhere :-)


-- 
Daniel Jacobowitz
CodeSourcery

Re: [PATCH:MI] Return a subset of a variable object's children

[Nick Roberts]

 > > I don't know :-) I think that myArray@10 is gdb CLI trick; I never though
 > > it could be used in MI, but thinking of it, I cannot see a reason why not.
 > 
 > Yes, it can, and it's documented - just not in the MI chapter.  The
 > bit on the end of -var-create is just a GDB source language
 > expression, so refer to Artificial Arrays in the manual.
 > 
 > If this is the same as slicing, then it'd be nice to mention the
 > word "slice" in that chapter somewhere :-)

This is not the same as what I am proposing though.  Doing

-var-create - * myArray@10

and

-var-create - * (myArray + 100)@20

say would create two separate varobjs which the frontend would have to
juggle.  I am proposing one varobj to be a parent of all the children.

-- 
Nick                                           http://www.inet.net.nz/~nickrob

RE: [PATCH:MI] Return a subset of a variable object's children

[Marc Khouzam]

> > DSF-GDB tries to be as efficient as possible by avoiding request that go
> > to the backend.  Therefore, although a varObj is no longer being showed,
> > we want to keep it around in case it will be showed again.  Effectively,
> > we are caching. And our cache size is 1000 varObj.
> 
> What does it mean "no longer being showed". If you have a variables tree,
> and some variable is in the tree, but is not visible on the screen right now,
> there's no need whatsoever to delete the variable object for that variable.

Ok, I had misunderstood your previous explanation on when to delete varObj.
When I say "not shown" I mean that it is not visible on the screen right now.
So in DSF, we don't delete varObj that are not on the screen, but keep them
in what I described earlier as a cache.

> If that variable is huge structure, then updating it (inside GDB) on each
> step, can take some time, especially on a slow remote target. But, I think
> you send -var-update once per each visible variable, so this is not an issue
> for you.

> If a variable is deleted from variables tree (because you left the scope
> where that variable is defined), then you should delete variable object.
> There's no reliable way to detect that exactly same variable is in scope
> again and must be shown, so no reuse is possible.

We chose a more passive approach.  The frontend relies entirely on GDB to 
know if a varObj has gone out-of-scope.  Since GDB reports this
only if the varObj is given the var-update command, we don't get to know
it very often, because we only use the var-update command for varObj we want
to know the value of.
I believe an example is in order :-)

Say you have local variable bar in method foo().
We create a varObj for bar.  Once we return from foo(), there is no longer
a local variable bar, so we don't send a -var-update for it, which in turn
means we don't realize that it is out-of-scope and we don't delete it.

What this implies in the end is that we only delete a varObj if our cache
is full.  The only exception to this rule is when the expression of a 
variable object is re-used which will trigger a var-update, which will
show that the old varObj is out-of-scope (so we create a new one.)
(if after returning from foo(), there is another local variable bar)

> Honestly, except for the fact that -var-update reads target memory, I'm sure
> that GDB can create more variable objects that SWT can create TreeItems :-)

That is exactly what I wanted to know.  I won't worry so much about
having too many varObj :-)

> > > There are natural scalability bounds in the UI -- if there are no
> > > arrays involved, and you try to show 1000 items (backed by 1000 variable objects),
> > > the UI is already useless. GDB is not the problem here.
> > 
> > Right, but for us the 1000 varObj are not meant to all be displayed at once
> > but are a caching system.
> 
> I'm still not sure what you mean -- you mean that a variable object is not shown
> in any widget, but only is stored in the cache? What is the key of the cache?

Ah yes, the key of the cache.  That gave me a big headache :-)
It is a object containing: the expression for the varObj, the thread it is part of,
and the frame level.
If the same expression (variable name) is used in the same thread at the same frame
level, then we first think it is the same varObj and we use -var-update; we then
get an out-of-scope and we delete the old varObj and create a new one.


> > > What I don't see as a valid use case is one where:
> > > 
> > > 1. The size of array is too large to keep all varobj always created.
> > 
> > I'm not sure what you mean here.  Before Nick's patch, issuing -var-list-children
> > on a large array would create a lot of varObjs.  In our case, it would have
> > filled up our varObj cache, although we would only be showing very few of the
> > array elements.
> 
> Well, then, the problem is in *your* code :-) If GDB has no problem creating
> variables objects for all children, but your cache fills up, then you have wrong
> size of a cache.

That may be true.  But it would be nice to have the freedom to control this
ourselves, and therefore, not to be forced to create a very large amount
of children; which is where Nick's patch comes in, or the "slicing of arrays",
our the DSF solution of not listing the children of arrays.

> > > 2. User would like to just scroll though all the array items, without
> > > clear intentions.
> > 
> > This is probably not a valid use case... although user behavior, even when dumb :-)
> > should not break the frontend.
> 
> Yes, this is my "Oops" use case -- we should create some reasonable number of children
> initially, and incrementally fetch more per user request, thereby giving the user a
> chance to stop :-)

Exactly.

> 
> > But you got me thinking now... Maybe var-create using with an array slice is all we
> > need for dealing with a large array?
> 
> I don't know. It means that if you want to get more children, you have to delete variable
> object, and create new one and list children.

Not really.  I believe what CDT does is that each array bigger than 100 is divided into
groups of 100 children.  So, there is a new varObj for each group of 100; the "slice"
is not increased but instead, a new "slice" is created.

Marc

Re: [PATCH:MI] Return a subset of a variable object's children

[André Pönitz]

On Thursday 01 May 2008 08:27:10 you wrote:
>  > > Are you arguing against the general concept of variable objects?
>  > 
>  > Erm... yes. Could be seen as such.
> 
> With respect,  I'm not sure that you know what you are talking about. 
>
> The  concept of variable objects predates MI and was take from Insight.  It hasn't
> been developed overnight but over a long time by experienced Gdb developers,
> i.e, not the likes of me.

[With respect, I never was very fond of "proof by authority". Anyway, maybe
I should have mentioned "for my usage pattern" or "in my environment"
a bit more often. Sorry for that. Having said that, it's not uncommon that
new approaches render venerable code once deviced by venerable people 
obsolete. For me it looks like the new Python interface has a huge potential
in this direction...]

>  > > ...
>  > > Presumably your one-shot 'children listing' requires all values to be
>  > > printed each time.  Variable objects work by just printing the values
>  > > which have changed.
>  >  
>  > I am fully aware of that. It has a very limited utility, though...
>  > 
>  > I basically have to fill, say, 20 lines in the typical 'Locals & Watchers'
>  > view.
>  > 
>  > With a one-shot approach I can fire off 20 simple request, getting back
>  > 20 simple results (say, a total of 1000 bytes of data) compare them with
>  > the previous results myself, re-populate the view, mark changes etc.
>  > No big deal as it happens at most once per user interaction. Easy and 
>  > predictable. 
> 
> How do you know if an expression is still in scope? How do you know that  
> an original variable is not currently shadowed by another with the same name?
> How do you know how many elements are in an array, members in a structure?
> These are some of the questions that variable objects address.

Well, given that I have a 'program counter position' in each stack frame
on the current stack and that I do 'one-shot' evaluations I would expect
an expression to be evaluated in that context. Not being able to access
shadowed variables does hurt _me_ not much. It's definitely less harmful
than, say, getting no local variable information for constructor bodies.

Apart from that, I do not even know whether a C++ object has already
been constructed and whether it's safe to access it's members. 
Lately a coworker had a funny case (gdb 6.5 on MinGW IIRC) that accessing
a variable in one stack frame and a reference to it in another yielded 
different values (same address, non-reproducable but witnessed by
half a dozen people). I also better do a stack down and up before
asking any hard questions about the current frame on MinGW as the
information then just has a tendency to look better. Not to mention
that I grow accustomed to ignore any reports for frames at line 150
in stl_algobase.h. Etc...

I am not complaining. At all. Gdb is much more reliable and fun to work 
with than some other debuggers. Believe me, If I had the impression I 
could do your job (or even a part of that part that I am aware of) I would
have probably tried ;-)

So really no offense meant. I am just trying to illustrate my point that I
do not expect a 100% perfect solution, and that I am therefore willing 
to trade _the potential_ of getting a 100% perfect solution (which I
consider a non-achievable illusion anyway) for an easy-to-use solution 
that is not too far off. I mean, nobody uses _debugger output_ to actually
_steer_ mission critical applications. The debugger is just a means to collect
information. If one piece of information is missing or even wrong, well, so
be it. The application itself can still be fixed. Mission accomplished ;-)

>  > With the 'update notifications' I get an unpredictable amount of data
>  > (there could be an array with a few thousand children after all, all
>  > being changed, stalling my communication line for an unpredicatable
>  > amount of time),
> 
> This is what we're currently discussing: ways to ensure that thousands of
> variable objects aren't created when viewing arrays with thousands of elements.
> 
>  >                 _and_ the data I get is insufficient as it, for instance, 
>  > does not tell me that a std::string has changed somewhere in the middle.
>  > So I need to do a few 'one-shot evaluations' anyway to get the desired
>  > results. All in all, less satisfactory.
> 
> And maybe the Python interface can address this.

Indeed. But deploying the Python interface "for everything" starting with
listing the locals, running custom visualizers on all of them etc basically
means "no need for varobjects anymore". Which fits, more or less,
into my (limited, as you noticed) view of the world ;-)

Regards,
Andre'

Re: [PATCH:MI] Return a subset of a variable object's children

[Nick Roberts]

 >  > > Clearly I'm not going to do either but we could simply go back to using
 >  > > the linked list structures that were already in varobj.c.  It's a
 >  > > question of whether the convenience outweighs the handicap of having to
 >  > > work with vectors all the time or not.  IMHO it doesn't.
 >  > 
 >  > I disagree, and I haven't yet seen a practical "handicap". We're not going
 >  > back to hand-written data structures for MI implementation.
 > 
 > Well it's a shame that convenience should dictate the type of structures that
 > are used because I can get this patch to work with an old Gdb using linked
 > lists.

As you've made your decision, the patch below isn't a RFA or RFC but a record
for the archives to ensure it's not lost like other patches that I've left
lying around.  I feel sure that one day the issue will arise again, probably
when Eclipse finds a requirement for it.

It's against current Gdb and uses the old linked lists.  This means that the
size of the data structure is only proportional to the number of children
created and not the total number of possible children.  Unlike the previous
patch, children are listed in sequence for both creation and update.  Also
children can be deleted.  Searching for a location to insert a child in the
list is linear (as before) but I anticipate that a front end would only ever
create a small number at any one time and delete children (that are not
visible) if the number became too great.  In any case the search could be made
binary (O(log N)), if necessary.

Oh, yes, and it's a unified diff!

Here's a sample output.  Note that var1.7 is deleted and therefore not
listed in the output of -var-update.

-var-create - * m
^done,name="var1",numchild="10",value="[10]",type="int [10]",thread-id="1"
(gdb) 
-var-list-children -f 7 -n 2 var1
^done,numchild="2",children=[child={name="var1.7",exp="7",numchild="0",value="-1209567320",type="int",thread-id="1"},child={name="var1.8",exp="8",numchild="0",value="134513116",type="int",thread-id="1"}]
(gdb) 
-var-list-children -f 4 -n 3 -s 2 var1
^done,numchild="4",children=[child={name="var1.4",exp="4",numchild="0",value="-1208142416",type="int",thread-id="1"},child={name="var1.6",exp="6",numchild="0",value="134513375",type="int",thread-id="1"},child={name="var1.7",exp="7",numchild="0",value="-1209567320",type="int",thread-id="1"},child={name="var1.8",exp="8",numchild="0",value="134513116",type="int",thread-id="1"}]
(gdb) 
-var-delete var1.7      
^done,ndeleted="1"
(gdb) 
n
&"n\n"
~"151\t  int i, n, m[10] = {0, 1, 4, 9, 16, 25, 36, 49, 64, 81};\n"
^done
(gdb) 
n
&"n\n"
~"152\t  int m1[2][3] = {{0, 1, 2}, {10, 11, 12}};\n"
^done
(gdb) 
-var-update --all-values *
^done,changelist=[{name="var1.4",value="16",in_scope="true",type_changed="false"},{name="var1.6",value="36",in_scope="true",type_changed="false"},{name="var1.8",value="64",in_scope="true",type_changed="false"}]
(gdb) 

-- 
Nick                                           http://www.inet.net.nz/~nickrob


2008-05-10  Nick Roberts  <nickrob@snap.net.nz>

	* mi/mi-cmd-var.c (mi_cmd_var_list_children): Add options to select
	a subset of children.

	* varobj.h (varobj_list_children): Declare new parameters.

	* varobj.c (struct varobj): New member current_children.
	(varobj_list_children): Create any new varobjs for children
	specified by mi_cmd_var_list_children.
	(create_child): Add parameter real_index.  Use it.
	+ Go back to using the linked list structures.


--- mi-cmd-var.c	20 Apr 2008 10:20:39 +1200	1.50
+++ mi-cmd-var.c	06 May 2008 21:59:54 +1200	
@@ -367,47 +367,114 @@ mi_print_value_p (struct type *type, enu
 enum mi_cmd_result
 mi_cmd_var_list_children (char *command, char **argv, int argc)
 {
-  struct varobj *var;  
-  VEC(varobj_p) *children;
-  struct varobj *child;
+  struct varobj *var;
+  struct varobj **childlist;
+  struct varobj **cc;
   struct cleanup *cleanup_children;
   int numchild;
   enum print_values print_values;
-  int ix;
+  int start, number, stride = 1;
 
-  if (argc != 1 && argc != 2)
-    error (_("mi_cmd_var_list_children: Usage: [PRINT_VALUES] NAME"));
+  enum opt
+    {
+      NO_VALUES, SIMPLE_VALUES, ALL_VALUES, FROM_OPT, NUMBER_OPT, STRIDE_OPT
+    };
 
-  /* Get varobj handle, if a valid var obj name was specified */
-  if (argc == 1)
-    var = varobj_get_handle (argv[0]);
-  else
-    var = varobj_get_handle (argv[1]);
-  if (var == NULL)
-    error (_("Variable object not found"));
+  static struct mi_opt opts[] =
+  {
+      { "-no-values", NO_VALUES, 0 },
+      { "-simple-values", SIMPLE_VALUES, 0 },
+      { "-all-values", ALL_VALUES, 0 },
+      { "f", FROM_OPT, 1 },
+      { "n", NUMBER_OPT, 1 },
+      { "s", STRIDE_OPT, 1 },
+      { 0, 0, 0 }
+  };
 
-  children = varobj_list_children (var);
-  ui_out_field_int (uiout, "numchild", VEC_length (varobj_p, children));
-  if (argc == 2)
-    print_values = mi_parse_values_option (argv[0]);
+  if (argc == 1 || argc == 2)
+    {
+      /* Get varobj handle, if a valid var obj name was specified */
+      if (argc == 1)
+	var = varobj_get_handle (argv[0]);
+      else
+	var = varobj_get_handle (argv[1]);
+      if (var == NULL)
+	error (_("Variable object not found"));
+
+      if (argc == 2)
+	print_values = mi_parse_values_option (argv[0]);
+      else
+	print_values = PRINT_NO_VALUES;
+
+      start = 0;
+      number = varobj_get_num_children (var);
+    }
   else
-    print_values = PRINT_NO_VALUES;
+    {
+      int optind = 0;
+      char *optarg;
+      while (1)
+	{
+	  int opt = mi_getopt ("mi_cmd_var_list_children",
+			       argc, argv, opts, &optind, &optarg);
+	  if (opt < 0)
+	    break;
+	  switch ((enum opt) opt)
+	    {
+	    case NO_VALUES:
+	      print_values = PRINT_NO_VALUES;
+	      break;
+	    case SIMPLE_VALUES:
+	      print_values = PRINT_SIMPLE_VALUES;
+	      break;
+	    case ALL_VALUES:
+	      print_values = PRINT_ALL_VALUES;
+	      break;
+	    case FROM_OPT:
+	      start = atol (optarg);
+	      if (start < 0)
+		start = 0;
+	      break;
+	    case NUMBER_OPT:
+	      number = atol (optarg);
+	      break;
+	    case STRIDE_OPT:
+	      stride = atol (optarg);
+	      if (stride < 1)
+	      	error (_("Positive stride values only"));
+	      break;
+	    }
+	}
+      if (optind >= argc)
+	error (_("Missing VARNAME"));
+
+      if (optind < argc - 1)
+	error (_("Garbage at end of command"));
+
+      var = varobj_get_handle (argv[optind]);
+    }
+
+  numchild = varobj_list_children (var, start, number, stride, &childlist);
+  ui_out_field_int (uiout, "numchild", numchild);
 
-  if (VEC_length (varobj_p, children) == 0)
+  if (numchild <= 0)
     return MI_CMD_DONE;
 
   if (mi_version (uiout) == 1)
     cleanup_children = make_cleanup_ui_out_tuple_begin_end (uiout, "children");
   else
     cleanup_children = make_cleanup_ui_out_list_begin_end (uiout, "children");
-  for (ix = 0; VEC_iterate (varobj_p, children, ix, child); ++ix)
+  cc = childlist;
+  while (*cc != NULL)
     {
       struct cleanup *cleanup_child;
       cleanup_child = make_cleanup_ui_out_tuple_begin_end (uiout, "child");
-      print_varobj (child, print_values, 1 /* print expression */);
+      print_varobj (*cc, print_values, 1 /* print expression */);
+      cc++;
       do_cleanups (cleanup_child);
     }
   do_cleanups (cleanup_children);
+  xfree (childlist);
   return MI_CMD_DONE;
 }
 

--- varobj.h	10 Apr 2008 08:41:40 +1200	1.18
+++ varobj.h	06 May 2008 22:00:27 +1200	
@@ -20,7 +20,6 @@
 
 #include "symtab.h"
 #include "gdbtypes.h"
-#include "vec.h"
 
 /* Enumeration for the format types */
 enum varobj_display_formats
@@ -62,9 +61,6 @@ extern char *varobj_language_string[];
 /* Struct thar describes a variable object instance */
 struct varobj;
 
-typedef struct varobj *varobj_p;
-DEF_VEC_P (varobj_p);
-
 /* API functions */
 
 extern struct varobj *varobj_create (char *objname,
@@ -89,17 +85,16 @@ extern enum varobj_display_formats varob
 extern enum varobj_display_formats varobj_get_display_format (
 							struct varobj *var);
 
+extern int varobj_get_num_children (struct varobj *var);
+
+extern int varobj_list_children (struct varobj *var, int start, int number, int stride,
+				 struct varobj ***childlist);
 extern int varobj_get_thread_id (struct varobj *var);
 
 extern void varobj_set_frozen (struct varobj *var, int frozen);
 
 extern int varobj_get_frozen (struct varobj *var);
 
-extern int varobj_get_num_children (struct varobj *var);
-
-/* Return the list of children of VAR.  The returned vector
-   should not be modified in any way.  */
-extern VEC (varobj_p)* varobj_list_children (struct varobj *var);
 
 extern char *varobj_get_type (struct varobj *var);
 

--- varobj.c	10 May 2008 14:38:50 +1200	1.112
+++ varobj.c	10 May 2008 14:39:47 +1200	
@@ -30,7 +30,6 @@
 #include "gdb_string.h"
 
 #include "varobj.h"
-#include "vec.h"
 #include "gdbthread.h"
 #include "inferior.h"
 
@@ -131,11 +130,15 @@ struct varobj
   /* The number of (immediate) children this variable has */
   int num_children;
 
+  /* The number of (immediate) children this variable that exist as varobjs.  
+     current_children < num_children  */
+  int current_children;
+
   /* If this object is a child, this points to its immediate parent. */
   struct varobj *parent;
 
-  /* Children of this object.  */
-  VEC (varobj_p) *children;
+  /* A list of this object's children */
+  struct varobj_child *children;
 
   /* Description of the root variable. Points to root variable for children. */
   struct varobj_root *root;
@@ -160,6 +163,29 @@ struct varobj
   int not_fetched;
 };
 
+/* Every variable keeps a linked list of its children, described
+   by the following structure. */
+/* FIXME: Deprecated.  All should use vlist instead */
+
+struct varobj_child
+{
+
+  /* Pointer to the child's data */
+  struct varobj *child;
+
+  /* Pointer to the next child */
+  struct varobj_child *next;
+};
+
+/* A stack of varobjs */
+/* FIXME: Deprecated.  All should use vlist instead */
+
+struct vstack
+{
+  struct varobj *var;
+  struct vstack *next;
+};
+
 struct cpstack
 {
   char *name;
@@ -187,8 +213,14 @@ static int install_variable (struct varo
 
 static void uninstall_variable (struct varobj *);
 
+static struct varobj *child_exists (struct varobj *, char *);
+
 static struct varobj *create_child (struct varobj *, int, char *);
 
+static void save_child_in_parent (struct varobj *, struct varobj *);
+
+static void remove_child_from_parent (struct varobj *, struct varobj *);
+
 /* Utility routines */
 
 static struct varobj *new_variable (void);
@@ -207,6 +239,10 @@ static struct type *get_target_type (str
 
 static enum varobj_display_formats variable_default_display (struct varobj *);
 
+static void vpush (struct vstack **pstack, struct varobj *var);
+
+static struct varobj *vpop (struct vstack **pstack);
+
 static void cppush (struct cpstack **pstack, char *name);
 
 static char *cppop (struct cpstack **pstack);
@@ -751,41 +787,58 @@ varobj_get_num_children (struct varobj *
 /* Creates a list of the immediate children of a variable object;
    the return code is the number of such children or -1 on error */
 
-VEC (varobj_p)*
-varobj_list_children (struct varobj *var)
+int
+varobj_list_children (struct varobj *var, int start, int number, int stride,
+		      struct varobj ***childlist)
 {
   struct varobj *child;
+  struct varobj_child *vc;
   char *name;
-  int i;
+  int i, new_children = 0;
 
-  if (var->num_children == -1)
-    var->num_children = number_of_children (var);
+  /* sanity check: have we been passed a pointer? */
+  if (childlist == NULL)
+    return -1;
+
+  *childlist = NULL;
 
-  /* If that failed, give up.  */
   if (var->num_children == -1)
-    return var->children;
+    var->num_children = number_of_children (var);
 
-  /* If we're called when the list of children is not yet initialized,
-     allocate enough elements in it.  */
-  while (VEC_length (varobj_p, var->children) < var->num_children)
-    VEC_safe_push (varobj_p, var->children, NULL);
+  /* Allocate maximum required memory */
+  *childlist = xmalloc ((var->current_children + number + 1)
+			* sizeof (struct varobj *));
 
-  for (i = 0; i < var->num_children; i++)
+  for (i = 0; i < number; i++)
     {
-      varobj_p existing = VEC_index (varobj_p, var->children, i);
+      int real_index = start + i * stride;
+      if (real_index >= var->num_children)
+	break;
+
+      /* Mark as the end in case we bail out */
+      *((*childlist) + i) = NULL;
+
+       /* check if child exists, if not create */
+      name = name_of_child (var, real_index);
+      child = child_exists (var, name);
 
-      if (existing == NULL)
+      if (child == NULL)
 	{
-	  /* Either it's the first call to varobj_list_children for
-	     this variable object, and the child was never created,
-	     or it was explicitly deleted by the client.  */
-	  name = name_of_child (var, i);
-	  existing = create_child (var, i, name);
-	  VEC_replace (varobj_p, var->children, i, existing);
+	  child = create_child (var, real_index, name);
+	  new_children++;
 	}
     }
 
-  return var->children;
+  var->current_children += new_children;
+
+  /* Revert the order. */
+  for (vc = var->children, i = 0; vc != NULL; vc = vc->next, i++)
+    *((*childlist) + var->current_children - 1 - i) = vc->child;
+
+  /* End of list is marked by a NULL pointer */
+  *((*childlist) + i) = NULL;
+
+  return var->current_children;
 }
 
 /* Obtain the type of an object Variable as a string similar to the one gdb
@@ -1145,8 +1198,8 @@ varobj_update (struct varobj **varp, str
   struct varobj **cv;
   struct varobj **templist = NULL;
   struct value *new;
-  VEC (varobj_p) *stack = NULL;
-  VEC (varobj_p) *result = NULL;
+  struct vstack *stack = NULL;
+  struct vstack *result = NULL;
   struct frame_info *fi;
 
   /* sanity check: have we been passed a pointer?  */
@@ -1162,6 +1215,9 @@ varobj_update (struct varobj **varp, str
 
   if (!(*varp)->root->is_valid)
     return INVALID;
+ 
+  /* Initialize a stack for temporary results */
+  vpush (&result, NULL);
 
   if ((*varp)->root->rootvar == *varp)
     {
@@ -1173,47 +1229,47 @@ varobj_update (struct varobj **varp, str
       new = value_of_root (varp, &type_changed);
       
       /* If this is a floating varobj, and its type has changed,
-	 them note that it's changed.  */
+	 then note that it's changed.  */
       if (type_changed)
-	VEC_safe_push (varobj_p, result, *varp);
-      
-        if (install_new_value ((*varp), new, type_changed))
-	  {
-	    /* If type_changed is 1, install_new_value will never return
-	       non-zero, so we'll never report the same variable twice.  */
-	    gdb_assert (!type_changed);
-	    VEC_safe_push (varobj_p, result, *varp);
-	  }
+	{
+	  vpush (&result, *varp);
+	  changed++;
+	}
+
+      if (install_new_value ((*varp), new, type_changed))
+	{
+	  /* If type_changed is 1, install_new_value will never return
+	     non-zero, so we'll never report the same variable twice.  */
+	  gdb_assert (!type_changed);
+	  vpush (&result, (*varp));
+	  changed++;
+	}
 
       if (new == NULL)
 	{
 	  /* This means the varobj itself is out of scope.
 	     Report it.  */
-	  VEC_free (varobj_p, result);
+	  vpop (&result);
 	  return NOT_IN_SCOPE;
 	}
     }
+ 
+  /* Initialize a stack */
+  vpush (&stack, NULL);
 
-  VEC_safe_push (varobj_p, stack, *varp);
-
-  /* Walk through the children, reconstructing them all.  */
-  while (!VEC_empty (varobj_p, stack))
+  /* Walk through the children, reconstructing them all. */
+  v = *varp;
+  while (v != NULL)
     {
-      v = VEC_pop (varobj_p, stack);
-
-      /* Push any children.  Use reverse order so that the first
-	 child is popped from the work stack first, and so
-	 will be added to result first.  This does not
-	 affect correctness, just "nicer".  */
-      for (i = VEC_length (varobj_p, v->children)-1; i >= 0; --i)
+      /* Push any unfrozen children */
+      if (v->children != NULL)
 	{
-	  varobj_p c = VEC_index (varobj_p, v->children, i);
-	  /* Child may be NULL if explicitly deleted by -var-delete.  */
-	  if (c != NULL && !c->frozen)
-	    VEC_safe_push (varobj_p, stack, c);
+	  struct varobj_child *c;
+	  for (c = v->children; c != NULL; c = c->next)
+	    if (!c->child->frozen) vpush (&stack, c->child);
 	}
 
-      /* Update this variable, unless it's a root, which is already
+      /* Update this variable unless it's a root, which is already
 	 updated.  */
       if (v->root->rootvar != v)
 	{	  
@@ -1221,27 +1277,48 @@ varobj_update (struct varobj **varp, str
 	  if (install_new_value (v, new, 0 /* type not changed */))
 	    {
 	      /* Note that it's changed */
-	      VEC_safe_push (varobj_p, result, v);
+	      vpush (&result, v);
 	      v->updated = 0;
+	      changed++;
 	    }
 	}
+
+      /* Get next child */
+      v = vpop (&stack);
     }
 
-  /* Alloc (changed + 1) list entries.  */
-  changed = VEC_length (varobj_p, result);
+  /* Alloc (changed + 1) list entries */
+  /* FIXME: add a cleanup for the allocated list(s)
+     because one day the select_frame called below can longjump */
   *changelist = xmalloc ((changed + 1) * sizeof (struct varobj *));
-  cv = *changelist;
+  if (changed > 1)
+    {
+      templist = xmalloc ((changed + 1) * sizeof (struct varobj *));
+      cv = templist;
+    }
+  else
+    cv = *changelist;
 
-  for (i = 0; i < changed; ++i)
+  /* Copy from result stack to list */
+  vleft = changed;
+  *cv = vpop (&result);
+  while ((*cv != NULL) && (vleft > 0))
     {
-      *cv = VEC_index (varobj_p, result, i);
-      gdb_assert (*cv != NULL);
-      ++cv;
+      vleft--;
+      cv++;
+      *cv = vpop (&result);
     }
-  *cv = 0;
+  if (vleft)
+    warning (_("varobj_update: assertion failed - vleft <> 0"));
 
-  VEC_free (varobj_p, stack);
-  VEC_free (varobj_p, result);
+  if (changed > 1)
+    {
+      /* Now we revert the order. */
+      for (i = 0; i < changed; i++)
+	//	*(*changelist + i) = *(templist + i);
+	*(*changelist + i) = *(templist + changed - 1 - i);
+      *(*changelist + changed) = NULL;
+    }
 
   if (type_changed)
     return TYPE_CHANGED;
@@ -1277,19 +1354,18 @@ delete_variable_1 (struct cpstack **resu
 		   struct varobj *var, int only_children_p,
 		   int remove_from_parent_p)
 {
-  int i;
+  struct varobj_child *vc;
+  struct varobj_child *next;
 
   /* Delete any children of this variable, too. */
-  for (i = 0; i < VEC_length (varobj_p, var->children); ++i)
-    {   
-      varobj_p child = VEC_index (varobj_p, var->children, i);
-      if (!child)
-	continue;
+  for (vc = var->children; vc != NULL; vc = next)
+    {
       if (!remove_from_parent_p)
-	child->parent = NULL;
-      delete_variable_1 (resultp, delcountp, child, 0, only_children_p);
+	vc->child->parent = NULL;
+      delete_variable_1 (resultp, delcountp, vc->child, 0, only_children_p);
+      next = vc->next;
+      xfree (vc);
     }
-  VEC_free (varobj_p, var->children);
 
   /* if we were called to delete only the children we are done here */
   if (only_children_p)
@@ -1311,7 +1387,7 @@ delete_variable_1 (struct cpstack **resu
      discarding the list afterwards */
   if ((remove_from_parent_p) && (var->parent != NULL))
     {
-      VEC_replace (varobj_p, var->parent->children, var->index, NULL);
+      remove_child_from_parent (var->parent, var);
     }
 
   if (var->obj_name != NULL)
@@ -1440,6 +1516,22 @@ uninstall_variable (struct varobj *var)
 
 }
 
+/* Does a child with the name NAME exist in VAR? If so, return its data.
+   If not, return NULL. */
+static struct varobj *
+child_exists (struct varobj *var, char *name)
+{
+  struct varobj_child *vc;
+
+  for (vc = var->children; vc != NULL; vc = vc->next)
+    {
+      if (strcmp (vc->child->name, name) == 0)
+	return vc->child;
+    }
+
+  return NULL;
+}
+
 /* Create and install a child of the parent of the given name */
 static struct varobj *
 create_child (struct varobj *parent, int index, char *name)
@@ -1460,6 +1552,9 @@ create_child (struct varobj *parent, int
   child->obj_name = childs_name;
   install_variable (child);
 
+  /* Save a pointer to this child in the parent */
+  save_child_in_parent (parent, child);
+
   /* Compute the type of the child.  Must do this before
      calling install_new_value.  */
   if (value != NULL)
@@ -1474,6 +1569,57 @@ create_child (struct varobj *parent, int
 
   return child;
 }
+
+/* FIXME: This should be a generic add to list */
+/* Save CHILD in the PARENT's data. */
+static void
+save_child_in_parent (struct varobj *parent, struct varobj *child)
+{
+  struct varobj_child *vc, *prev = NULL, *next;
+
+  /* Insert the child according to index.  */
+  vc = parent->children;
+  while (vc != NULL && vc->child->index > child->index)
+    {
+      prev = vc;
+      vc = vc->next;
+    }
+
+  next = vc;
+  vc = (struct varobj_child *) xmalloc (sizeof (struct varobj_child));
+
+  vc->next = next;
+  vc->child = child;
+
+  if (prev == NULL)
+    parent->children = vc;
+  else
+    prev->next = vc;
+}
+
+/* FIXME: This should be a generic remove from list */
+/* Remove the CHILD from the PARENT's list of children. */
+static void
+remove_child_from_parent (struct varobj *parent, struct varobj *child)
+{
+  struct varobj_child *vc, *prev;
+
+  /* Find the child in the parent's list */
+  prev = NULL;
+  for (vc = parent->children; vc != NULL;)
+    {
+      if (vc->child == child)
+	break;
+      prev = vc;
+      vc = vc->next;
+    }
+
+  if (prev == NULL)
+    parent->children = vc->next;
+  else
+    prev->next = vc->next;
+
+}
 \f
 
 /*
@@ -1494,6 +1640,7 @@ new_variable (void)
   var->type = NULL;
   var->value = NULL;
   var->num_children = -1;
+  var->current_children = 0;
   var->parent = NULL;
   var->children = NULL;
   var->format = 0;
@@ -1629,6 +1776,36 @@ variable_default_display (struct varobj 
 
 /* FIXME: The following should be generic for any pointer */
 static void
+vpush (struct vstack **pstack, struct varobj *var)
+{
+  struct vstack *s;
+
+  s = (struct vstack *) xmalloc (sizeof (struct vstack));
+  s->var = var;
+  s->next = *pstack;
+  *pstack = s;
+}
+
+/* FIXME: The following should be generic for any pointer */
+static struct varobj *
+vpop (struct vstack **pstack)
+{
+  struct vstack *s;
+  struct varobj *v;
+
+  if ((*pstack)->var == NULL && (*pstack)->next == NULL)
+    return NULL;
+
+  s = *pstack;
+  v = s->var;
+  *pstack = (*pstack)->next;
+  xfree (s);
+
+  return v;
+}
+
+/* FIXME: The following should be generic for any pointer */
+static void
 cppush (struct cpstack **pstack, char *name)
 {
   struct cpstack *s;

RE: [PATCH:MI] Return a subset of a variable object's children

[Vladimir Prus]

Marc Khouzam wrote:

>> If a variable is deleted from variables tree (because you left the scope
>> where that variable is defined), then you should delete variable object.
>> There's no reliable way to detect that exactly same variable is in scope
>> again and must be shown, so no reuse is possible.
> 
> We chose a more passive approach.  The frontend relies entirely on GDB to
> know if a varObj has gone out-of-scope.  Since GDB reports this
> only if the varObj is given the var-update command, we don't get to know
> it very often, because we only use the var-update command for varObj we want
> to know the value of.
> I believe an example is in order :-)
> 
> Say you have local variable bar in method foo().
> We create a varObj for bar.  Once we return from foo(), there is no longer
> a local variable bar, so we don't send a -var-update for it, which in turn
> means we don't realize that it is out-of-scope and we don't delete it.
> 
> What this implies in the end is that we only delete a varObj if our cache
> is full.  The only exception to this rule is when the expression of a
> variable object is re-used which will trigger a var-update, which will
> show that the old varObj is out-of-scope (so we create a new one.)
> (if after returning from foo(), there is another local variable bar)

That does not sound like an ideal approach, because given that GDB knows 
for sure which variable objects are in scope, and which are not, you are
using some heuristics to to guess that, and it will never be 100% ideal.
Having said that, while GDB knows everything, MI interfaces are somewhat
lacking as far as local variables are concerned. I plan to design command(s)
that allow to easily and reliably track local variables, and will post those.

>> > > There are natural scalability bounds in the UI -- if there are no
>> > > arrays involved, and you try to show 1000 items (backed by 1000 variable objects),
>> > > the UI is already useless. GDB is not the problem here.
>> > 
>> > Right, but for us the 1000 varObj are not meant to all be displayed at once
>> > but are a caching system.
>> 
>> I'm still not sure what you mean -- you mean that a variable object is not shown
>> in any widget, but only is stored in the cache? What is the key of the cache?
> 
> Ah yes, the key of the cache.  That gave me a big headache :-)
> It is a object containing: the expression for the varObj, the thread it is part of,
> and the frame level.
> If the same expression (variable name) is used in the same thread at the same frame
> level, then we first think it is the same varObj and we use -var-update; we then
> get an out-of-scope and we delete the old varObj and create a new one.

I see. I hope this approach won't be necessary soon (though only if you're willing
to conditionalize the logic in gdb version).

- Volodya

Re: [PATCH:MI] Return a subset of a variable object's children

[Vladimir Prus]

On Saturday 10 May 2008 06:54:01 you wrote:
>  >  > > Clearly I'm not going to do either but we could simply go back to using
>  >  > > the linked list structures that were already in varobj.c.  It's a
>  >  > > question of whether the convenience outweighs the handicap of having to
>  >  > > work with vectors all the time or not.  IMHO it doesn't.
>  >  > 
>  >  > I disagree, and I haven't yet seen a practical "handicap". We're not going
>  >  > back to hand-written data structures for MI implementation.
>  > 
>  > Well it's a shame that convenience should dictate the type of structures that
>  > are used because I can get this patch to work with an old Gdb using linked
>  > lists.
> 
> As you've made your decision, the patch below isn't a RFA or RFC but a record
> for the archives to ensure it's not lost like other patches that I've left
> lying around.  I feel sure that one day the issue will arise again, probably
> when Eclipse finds a requirement for it.
> 
> It's against current Gdb and uses the old linked lists.  This means that the
> size of the data structure is only proportional to the number of children
> created and not the total number of possible children.  Unlike the previous
> patch, children are listed in sequence for both creation and update.  Also
> children can be deleted.  Searching for a location to insert a child in the
> list is linear (as before) but I anticipate that a front end would only ever
> create a small number at any one time and delete children (that are not
> visible) if the number became too great.  In any case the search could be made
> binary (O(log N)), if necessary.
> 
> Oh, yes, and it's a unified diff!

I think that the current data structures surely allow to implement the behaviour
that is useful for frontends. I raise some questions about the original version
of your patch in

	http://article.gmane.org/gmane.comp.gdb.patches/40676

which you did not respond to. Did you miss that email? Do you plan to have the
original version of your patch adjusted and checked in?

For the record, the usecases that I think will benefit from this new functionality
are outlined in:

	http://article.gmane.org/gmane.comp.gdb.patches/40726


Thanks,
Volodya

Re: [PATCH:MI] Return a subset of a variable object's children

[Nick Roberts]

 > I think that the current data structures surely allow to implement the
 > behaviour that is useful for frontends. I raise some questions about the
 > original version of your patch in
 > 
 > 	http://article.gmane.org/gmane.comp.gdb.patches/40676
 > 
 > which you did not respond to. Did you miss that email? 

I think these question were addressed in subsequent e-mails.

1) I now post patches in unified diff format.

2) I explained why I thought the stride was useful, e.g., 
   (http://sourceware.org/ml/gdb-patches/2008-04/msg00680.html)

3) I acknowledged that the children should be reported in order 
   (http://sourceware.org/ml/gdb-patches/2008-04/msg00675.html)
   and my last patch did this using linked lists.

 >                                                         Do you plan to have
 > the original version of your patch adjusted and checked in?

No.  I couldn't get it to report the additional children/update in the correct
order using vectors.
 
 > For the record, the usecases that I think will benefit from this new
 > functionality are outlined in:
 > 
 > 	http://article.gmane.org/gmane.comp.gdb.patches/40726

I think the use cases are more extensive but I don't currently have any
user feedback to back up that assertion.

-- 
Nick                                           http://www.inet.net.nz/~nickrob