[rfa/dwarf] Support for attributes pointing to a different CU

[rfa/dwarf] Support for attributes pointing to a different CU

[Daniel Jacobowitz]

This is the last logical piece I could break out of the intercu support. 
We're almost there.  Only things left after this are dependence tracking
and the queueing of multiple full CUs - 400 lines or so.

The core change of this patch is in follow_die_ref.  Instead of taking an
offset and returning a DIE, it now takes an attribute and the CU in which
the attribute was found, and returns a DIE and the CU in which the DIE
was found.

The bulky text change of this patch is in dwarf2_attr.  It divides into two
interfaces: anywhere that we are prepared to find a reference to another
DIE, we need to call dwarf2_attr_with_cu instead.  Then, any further
operations on the returned reference need to be sure to use the returned CU.
The reason for this is a little subtle - we're returning a reference.  If
the reference is, for example, DW_FORM_ref4, then it's an offset within some
particular CU.  If the attribute containing this reference was found by
chasing DW_AT_specification, and DW_AT_specification was DW_FORM_ref_addr,
then the return value of dwarf2_attr_with_cu is a relative offset to a
different CU than the caller passed in.  So if we are going to follow the
reference, we need to know the compilation unit in which it resides.

A lot of places call dwarf2_attr_with_cu and ignore the result.  That's
because they don't keep the result (just compare it against NULL), or
because they don't actually cope with the case of a reference (variable
array bounds, for instance).  The error() there will probably turn up a few
more cases that need to accept this, but I wanted this to be a noisy
failure.

OK?

-- 
Daniel Jacobowitz

2004-09-23  Daniel Jacobowitz  <dan@debian.org>

	* dwarf2read.c (dwarf2_attr_with_cu): Renamed from dwarf2_attr.
	Add SPEC_CU argument and set it.  Update call to follow_die_ref.
	(dwarf2_attr): New function.
	(die_specification): Add SPEC_CU argument.  Update call to
	follow_die_ref.
	(dwarf2_extension): Likewise.
	(follow_die_ref): Take more abstract arguments.  Issue an error
	for DW_FORM_ref_addr.
	(read_func_scope): Update call to die_specification.  Pass SPEC_CU
	to determine_prefix.
	(determine_class_name): Likewise.
	(dwarf2_add_member_fn): Use dwarf2_attr_with_cu.
	(read_namespace, namespace_name): Update calls to dwarf2_extension.
	Use SPEC_CU.
	(read_subrange_type): Use dwarf2_attr_with_cu.
	(die_is_declaration, new_symbol): Likewise.
	(die_type, die_containing_type): Likewise.  Update call to
	follow_die_ref.

Index: dwarf2read.c
===================================================================
RCS file: /cvs/src/src/gdb/dwarf2read.c,v
retrieving revision 1.163
diff -u -p -r1.163 dwarf2read.c
--- dwarf2read.c	21 Sep 2004 15:04:41 -0000	1.163
+++ dwarf2read.c	23 Sep 2004 04:40:29 -0000
@@ -807,13 +807,19 @@ static void set_cu_language (unsigned in
 static struct attribute *dwarf2_attr (struct die_info *, unsigned int,
 				      struct dwarf2_cu *);
 
+static struct attribute *dwarf2_attr_with_cu (struct die_info *,
+					      unsigned int,
+					      struct dwarf2_cu *,
+					      struct dwarf2_cu **);
+
 static int dwarf2_flag_true_p (struct die_info *die, unsigned name,
                                struct dwarf2_cu *cu);
 
 static int die_is_declaration (struct die_info *, struct dwarf2_cu *cu);
 
 static struct die_info *die_specification (struct die_info *die,
-					   struct dwarf2_cu *);
+					   struct dwarf2_cu *,
+					   struct dwarf2_cu **);
 
 static void free_line_header (struct line_header *lh);
 
@@ -953,7 +959,8 @@ static char *dwarf2_linkage_name (struct
 static char *dwarf2_name (struct die_info *die, struct dwarf2_cu *);
 
 static struct die_info *dwarf2_extension (struct die_info *die,
-					  struct dwarf2_cu *);
+					  struct dwarf2_cu *,
+					  struct dwarf2_cu **);
 
 static char *dwarf_tag_name (unsigned int);
 
@@ -988,7 +995,9 @@ static unsigned int dwarf2_get_ref_die_o
 
 static int dwarf2_get_attr_constant_value (struct attribute *, int);
 
-static struct die_info *follow_die_ref (unsigned int);
+static struct die_info *follow_die_ref (struct attribute *,
+					struct dwarf2_cu *,
+					struct dwarf2_cu **);
 
 static struct type *dwarf2_fundamental_type (struct objfile *, int,
 					     struct dwarf2_cu *);
@@ -2723,7 +2732,8 @@ read_func_scope (struct die_info *die, s
   if (cu->language == language_cplus
       || cu->language == language_java)
     {
-      struct die_info *spec_die = die_specification (die, cu);
+      struct dwarf2_cu *spec_cu;
+      struct die_info *spec_die = die_specification (die, cu, &spec_cu);
 
       /* NOTE: carlton/2004-01-23: We have to be careful in the
          presence of DW_AT_specification.  For example, with GCC 3.4,
@@ -2749,7 +2759,7 @@ read_func_scope (struct die_info *die, s
 	
       if (spec_die != NULL)
 	{
-	  char *specification_prefix = determine_prefix (spec_die, cu);
+	  char *specification_prefix = determine_prefix (spec_die, spec_cu);
 	  processing_current_prefix = specification_prefix;
 	  back_to = make_cleanup (xfree, specification_prefix);
 	}
@@ -3351,6 +3361,7 @@ dwarf2_add_member_fn (struct field_info 
   char *fieldname;
   char *physname;
   struct nextfnfield *new_fnfield;
+  struct dwarf2_cu *spec_cu;
 
   /* Get name of member function.  */
   attr = dwarf2_attr (die, DW_AT_name, cu);
@@ -3431,7 +3442,7 @@ dwarf2_add_member_fn (struct field_info 
 	       physname);
 
   /* Get fcontext from DW_AT_containing_type if present.  */
-  if (dwarf2_attr (die, DW_AT_containing_type, cu) != NULL)
+  if (dwarf2_attr_with_cu (die, DW_AT_containing_type, cu, &spec_cu) != NULL)
     fnp->fcontext = die_containing_type (die, cu);
 
   /* dwarf2 doesn't have stubbed physical names, so the setting of is_const
@@ -3658,13 +3669,16 @@ read_structure_type (struct die_info *di
 	dwarf2_attach_fields_to_type (&fi, type, cu);
       if (fi.nfnfields)
 	{
+	  struct dwarf2_cu *spec_cu;
+
 	  dwarf2_attach_fn_fields_to_type (&fi, type, cu);
 
 	  /* Get the type which refers to the base class (possibly this
 	     class itself) which contains the vtable pointer for the current
 	     class from the DW_AT_containing_type attribute.  */
 
-	  if (dwarf2_attr (die, DW_AT_containing_type, cu) != NULL)
+	  if (dwarf2_attr_with_cu (die, DW_AT_containing_type,
+				   cu, &spec_cu) != NULL)
 	    {
 	      struct type *t = die_containing_type (die, cu);
 
@@ -3802,7 +3816,8 @@ static char *
 determine_class_name (struct die_info *die, struct dwarf2_cu *cu)
 {
   struct cleanup *back_to = NULL;
-  struct die_info *spec_die = die_specification (die, cu);
+  struct dwarf2_cu *spec_cu;
+  struct die_info *spec_die = die_specification (die, cu, &spec_cu);
   char *new_prefix = NULL;
 
   /* If this is the definition of a class that is declared by another
@@ -3810,7 +3825,7 @@ determine_class_name (struct die_info *d
      read_func_scope for a similar example.  */
   if (spec_die != NULL)
     {
-      char *specification_prefix = determine_prefix (spec_die, cu);
+      char *specification_prefix = determine_prefix (spec_die, spec_cu);
       processing_current_prefix = specification_prefix;
       back_to = make_cleanup (xfree, specification_prefix);
     }
@@ -4113,6 +4128,7 @@ read_namespace (struct die_info *die, st
   int is_anonymous;
   struct die_info *current_die;
   struct cleanup *back_to = make_cleanup (null_cleanup, 0);
+  struct dwarf2_cu *spec_cu;
 
   name = namespace_name (die, &is_anonymous, cu);
 
@@ -4133,7 +4149,7 @@ read_namespace (struct die_info *die, st
      before.  Also, add a using directive if it's an anonymous
      namespace.  */
 
-  if (dwarf2_extension (die, cu) == NULL)
+  if (dwarf2_extension (die, cu, &spec_cu) == NULL)
     {
       struct type *type;
 
@@ -4177,14 +4193,15 @@ namespace_name (struct die_info *die, in
 {
   struct die_info *current_die;
   const char *name = NULL;
+  struct dwarf2_cu *spec_cu;
 
   /* Loop through the extensions until we find a name.  */
 
-  for (current_die = die;
+  for (current_die = die, spec_cu = cu;
        current_die != NULL;
-       current_die = dwarf2_extension (die, cu))
+       current_die = dwarf2_extension (die, spec_cu, &spec_cu))
     {
-      name = dwarf2_name (current_die, cu);
+      name = dwarf2_name (current_die, spec_cu);
       if (name != NULL)
 	break;
     }
@@ -4588,6 +4605,7 @@ read_subrange_type (struct die_info *die
   struct attribute *attr;
   int low = 0;
   int high = -1;
+  struct dwarf2_cu *spec_cu;
   
   /* If we have already decoded this die, then nothing more to do.  */
   if (die->type)
@@ -4610,11 +4628,14 @@ read_subrange_type (struct die_info *die
       low = 1;
     }
 
-  attr = dwarf2_attr (die, DW_AT_lower_bound, cu);
+  /* FIXME: For variable sized arrays either of these could be
+     a variable rather than a constant value.  We'll allow it,
+     but we don't know how to handle it.  */
+  attr = dwarf2_attr_with_cu (die, DW_AT_lower_bound, cu, &spec_cu);
   if (attr)
     low = dwarf2_get_attr_constant_value (attr, 0);
 
-  attr = dwarf2_attr (die, DW_AT_upper_bound, cu);
+  attr = dwarf2_attr_with_cu (die, DW_AT_upper_bound, cu, &spec_cu);
   if (attr)
     {       
       if (attr->form == DW_FORM_block1)
@@ -5946,7 +5967,8 @@ set_cu_language (unsigned int lang, stru
 /* Return the named attribute or NULL if not there.  */
 
 static struct attribute *
-dwarf2_attr (struct die_info *die, unsigned int name, struct dwarf2_cu *cu)
+dwarf2_attr_with_cu (struct die_info *die, unsigned int name,
+		     struct dwarf2_cu *cu, struct dwarf2_cu **spec_cu)
 {
   unsigned int i;
   struct attribute *spec = NULL;
@@ -5955,6 +5977,26 @@ dwarf2_attr (struct die_info *die, unsig
     {
       if (die->attrs[i].name == name)
 	{
+	  /* If our caller did not expect a reference, but got one,
+	     we can't continue.  We won't be able to understand the
+	     data, and if the reference was to a different compilation
+	     unit we'd get garbage following any references from the
+	     returned DIE.  */
+	  if (spec_cu == NULL
+	      && (die->attrs[i].form == DW_FORM_ref_addr
+		  || die->attrs[i].form == DW_FORM_ref1
+		  || die->attrs[i].form == DW_FORM_ref2
+		  || die->attrs[i].form == DW_FORM_ref4
+		  || die->attrs[i].form == DW_FORM_ref8
+		  || die->attrs[i].form == DW_FORM_ref_udata))
+	    error ("Dwarf Error: attempt to follow a reference "
+		   "from DIE at 0x%lx [in module %s] discards "
+		   "compilation unit", (long) die->offset,
+		   bfd_get_filename (cu->objfile->obfd));
+
+	  if (spec_cu != NULL)
+	    *spec_cu = cu;
+
 	  return &die->attrs[i];
 	}
       if (die->attrs[i].name == DW_AT_specification
@@ -5963,16 +6005,23 @@ dwarf2_attr (struct die_info *die, unsig
     }
   if (spec)
     {
-      struct die_info *ref_die =
-      follow_die_ref (dwarf2_get_ref_die_offset (spec, cu));
+      struct dwarf2_cu *ref_cu;
+      struct die_info *ref_die = follow_die_ref (spec, cu, &ref_cu);
 
-      if (ref_die)
-	return dwarf2_attr (ref_die, name, cu);
+      return dwarf2_attr_with_cu (ref_die, name, ref_cu, spec_cu);
     }
 
   return NULL;
 }
 
+/* Return the named attribute or NULL if not there.  */
+
+static struct attribute *
+dwarf2_attr (struct die_info *die, unsigned int name, struct dwarf2_cu *cu)
+{
+  return dwarf2_attr_with_cu (die, name, cu, NULL);
+}
+
 /* Return non-zero iff the attribute NAME is defined for the given DIE,
    and holds a non-zero value.  This function should only be used for
    DW_FORM_flag attributes.  */
@@ -5988,6 +6037,8 @@ dwarf2_flag_true_p (struct die_info *die
 static int
 die_is_declaration (struct die_info *die, struct dwarf2_cu *cu)
 {
+  struct dwarf2_cu *spec_cu;
+
   /* A DIE is a declaration if it has a DW_AT_declaration attribute
      which value is non-zero.  However, we have to be careful with
      DIEs having a DW_AT_specification attribute, because dwarf2_attr()
@@ -5996,21 +6047,23 @@ die_is_declaration (struct die_info *die
      to a different DIE referenced by the specification attribute,
      even though the given DIE does not have a declaration attribute.  */
   return (dwarf2_flag_true_p (die, DW_AT_declaration, cu)
-	  && dwarf2_attr (die, DW_AT_specification, cu) == NULL);
+	  && ! dwarf2_attr_with_cu (die, DW_AT_specification, cu, &spec_cu));
 }
 
 /* Return the die giving the specification for DIE, if there is
    one.  */
 
 static struct die_info *
-die_specification (struct die_info *die, struct dwarf2_cu *cu)
+die_specification (struct die_info *die, struct dwarf2_cu *cu,
+		   struct dwarf2_cu **spec_cu)
 {
-  struct attribute *spec_attr = dwarf2_attr (die, DW_AT_specification, cu);
+  struct attribute *spec_attr
+    = dwarf2_attr_with_cu (die, DW_AT_specification, cu, spec_cu);
 
   if (spec_attr == NULL)
     return NULL;
   else
-    return follow_die_ref (dwarf2_get_ref_die_offset (spec_attr, cu));
+    return follow_die_ref (spec_attr, *spec_cu, spec_cu);
 }
 
 /* Free the line_header structure *LH, and any arrays and strings it
@@ -6664,6 +6717,8 @@ new_symbol (struct die_info *die, struct
 	    }
 	  else
 	    {
+	      struct dwarf2_cu *spec_cu;
+
 	      /* We do not know the address of this symbol.
 	         If it is an external symbol and we have type information
 	         for it, enter the symbol as a LOC_UNRESOLVED symbol.
@@ -6672,7 +6727,8 @@ new_symbol (struct die_info *die, struct
 	         referenced.  */
 	      attr2 = dwarf2_attr (die, DW_AT_external, cu);
 	      if (attr2 && (DW_UNSND (attr2) != 0)
-		  && dwarf2_attr (die, DW_AT_type, cu) != NULL)
+		  && dwarf2_attr_with_cu (die, DW_AT_type,
+					  cu, &spec_cu) != NULL)
 		{
 		  SYMBOL_CLASS (sym) = LOC_UNRESOLVED;
 		  add_symbol_to_list (sym, &global_symbols);
@@ -6941,9 +6997,9 @@ die_type (struct die_info *die, struct d
   struct type *type;
   struct attribute *type_attr;
   struct die_info *type_die;
-  unsigned int ref;
+  struct dwarf2_cu *spec_cu;
 
-  type_attr = dwarf2_attr (die, DW_AT_type, cu);
+  type_attr = dwarf2_attr_with_cu (die, DW_AT_type, cu, &spec_cu);
   if (!type_attr)
     {
       /* A missing DW_AT_type represents a void type.  */
@@ -6951,16 +7007,16 @@ die_type (struct die_info *die, struct d
     }
   else
     {
-      ref = dwarf2_get_ref_die_offset (type_attr, cu);
-      type_die = follow_die_ref (ref);
+      type_die = follow_die_ref (type_attr, spec_cu, &spec_cu);
       if (!type_die)
 	{
-	  error ("Dwarf Error: Cannot find referent at offset %d [in module %s]", 
-			  ref, cu->objfile->name);
+	  error ("Dwarf Error: Cannot find referent from DIE "
+		 "at offset %d [in module %s]",
+		 die->offset, cu->objfile->name);
 	  return NULL;
 	}
     }
-  type = tag_type_to_type (type_die, cu);
+  type = tag_type_to_type (type_die, spec_cu);
   if (!type)
     {
       dump_die (type_die);
@@ -6979,20 +7035,19 @@ die_containing_type (struct die_info *di
   struct type *type = NULL;
   struct attribute *type_attr;
   struct die_info *type_die = NULL;
-  unsigned int ref;
+  struct dwarf2_cu *spec_cu;
 
-  type_attr = dwarf2_attr (die, DW_AT_containing_type, cu);
+  type_attr = dwarf2_attr_with_cu (die, DW_AT_containing_type, cu, &spec_cu);
   if (type_attr)
     {
-      ref = dwarf2_get_ref_die_offset (type_attr, cu);
-      type_die = follow_die_ref (ref);
+      type_die = follow_die_ref (type_attr, spec_cu, &spec_cu);
       if (!type_die)
 	{
-	  error ("Dwarf Error: Cannot find referent at offset %d [in module %s]", ref, 
-			  cu->objfile->name);
+	  error ("Dwarf Error: Cannot find referent from DIE at offset %d "
+		 "[in module %s]", die->offset, cu->objfile->name);
 	  return NULL;
 	}
-      type = tag_type_to_type (type_die, cu);
+      type = tag_type_to_type (type_die, spec_cu);
     }
   if (!type)
     {
@@ -7372,21 +7427,21 @@ dwarf2_name (struct die_info *die, struc
    is none.  */
 
 static struct die_info *
-dwarf2_extension (struct die_info *die, struct dwarf2_cu *cu)
+dwarf2_extension (struct die_info *die, struct dwarf2_cu *cu,
+		  struct dwarf2_cu **spec_cu)
 {
   struct attribute *attr;
   struct die_info *extension_die;
-  unsigned int ref;
 
-  attr = dwarf2_attr (die, DW_AT_extension, cu);
+  attr = dwarf2_attr_with_cu (die, DW_AT_extension, cu, spec_cu);
   if (attr == NULL)
     return NULL;
 
-  ref = dwarf2_get_ref_die_offset (attr, cu);
-  extension_die = follow_die_ref (ref);
+  extension_die = follow_die_ref (attr, *spec_cu, spec_cu);
   if (!extension_die)
     {
-      error ("Dwarf Error: Cannot find referent at offset %d.", ref);
+      error ("Dwarf Error: Cannot find referent from DIE at offset %d.",
+	     die->offset);
     }
 
   return extension_die;
@@ -8342,19 +8397,31 @@ dwarf2_get_attr_constant_value (struct a
 }
 
 static struct die_info *
-follow_die_ref (unsigned int offset)
+follow_die_ref (struct attribute *attr, struct dwarf2_cu *cu,
+		struct dwarf2_cu **spec_cu)
 {
   struct die_info *die;
+  unsigned int offset;
   int h;
+  struct die_info temp_die;
+
+  offset = dwarf2_get_ref_die_offset (attr, cu);
+
+  if (attr->form == DW_FORM_ref_addr
+      && (DW_ADDR (attr) < cu->header.offset
+	  || DW_ADDR (attr) >= cu->header.offset + cu->header.length))
+    {
+      error ("Dwarf Error: unsupported inter-compilation-unit reference");
+    }
+  else
+    *spec_cu = cu;
 
   h = (offset % REF_HASH_SIZE);
   die = die_ref_table[h];
   while (die)
     {
       if (die->offset == offset)
-	{
-	  return die;
-	}
+	return die;
       die = die->next_ref;
     }
   return NULL;

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

Daniel Jacobowitz <drow@false.org> writes:
> This is the last logical piece I could break out of the intercu support. 
> We're almost there.  Only things left after this are dependence tracking
> and the queueing of multiple full CUs - 400 lines or so.
> 
> The core change of this patch is in follow_die_ref.  Instead of taking an
> offset and returning a DIE, it now takes an attribute and the CU in which
> the attribute was found, and returns a DIE and the CU in which the DIE
> was found.
> 
> The bulky text change of this patch is in dwarf2_attr.  It divides into two
> interfaces: anywhere that we are prepared to find a reference to another
> DIE, we need to call dwarf2_attr_with_cu instead.  Then, any further
> operations on the returned reference need to be sure to use the returned CU.
> The reason for this is a little subtle - we're returning a reference.  If
> the reference is, for example, DW_FORM_ref4, then it's an offset within some
> particular CU.  If the attribute containing this reference was found by
> chasing DW_AT_specification, and DW_AT_specification was DW_FORM_ref_addr,
> then the return value of dwarf2_attr_with_cu is a relative offset to a
> different CU than the caller passed in.  So if we are going to follow the
> reference, we need to know the compilation unit in which it resides.

I may be missing something, but it seems to me some of this effort is
necessary because we hold some die references as CU-relative, and
others as .debug_info relative, depending on the form the attribute
happened to use.  Wouldn't it be a simplification to have
read_attribute_value always store all DW_FORM_ref* attributes as
DW_ADDR offsets from the beginning of .debug_info?  Then dwarf2_attr's
interface could remain unchanged, and we could drop all the spec_cu
variables carrying the CU to the next call to
dwarf2_get_ref_die_offset.  Then, dwarf2_get_ref_die_offset could be
deleted altogether, and its calls replaced with uses of DW_ADDR.

(This change wasn't possible in the past because we didn't pass a CU
to dwarf2_attr.)

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Daniel Jacobowitz]

On Thu, Sep 23, 2004 at 05:17:54PM -0500, Jim Blandy wrote:
> 
> Daniel Jacobowitz <drow@false.org> writes:
> > This is the last logical piece I could break out of the intercu support. 
> > We're almost there.  Only things left after this are dependence tracking
> > and the queueing of multiple full CUs - 400 lines or so.
> > 
> > The core change of this patch is in follow_die_ref.  Instead of taking an
> > offset and returning a DIE, it now takes an attribute and the CU in which
> > the attribute was found, and returns a DIE and the CU in which the DIE
> > was found.
> > 
> > The bulky text change of this patch is in dwarf2_attr.  It divides into two
> > interfaces: anywhere that we are prepared to find a reference to another
> > DIE, we need to call dwarf2_attr_with_cu instead.  Then, any further
> > operations on the returned reference need to be sure to use the returned CU.
> > The reason for this is a little subtle - we're returning a reference.  If
> > the reference is, for example, DW_FORM_ref4, then it's an offset within some
> > particular CU.  If the attribute containing this reference was found by
> > chasing DW_AT_specification, and DW_AT_specification was DW_FORM_ref_addr,
> > then the return value of dwarf2_attr_with_cu is a relative offset to a
> > different CU than the caller passed in.  So if we are going to follow the
> > reference, we need to know the compilation unit in which it resides.
> 
> I may be missing something, but it seems to me some of this effort is
> necessary because we hold some die references as CU-relative, and
> others as .debug_info relative, depending on the form the attribute
> happened to use.  Wouldn't it be a simplification to have
> read_attribute_value always store all DW_FORM_ref* attributes as
> DW_ADDR offsets from the beginning of .debug_info?  Then dwarf2_attr's
> interface could remain unchanged, and we could drop all the spec_cu
> variables carrying the CU to the next call to
> dwarf2_get_ref_die_offset.  Then, dwarf2_get_ref_die_offset could be
> deleted altogether, and its calls replaced with uses of DW_ADDR.
> 
> (This change wasn't possible in the past because we didn't pass a CU
> to dwarf2_attr.)

This is why I like second-person review :-)  That's a brilliant idea. 
I will try it.

-- 
Daniel Jacobowitz

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Daniel Jacobowitz]

On Thu, Sep 23, 2004 at 05:17:54PM -0500, Jim Blandy wrote:
> 
> Daniel Jacobowitz <drow@false.org> writes:
> > This is the last logical piece I could break out of the intercu support. 
> > We're almost there.  Only things left after this are dependence tracking
> > and the queueing of multiple full CUs - 400 lines or so.
> > 
> > The core change of this patch is in follow_die_ref.  Instead of taking an
> > offset and returning a DIE, it now takes an attribute and the CU in which
> > the attribute was found, and returns a DIE and the CU in which the DIE
> > was found.
> > 
> > The bulky text change of this patch is in dwarf2_attr.  It divides into two
> > interfaces: anywhere that we are prepared to find a reference to another
> > DIE, we need to call dwarf2_attr_with_cu instead.  Then, any further
> > operations on the returned reference need to be sure to use the returned CU.
> > The reason for this is a little subtle - we're returning a reference.  If
> > the reference is, for example, DW_FORM_ref4, then it's an offset within some
> > particular CU.  If the attribute containing this reference was found by
> > chasing DW_AT_specification, and DW_AT_specification was DW_FORM_ref_addr,
> > then the return value of dwarf2_attr_with_cu is a relative offset to a
> > different CU than the caller passed in.  So if we are going to follow the
> > reference, we need to know the compilation unit in which it resides.
> 
> I may be missing something, but it seems to me some of this effort is
> necessary because we hold some die references as CU-relative, and
> others as .debug_info relative, depending on the form the attribute
> happened to use.  Wouldn't it be a simplification to have
> read_attribute_value always store all DW_FORM_ref* attributes as
> DW_ADDR offsets from the beginning of .debug_info?  Then dwarf2_attr's
> interface could remain unchanged, and we could drop all the spec_cu
> variables carrying the CU to the next call to
> dwarf2_get_ref_die_offset.  Then, dwarf2_get_ref_die_offset could be
> deleted altogether, and its calls replaced with uses of DW_ADDR.
> 
> (This change wasn't possible in the past because we didn't pass a CU
> to dwarf2_attr.)

Actually, it doesn't matter - dwarf2_attr doesn't need the CU for this
anyway.  This suggestion killed basically all of the previous patch;
there's not enough to bother breaking apart, so I'm including it with
the final bits.  Here they are.

The die_ref_table moves from global into the dwarf2_cu structure.

I apply your suggestion to partial DIE following, resulting in the
deletion of a bunch of one of my previous patches.  I also apply it to
full DIE reference following.  This results in a certain amount of
shuffling of deck chairs.

There's a little bit of new code to make sure that, during full symbol
reading, we can get at the psymtab from the per_cu structure.

psymtab_to_symtab_1 is broken into two halves, to load the DIEs and to
generate symbols from them.  If we're in inter-compilation-unit mode,
we read all the DIEs we'll need before we process any of them.  New CUs
are loaded by adding them to the queue in read_full_die when we
encounter a reference pointing at them.  When this happens, we also
record the dependency of the first CU on the second, so that it is not
flushed from cache before the CU that references it - we need to do
this so that references to other CUs don't trigger loading of the CU at
an unexpected time, breaking the load-then-process invariant.

The breaking up into two halves also results in a certain amount of
deck shuffling, because cu becomes a pointer instead of a stack
variable.  The only interesting bit in what used to be
psymtab_to_symtab_1 is the hunk at old line 2372 in the diff.

I've tried to make it as clear as possible.  If you see anything you'd
prefer as a separate patch, let me know.

OK?

-- 
Daniel Jacobowitz

2004-09-23  Daniel Jacobowitz  <dan@debian.org>

	* dwarf2read.c (REF_HASH_SIZE): Move earlier.
	(die_ref_table): Remove.
	(struct dwarf2_cu): Add DIES, DEPENDENCIES, and DIE_REF_TABLE.
	(struct dwarf2_per_cu_data): Add PSYMTAB.
	(struct dwarf2_queue_item, dwarf2_queue): New.
	(find_partial_die): Remove third argument.
	(type_at_offset): Remove dead code.
	(make_cleanup_free_die_list, dwarf2_empty_hash_tables): Remove.
	(store_in_ref_table): Add CU argument.
	(follow_die_ref): Take DIE, attribute, and CU arguments.  Handle
	inter-compilation-unit references.
	(dwarf2_find_comp_unit_psymtab): New function.
	(load_full_comp_unit, process_full_comp_unit): New functions, based
	on psymtab_to_symtab_1.
	(psymtab_to_symtab_1): Use them.
	(dwarf2_add_dependence): New function.
	(dwarf2_build_psymtabs_hard): Set the psymtab in per_cu.
	(partial_die_parent_scope, guess_structure_name): Update for changes
	to find_partial_die.
	(dwarf2_psymtab_to_symtab): Initialize dwarf2_per_objfile here.
	(queue_comp_unit, process_queue, dwarf2_release_queue): New.
	(read_subrange_type): Add a comment.
	(read_comp_unit): Don't call dwarf2_empty_hash_tables.
	(read_die_and_children): Update call to store_in_ref_table.
	(do_free_die_list_cleanup): Remove.
	(fixup_partial_die): Update for changes to find_partial_die.
	(read_full_die): Handle queueing absolute references.
	(read_attribute_value): Use DW_ADDR for all DW_FORM_ref* forms.
	(dwarf2_attr, die_specification, die_type)
	(die_containing_type, dwarf2_extension): Update calls to
	follow_die_ref.
	(dump_die): Update DW_FORM_ref* handling.
	(dwarf2_get_ref_die_offset): Likewise.
	(free_one_comp_unit): Release the dies list.
	(dwarf2_mark_helper): New function.
	(dwarf2_mark): Use it.

--- src/gdb/dwarf2read.c	2004-09-23 18:58:04.000000000 -0400
+++ src/gdb/dwarf2read.c	2004-09-23 20:25:21.000000000 -0400
@@ -258,6 +258,11 @@ struct comp_unit_head
     int base_known;
   };
 
+/* Fixed size for the DIE hash table.  */
+#ifndef REF_HASH_SIZE
+#define REF_HASH_SIZE 1021
+#endif
+
 /* Internal state when decoding a particular compilation unit.  */
 struct dwarf2_cu
 {
@@ -325,6 +330,17 @@ struct dwarf2_cu
   /* How many compilation units ago was this CU last referenced?  */
   int last_used;
 
+  /* A hash table of die offsets for following references.  */
+  struct die_info *die_ref_table[REF_HASH_SIZE];
+
+  /* Full DIEs if read in.  */
+  struct die_info *dies;
+
+  /* A set of pointers to dwarf2_per_cu_data objects for compilation
+     units referenced by this one.  Only set during full symbol processing;
+     partial symbol tables do not have dependencies.  */
+  htab_t dependencies;
+
   /* Mark used when releasing cached dies.  */
   unsigned int mark : 1;
 
@@ -360,6 +376,11 @@ struct dwarf2_per_cu_data
      it.  */
 
   htab_t type_hash;
+
+  /* The partial symbol table associated with this compilation unit.  This
+     is set opportunistically during partial symbol reading, and guaranteed
+     initialized before full DIEs are read in.  */
+  struct partial_symtab *psymtab;
 };
 
 /* The line number information for a compilation unit (found in the
@@ -551,13 +572,6 @@ struct dwarf_block
 #define ATTR_ALLOC_CHUNK 4
 #endif
 
-/* A hash table of die offsets for following references.  */
-#ifndef REF_HASH_SIZE
-#define REF_HASH_SIZE 1021
-#endif
-
-static struct die_info *die_ref_table[REF_HASH_SIZE];
-
 /* Allocate fields for structs, unions and enums in this size.  */
 #ifndef DW_FIELD_ALLOC_CHUNK
 #define DW_FIELD_ALLOC_CHUNK 4
@@ -641,6 +655,17 @@ struct field_info
     int nfnfields;
   };
 
+/* One item on the queue of compilation units to read in full symbols
+   for.  */
+struct dwarf2_queue_item
+{
+  struct dwarf2_per_cu_data *per_cu;
+  struct dwarf2_queue_item *next;
+};
+
+/* The current queue.  */
+static struct dwarf2_queue_item *dwarf2_queue;
+
 /* Loaded secondary compilation units are kept in memory until they
    have not been referenced for the processing of this many
    compilation units.  Set this to zero to disable caching.  Cache
@@ -755,8 +780,7 @@ static char *read_partial_die (struct pa
 			       bfd *, char *, struct dwarf2_cu *);
 
 static struct partial_die_info *find_partial_die (unsigned long,
-						  struct dwarf2_cu *,
-						  struct dwarf2_cu **);
+						  struct dwarf2_cu *);
 
 static void fixup_partial_die (struct partial_die_info *,
 			       struct dwarf2_cu *);
@@ -844,10 +868,6 @@ static struct type *die_type (struct die
 static struct type *die_containing_type (struct die_info *,
 					 struct dwarf2_cu *);
 
-#if 0
-static struct type *type_at_offset (unsigned int, struct objfile *);
-#endif
-
 static struct type *tag_type_to_type (struct die_info *, struct dwarf2_cu *);
 
 static void read_type_die (struct die_info *, struct dwarf2_cu *);
@@ -944,8 +964,6 @@ static struct die_info *read_die_and_sib
 
 static void free_die_list (struct die_info *);
 
-static struct cleanup *make_cleanup_free_die_list (struct die_info *);
-
 static void process_die (struct die_info *, struct dwarf2_cu *);
 
 static char *dwarf2_linkage_name (struct die_info *, struct dwarf2_cu *);
@@ -979,16 +997,17 @@ static void dump_die (struct die_info *)
 
 static void dump_die_list (struct die_info *);
 
-static void store_in_ref_table (unsigned int, struct die_info *);
-
-static void dwarf2_empty_hash_tables (void);
+static void store_in_ref_table (unsigned int, struct die_info *,
+				struct dwarf2_cu *);
 
 static unsigned int dwarf2_get_ref_die_offset (struct attribute *,
 					       struct dwarf2_cu *);
 
 static int dwarf2_get_attr_constant_value (struct attribute *, int);
 
-static struct die_info *follow_die_ref (unsigned int);
+static struct die_info *follow_die_ref (struct die_info *,
+					struct attribute *,
+					struct dwarf2_cu *);
 
 static struct type *dwarf2_fundamental_type (struct objfile *, int,
 					     struct dwarf2_cu *);
@@ -1034,6 +1053,9 @@ static struct dwarf2_per_cu_data *dwarf2
 static struct dwarf2_per_cu_data *dwarf2_find_comp_unit
   (unsigned long offset, struct objfile *objfile);
 
+static struct partial_symtab *dwarf2_find_comp_unit_psymtab
+  (unsigned int offset, struct objfile *objfile);
+
 static void free_one_comp_unit (void *);
 
 static void free_cached_comp_units (void *);
@@ -1045,13 +1067,20 @@ static void free_one_cached_comp_unit (v
 static void set_die_type (struct die_info *, struct type *,
 			  struct dwarf2_cu *);
 
-#if 0
 static void reset_die_and_siblings_types (struct die_info *,
 					  struct dwarf2_cu *);
-#endif
 
 static void create_all_comp_units (struct objfile *);
 
+static struct dwarf2_cu *load_full_comp_unit (struct partial_symtab *,
+					      struct dwarf2_per_cu_data *);
+
+static void process_full_comp_unit (struct partial_symtab *,
+				    struct dwarf2_cu *);
+
+static void dwarf2_add_dependence (struct dwarf2_cu *,
+				   struct dwarf2_per_cu_data *);
+
 static void dwarf2_mark (struct dwarf2_cu *);
 
 static void dwarf2_clear_marks (struct dwarf2_per_cu_data *);
@@ -1486,6 +1515,8 @@ dwarf2_build_psymtabs_hard (struct objfi
 	     in free_stack_comp_unit when we finish with this
 	     compilation unit.  */
 	  per_cu->cu = &cu;
+
+	  per_cu->psymtab = pst;
 	}
       else
 	cu.per_cu = NULL;
@@ -1778,15 +1809,13 @@ partial_die_parent_scope (struct partial
 {
   char *grandparent_scope;
   struct partial_die_info *parent, *real_pdi;
-  struct dwarf2_cu *spec_cu;
 
   /* We need to look at our parent DIE; if we have a DW_AT_specification,
      then this means the parent of the specification DIE.  */
 
   real_pdi = pdi;
-  spec_cu = cu;
   while (real_pdi->has_specification)
-    real_pdi = find_partial_die (real_pdi->spec_offset, spec_cu, &spec_cu);
+    real_pdi = find_partial_die (real_pdi->spec_offset, cu);
 
   parent = real_pdi->die_parent;
   if (parent == NULL)
@@ -1797,7 +1826,7 @@ partial_die_parent_scope (struct partial
 
   fixup_partial_die (parent, cu);
 
-  grandparent_scope = partial_die_parent_scope (parent, spec_cu);
+  grandparent_scope = partial_die_parent_scope (parent, cu);
 
   if (parent->tag == DW_TAG_namespace
       || parent->tag == DW_TAG_structure_type
@@ -2071,16 +2100,14 @@ guess_structure_name (struct partial_die
 
       struct partial_die_info *child_pdi = struct_pdi->die_child;
       struct partial_die_info *real_pdi;
-      struct dwarf2_cu *spec_cu;
 
       /* If this DIE (this DIE's specification, if any) has a parent, then
 	 we should not do this.  We'll prepend the parent's fully qualified
          name when we create the partial symbol.  */
 
       real_pdi = struct_pdi;
-      spec_cu = cu;
       while (real_pdi->has_specification)
-	real_pdi = find_partial_die (real_pdi->spec_offset, spec_cu, &spec_cu);
+	real_pdi = find_partial_die (real_pdi->spec_offset, cu);
 
       if (real_pdi->die_parent != NULL)
 	return;
@@ -2321,6 +2348,10 @@ dwarf2_psymtab_to_symtab (struct partial
 	      gdb_flush (gdb_stdout);
 	    }
 
+	  /* Restore our global data.  */
+	  dwarf2_per_objfile = objfile_data (pst->objfile,
+					     dwarf2_objfile_data_key);
+
 	  psymtab_to_symtab_1 (pst);
 
 	  /* Finish up the debug error message.  */
@@ -2330,21 +2361,115 @@ dwarf2_psymtab_to_symtab (struct partial
     }
 }
 
+/* Add PER_CU to the queue.  */
+
+static void
+queue_comp_unit (struct dwarf2_per_cu_data *per_cu)
+{
+  struct dwarf2_queue_item *item;
+
+  per_cu->queued = 1;
+  item = xmalloc (sizeof (*item));
+  item->per_cu = per_cu;
+  item->next = dwarf2_queue;
+  dwarf2_queue = item;
+}
+
+/* Process the queue.  */
+
+static void
+process_queue (struct objfile *objfile)
+{
+  struct dwarf2_queue_item *item, *last;
+
+  /* Initially, there is just one item on the queue.  Load its DIEs,
+     and the DIEs of any other compilation units it requires,
+     transitively.  */
+
+  while (dwarf2_queue->per_cu->cu == NULL)
+    {
+      item = dwarf2_queue;
+      while (item && item->per_cu->cu == NULL)
+	{
+	  if (item->per_cu->psymtab == NULL)
+	    item->per_cu->psymtab
+	      = dwarf2_find_comp_unit_psymtab (item->per_cu->offset, objfile);
+	  load_full_comp_unit (item->per_cu->psymtab, item->per_cu);
+
+	  item->per_cu->cu->read_in_chain = dwarf2_per_objfile->read_in_chain;
+	  dwarf2_per_objfile->read_in_chain = item->per_cu;
+
+	  item = item->next;
+	}
+    }
+
+  /* We have now queued everything we will need, and loaded DIEs for
+     all queued CUs.  For any that have already been read in, reset
+     the TYPE field in each DIE.  */
+  item = dwarf2_queue;
+  while (item)
+    {
+      if (item->per_cu->psymtab->readin)
+	{
+	  reset_die_and_siblings_types (item->per_cu->cu->dies,
+					item->per_cu->cu);
+	  item->per_cu->queued = 0;
+	}
+      item = item->next;
+    }
+
+  /* Now everything left on the queue needs to be read in.  Process
+     them, one at a time, removing from the queue as we finish.  */
+  item = dwarf2_queue;
+  while (item)
+    {
+      if (!item->per_cu->psymtab->readin)
+	{
+	  process_full_comp_unit (item->per_cu->psymtab,
+				  item->per_cu->cu);
+	  item->per_cu->queued = 0;
+	}
+      last = item;
+      dwarf2_queue = item = last->next;
+      xfree (last);
+    }
+}
+
+/* Free all allocated queue entries.  This function only releases anything if
+   an error was thrown; if the queue was processed then it would have been
+   freed as we went along.  */
+
+static void
+dwarf2_release_queue (void *dummy)
+{
+  struct dwarf2_queue_item *item, *last;
+
+  item = dwarf2_queue;
+  while (item)
+    {
+      /* Anything still marked queued is likely to be in an
+	 inconsistent state, so discard it.  */
+      if (item->per_cu->queued)
+	{
+	  if (item->per_cu->cu != NULL)
+	    free_one_cached_comp_unit (item->per_cu->cu);
+	  item->per_cu->queued = 0;
+	}
+
+      last = item;
+      item = item->next;
+      xfree (last);
+    }
+
+  dwarf2_queue = NULL;
+}
+
+/* Read in full symbols for PST, and anything it depends on.  */
+
 static void
 psymtab_to_symtab_1 (struct partial_symtab *pst)
 {
-  struct objfile *objfile = pst->objfile;
-  bfd *abfd = objfile->obfd;
-  struct dwarf2_cu cu;
-  struct die_info *dies;
-  unsigned long offset;
-  CORE_ADDR lowpc, highpc;
-  struct die_info *child_die;
-  char *info_ptr;
-  struct symtab *symtab;
   struct cleanup *back_to;
-  struct attribute *attr;
-  CORE_ADDR baseaddr;
   int i;
 
   for (i = 0; i < pst->number_of_dependencies; i++)
@@ -2372,40 +2497,126 @@ psymtab_to_symtab_1 (struct partial_symt
       return;
     }
 
-  dwarf2_per_objfile = objfile_data (pst->objfile, dwarf2_objfile_data_key);
+  back_to = make_cleanup (null_cleanup, NULL);
+
+  if (dwarf2_per_objfile->all_comp_units == NULL)
+    {
+      struct dwarf2_cu *cu;
+      cu = load_full_comp_unit (pst, NULL);
+      make_cleanup (free_one_comp_unit, cu);
+      process_full_comp_unit (pst, cu);
+    }
+  else
+    {
+      struct dwarf2_per_cu_data *per_cu;
+      unsigned long offset;
+
+      offset = DWARF_INFO_OFFSET (pst);
+      per_cu = dwarf2_find_comp_unit (offset, pst->objfile);
+
+      per_cu->psymtab = pst;
+
+      make_cleanup (dwarf2_release_queue, NULL);
+
+      queue_comp_unit (per_cu);
+
+      process_queue (pst->objfile);
+
+      /* Age the cache, releasing compilation units that have not
+	 been used recently.  */
+      age_cached_comp_units ();
+    }
+
+  do_cleanups (back_to);
+}
+
+/* Load the DIEs associated with PST and PER_CU into memory.  */
+
+static struct dwarf2_cu *
+load_full_comp_unit (struct partial_symtab *pst,
+		     struct dwarf2_per_cu_data *per_cu)
+{
+  bfd *abfd = pst->objfile->obfd;
+  struct dwarf2_cu *cu;
+  unsigned long offset;
+  char *info_ptr;
+  struct cleanup *back_to, *free_cu_cleanup;
+  struct attribute *attr;
+  CORE_ADDR baseaddr;
 
   /* Set local variables from the partial symbol table info.  */
   offset = DWARF_INFO_OFFSET (pst);
 
   info_ptr = dwarf2_per_objfile->info_buffer + offset;
-  baseaddr = ANOFFSET (objfile->section_offsets, SECT_OFF_TEXT (objfile));
-
-  /* We're in the global namespace.  */
-  processing_current_prefix = "";
 
-  memset (&cu, 0, sizeof (struct dwarf2_cu));
-  obstack_init (&cu.comp_unit_obstack);
-  back_to = make_cleanup (free_stack_comp_unit, &cu);
+  cu = xmalloc (sizeof (struct dwarf2_cu));
+  memset (cu, 0, sizeof (struct dwarf2_cu));
 
-  buildsym_init ();
-  make_cleanup (really_free_pendings, NULL);
+  /* If an error occurs while loading, release our storage.  */
+  free_cu_cleanup = make_cleanup (free_one_comp_unit, cu);
 
-  cu.objfile = objfile;
+  cu->objfile = pst->objfile;
 
   /* read in the comp_unit header  */
-  info_ptr = read_comp_unit_head (&cu.header, info_ptr, abfd);
+  info_ptr = read_comp_unit_head (&cu->header, info_ptr, abfd);
 
   /* Read the abbrevs for this compilation unit  */
-  dwarf2_read_abbrevs (abfd, &cu);
-  make_cleanup (dwarf2_free_abbrev_table, &cu);
+  dwarf2_read_abbrevs (abfd, cu);
+  back_to = make_cleanup (dwarf2_free_abbrev_table, cu);
+
+  cu->header.offset = offset;
+
+  cu->per_cu = per_cu;
+  if (cu->per_cu)
+    per_cu->cu = cu;
+
+  /* We use this obstack for block values in dwarf_alloc_block.  */
+  obstack_init (&cu->comp_unit_obstack);
+
+  cu->dies = read_comp_unit (info_ptr, abfd, cu);
+
+  /* We try not to read any attributes in this function, because not
+     all objfiles needed for references have been loaded yet, and symbol
+     table processing isn't initialized.  But we have to set the CU language,
+     or we won't be able to build types correctly.  */
+  attr = dwarf2_attr (cu->dies, DW_AT_language, cu);
+  if (attr)
+    set_cu_language (DW_UNSND (attr), cu);
+  else
+    set_cu_language (language_minimal, cu);
+
+  do_cleanups (back_to);
 
-  cu.header.offset = offset;
+  /* We've successfully allocated this compilation unit.  Let our caller
+     clean it up when finished with it.  */
+  discard_cleanups (free_cu_cleanup);
 
-  cu.list_in_scope = &file_symbols;
+  return cu;
+}
 
-  dies = read_comp_unit (info_ptr, abfd, &cu);
+/* Generate full symbol information for PST and CU, whose DIEs have
+   already been loaded into memory.  */
 
-  make_cleanup_free_die_list (dies);
+static void
+process_full_comp_unit (struct partial_symtab *pst, struct dwarf2_cu *cu)
+{
+  struct objfile *objfile = pst->objfile;
+  bfd *abfd = objfile->obfd;
+  CORE_ADDR lowpc, highpc;
+  struct symtab *symtab;
+  struct cleanup *back_to;
+  struct attribute *attr;
+  CORE_ADDR baseaddr;
+
+  baseaddr = ANOFFSET (objfile->section_offsets, SECT_OFF_TEXT (objfile));
+
+  /* We're in the global namespace.  */
+  processing_current_prefix = "";
+
+  buildsym_init ();
+  back_to = make_cleanup (really_free_pendings, NULL);
+
+  cu->list_in_scope = &file_symbols;
 
   /* Find the base address of the compilation unit for range lists and
      location lists.  It will normally be specified by DW_AT_low_pc.
@@ -2413,32 +2624,32 @@ psymtab_to_symtab_1 (struct partial_symt
      DW_AT_entry_pc.  It's been removed, but GCC still uses this for
      compilation units with discontinuous ranges.  */
 
-  cu.header.base_known = 0;
-  cu.header.base_address = 0;
+  cu->header.base_known = 0;
+  cu->header.base_address = 0;
 
-  attr = dwarf2_attr (dies, DW_AT_entry_pc, &cu);
+  attr = dwarf2_attr (cu->dies, DW_AT_entry_pc, cu);
   if (attr)
     {
-      cu.header.base_address = DW_ADDR (attr);
-      cu.header.base_known = 1;
+      cu->header.base_address = DW_ADDR (attr);
+      cu->header.base_known = 1;
     }
   else
     {
-      attr = dwarf2_attr (dies, DW_AT_low_pc, &cu);
+      attr = dwarf2_attr (cu->dies, DW_AT_low_pc, cu);
       if (attr)
 	{
-	  cu.header.base_address = DW_ADDR (attr);
-	  cu.header.base_known = 1;
+	  cu->header.base_address = DW_ADDR (attr);
+	  cu->header.base_known = 1;
 	}
     }
 
   /* Do line number decoding in read_file_scope () */
-  process_die (dies, &cu);
+  process_die (cu->dies, cu);
 
   /* Some compilers don't define a DW_AT_high_pc attribute for the
      compilation unit.  If the DW_AT_high_pc is missing, synthesize
      it, by scanning the DIE's below the compilation unit.  */
-  get_scope_pc_bounds (dies, &lowpc, &highpc, &cu);
+  get_scope_pc_bounds (cu->dies, &lowpc, &highpc, cu);
 
   symtab = end_symtab (highpc + baseaddr, objfile, SECT_OFF_TEXT (objfile));
 
@@ -2446,9 +2657,9 @@ psymtab_to_symtab_1 (struct partial_symt
      If the compilation is from a C file generated by language preprocessors,
      do not set the language if it was already deduced by start_subfile.  */
   if (symtab != NULL
-      && !(cu.language == language_c && symtab->language != language_c))
+      && !(cu->language == language_c && symtab->language != language_c))
     {
-      symtab->language = cu.language;
+      symtab->language = cu->language;
     }
   pst->symtab = symtab;
   pst->readin = 1;
@@ -4610,6 +4821,9 @@ read_subrange_type (struct die_info *die
       low = 1;
     }
 
+  /* FIXME: For variable sized arrays either of these could be
+     a variable rather than a constant value.  We'll allow it,
+     but we don't know how to handle it.  */
   attr = dwarf2_attr (die, DW_AT_lower_bound, cu);
   if (attr)
     low = dwarf2_get_attr_constant_value (attr, 0);
@@ -4655,10 +4869,6 @@ read_subrange_type (struct die_info *die
 static struct die_info *
 read_comp_unit (char *info_ptr, bfd *abfd, struct dwarf2_cu *cu)
 {
-  /* Reset die reference table; we are
-     building new ones now.  */
-  dwarf2_empty_hash_tables ();
-
   return read_die_and_children (info_ptr, abfd, cu, &info_ptr, NULL);
 }
 
@@ -4679,7 +4889,7 @@ read_die_and_children (char *info_ptr, b
   int has_children;
 
   cur_ptr = read_full_die (&die, abfd, info_ptr, cu, &has_children);
-  store_in_ref_table (die->offset, die);
+  store_in_ref_table (die->offset, die, cu);
 
   if (has_children)
     {
@@ -4758,19 +4968,6 @@ free_die_list (struct die_info *dies)
     }
 }
 
-static void
-do_free_die_list_cleanup (void *dies)
-{
-  free_die_list (dies);
-}
-
-static struct cleanup *
-make_cleanup_free_die_list (struct die_info *dies)
-{
-  return make_cleanup (do_free_die_list_cleanup, dies);
-}
-
-
 /* Read the contents of the section at OFFSET and of size SIZE from the
    object file specified by OBJFILE into the objfile_obstack and return it.  */
 
@@ -5310,17 +5507,13 @@ find_partial_die_in_comp_unit (unsigned 
 /* Find a partial DIE at OFFSET, which may or may not be in CU.  */
 
 static struct partial_die_info *
-find_partial_die (unsigned long offset, struct dwarf2_cu *cu,
-		  struct dwarf2_cu **target_cu)
+find_partial_die (unsigned long offset, struct dwarf2_cu *cu)
 {
   struct dwarf2_per_cu_data *per_cu;
 
   if (offset >= cu->header.offset
       && offset < cu->header.offset + cu->header.length)
-    {
-      *target_cu = cu;
-      return find_partial_die_in_comp_unit (offset, cu);
-    }
+    return find_partial_die_in_comp_unit (offset, cu);
 
   per_cu = dwarf2_find_containing_comp_unit (offset, cu->objfile);
 
@@ -5339,7 +5532,6 @@ find_partial_die (unsigned long offset, 
     }
 
   per_cu->cu->last_used = 0;
-  *target_cu = per_cu->cu;
   return find_partial_die_in_comp_unit (offset, per_cu->cu);
 }
 
@@ -5356,11 +5548,10 @@ fixup_partial_die (struct partial_die_in
   if (part_die->name == NULL && part_die->has_specification)
     {
       struct partial_die_info *spec_die;
-      struct dwarf2_cu *spec_cu;
 
-      spec_die = find_partial_die (part_die->spec_offset, cu, &spec_cu);
+      spec_die = find_partial_die (part_die->spec_offset, cu);
 
-      fixup_partial_die (spec_die, spec_cu);
+      fixup_partial_die (spec_die, cu);
 
       if (spec_die->name)
 	{
@@ -5434,6 +5625,38 @@ read_full_die (struct die_info **diep, b
     {
       info_ptr = read_attribute (&die->attrs[i], &abbrev->attrs[i],
 				 abfd, info_ptr, cu);
+
+      /* If this attribute is an absolute reference to a different
+	 compilation unit, make sure that compilation unit is loaded
+	 also.  */
+      if (die->attrs[i].form == DW_FORM_ref_addr
+	  && (DW_ADDR (&die->attrs[i]) < cu->header.offset
+	      || (DW_ADDR (&die->attrs[i])
+		  >= cu->header.offset + cu->header.length)))
+	{
+	  struct dwarf2_per_cu_data *per_cu;
+	  per_cu = dwarf2_find_containing_comp_unit (DW_ADDR (&die->attrs[i]),
+						     cu->objfile);
+
+	  /* Mark the dependence relation so that we don't flush PER_CU
+	     too early.  */
+	  dwarf2_add_dependence (cu, per_cu);
+
+	  /* If it's already on the queue, we have nothing to do.  */
+	  if (per_cu->queued)
+	    continue;
+
+	  /* If the compilation unit is already loaded, just mark it as
+	     used.  */
+	  if (per_cu->cu != NULL)
+	    {
+	      per_cu->cu->last_used = 0;
+	      continue;
+	    }
+
+	  /* Add it to the queue.  */
+	  queue_comp_unit (per_cu);
+       }
     }
 
   *diep = die;
@@ -5530,23 +5753,24 @@ read_attribute_value (struct attribute *
       info_ptr += bytes_read;
       break;
     case DW_FORM_ref1:
-      DW_UNSND (attr) = read_1_byte (abfd, info_ptr);
+      DW_ADDR (attr) = cu->header.offset + read_1_byte (abfd, info_ptr);
       info_ptr += 1;
       break;
     case DW_FORM_ref2:
-      DW_UNSND (attr) = read_2_bytes (abfd, info_ptr);
+      DW_ADDR (attr) = cu->header.offset + read_2_bytes (abfd, info_ptr);
       info_ptr += 2;
       break;
     case DW_FORM_ref4:
-      DW_UNSND (attr) = read_4_bytes (abfd, info_ptr);
+      DW_ADDR (attr) = cu->header.offset + read_4_bytes (abfd, info_ptr);
       info_ptr += 4;
       break;
     case DW_FORM_ref8:
-      DW_UNSND (attr) = read_8_bytes (abfd, info_ptr);
+      DW_ADDR (attr) = cu->header.offset + read_8_bytes (abfd, info_ptr);
       info_ptr += 8;
       break;
     case DW_FORM_ref_udata:
-      DW_UNSND (attr) = read_unsigned_leb128 (abfd, info_ptr, &bytes_read);
+      DW_ADDR (attr) = (cu->header.offset
+			+ read_unsigned_leb128 (abfd, info_ptr, &bytes_read));
       info_ptr += bytes_read;
       break;
     case DW_FORM_indirect:
@@ -5954,21 +6178,14 @@ dwarf2_attr (struct die_info *die, unsig
   for (i = 0; i < die->num_attrs; ++i)
     {
       if (die->attrs[i].name == name)
-	{
-	  return &die->attrs[i];
-	}
+	return &die->attrs[i];
       if (die->attrs[i].name == DW_AT_specification
 	  || die->attrs[i].name == DW_AT_abstract_origin)
 	spec = &die->attrs[i];
     }
-  if (spec)
-    {
-      struct die_info *ref_die =
-      follow_die_ref (dwarf2_get_ref_die_offset (spec, cu));
 
-      if (ref_die)
-	return dwarf2_attr (ref_die, name, cu);
-    }
+  if (spec)
+    return dwarf2_attr (follow_die_ref (die, spec, cu), name, cu);
 
   return NULL;
 }
@@ -6010,7 +6227,7 @@ die_specification (struct die_info *die,
   if (spec_attr == NULL)
     return NULL;
   else
-    return follow_die_ref (dwarf2_get_ref_die_offset (spec_attr, cu));
+    return follow_die_ref (die, spec_attr, cu);
 }
 
 /* Free the line_header structure *LH, and any arrays and strings it
@@ -6941,7 +7158,6 @@ die_type (struct die_info *die, struct d
   struct type *type;
   struct attribute *type_attr;
   struct die_info *type_die;
-  unsigned int ref;
 
   type_attr = dwarf2_attr (die, DW_AT_type, cu);
   if (!type_attr)
@@ -6950,16 +7166,8 @@ die_type (struct die_info *die, struct d
       return dwarf2_fundamental_type (cu->objfile, FT_VOID, cu);
     }
   else
-    {
-      ref = dwarf2_get_ref_die_offset (type_attr, cu);
-      type_die = follow_die_ref (ref);
-      if (!type_die)
-	{
-	  error ("Dwarf Error: Cannot find referent at offset %d [in module %s]", 
-			  ref, cu->objfile->name);
-	  return NULL;
-	}
-    }
+    type_die = follow_die_ref (die, type_attr, cu);
+
   type = tag_type_to_type (type_die, cu);
   if (!type)
     {
@@ -6979,19 +7187,11 @@ die_containing_type (struct die_info *di
   struct type *type = NULL;
   struct attribute *type_attr;
   struct die_info *type_die = NULL;
-  unsigned int ref;
 
   type_attr = dwarf2_attr (die, DW_AT_containing_type, cu);
   if (type_attr)
     {
-      ref = dwarf2_get_ref_die_offset (type_attr, cu);
-      type_die = follow_die_ref (ref);
-      if (!type_die)
-	{
-	  error ("Dwarf Error: Cannot find referent at offset %d [in module %s]", ref, 
-			  cu->objfile->name);
-	  return NULL;
-	}
+      type_die = follow_die_ref (die, type_attr, cu);
       type = tag_type_to_type (type_die, cu);
     }
   if (!type)
@@ -7004,24 +7204,6 @@ die_containing_type (struct die_info *di
   return type;
 }
 
-#if 0
-static struct type *
-type_at_offset (unsigned int offset, struct dwarf2_cu *cu)
-{
-  struct die_info *die;
-  struct type *type;
-
-  die = follow_die_ref (offset);
-  if (!die)
-    {
-      error ("Dwarf Error: Cannot find type referent at offset %d.", offset);
-      return NULL;
-    }
-  type = tag_type_to_type (die, cu);
-  return type;
-}
-#endif
-
 static struct type *
 tag_type_to_type (struct die_info *die, struct dwarf2_cu *cu)
 {
@@ -7375,21 +7557,12 @@ static struct die_info *
 dwarf2_extension (struct die_info *die, struct dwarf2_cu *cu)
 {
   struct attribute *attr;
-  struct die_info *extension_die;
-  unsigned int ref;
 
   attr = dwarf2_attr (die, DW_AT_extension, cu);
   if (attr == NULL)
     return NULL;
 
-  ref = dwarf2_get_ref_die_offset (attr, cu);
-  extension_die = follow_die_ref (ref);
-  if (!extension_die)
-    {
-      error ("Dwarf Error: Cannot find referent at offset %d.", ref);
-    }
-
-  return extension_die;
+  return follow_die_ref (die, attr, cu);
 }
 
 /* Convert a DIE tag into its string name.  */
@@ -8226,13 +8399,16 @@ dump_die (struct die_info *die)
 	case DW_FORM_block1:
 	  fprintf_unfiltered (gdb_stderr, "block: size %d", DW_BLOCK (&die->attrs[i])->size);
 	  break;
+	case DW_FORM_ref1:
+	case DW_FORM_ref2:
+	case DW_FORM_ref4:
+	  fprintf_unfiltered (gdb_stderr, "constant ref: %ld (adjusted)",
+			      (long) (DW_ADDR (&die->attrs[i])));
+	  break;
 	case DW_FORM_data1:
 	case DW_FORM_data2:
 	case DW_FORM_data4:
 	case DW_FORM_data8:
-	case DW_FORM_ref1:
-	case DW_FORM_ref2:
-	case DW_FORM_ref4:
 	case DW_FORM_udata:
 	case DW_FORM_sdata:
 	  fprintf_unfiltered (gdb_stderr, "constant: %ld", DW_UNSND (&die->attrs[i]));
@@ -8276,22 +8452,16 @@ dump_die_list (struct die_info *die)
 }
 
 static void
-store_in_ref_table (unsigned int offset, struct die_info *die)
+store_in_ref_table (unsigned int offset, struct die_info *die,
+		    struct dwarf2_cu *cu)
 {
   int h;
   struct die_info *old;
 
   h = (offset % REF_HASH_SIZE);
-  old = die_ref_table[h];
+  old = cu->die_ref_table[h];
   die->next_ref = old;
-  die_ref_table[h] = die;
-}
-
-
-static void
-dwarf2_empty_hash_tables (void)
-{
-  memset (die_ref_table, 0, sizeof (die_ref_table));
+  cu->die_ref_table[h] = die;
 }
 
 static unsigned int
@@ -8302,14 +8472,12 @@ dwarf2_get_ref_die_offset (struct attrib
   switch (attr->form)
     {
     case DW_FORM_ref_addr:
-      result = DW_ADDR (attr);
-      break;
     case DW_FORM_ref1:
     case DW_FORM_ref2:
     case DW_FORM_ref4:
     case DW_FORM_ref8:
     case DW_FORM_ref_udata:
-      result = cu->header.offset + DW_UNSND (attr);
+      result = DW_ADDR (attr);
       break;
     default:
       complaint (&symfile_complaints,
@@ -8342,21 +8510,41 @@ dwarf2_get_attr_constant_value (struct a
 }
 
 static struct die_info *
-follow_die_ref (unsigned int offset)
+follow_die_ref (struct die_info *src_die, struct attribute *attr,
+		struct dwarf2_cu *cu)
 {
   struct die_info *die;
+  unsigned int offset;
   int h;
+  struct die_info temp_die;
+  struct dwarf2_cu *target_cu;
+
+  offset = dwarf2_get_ref_die_offset (attr, cu);
+
+  if (DW_ADDR (attr) < cu->header.offset
+      || DW_ADDR (attr) >= cu->header.offset + cu->header.length)
+    {
+      struct dwarf2_per_cu_data *per_cu;
+      per_cu = dwarf2_find_containing_comp_unit (DW_ADDR (attr),
+						 cu->objfile);
+      target_cu = per_cu->cu;
+    }
+  else
+    target_cu = cu;
 
   h = (offset % REF_HASH_SIZE);
-  die = die_ref_table[h];
+  die = target_cu->die_ref_table[h];
   while (die)
     {
       if (die->offset == offset)
-	{
-	  return die;
-	}
+	return die;
       die = die->next_ref;
     }
+
+  error ("Dwarf Error: Cannot find DIE at 0x%lx referenced from DIE "
+	 "at 0x%lx [in module %s]",
+	 (long) src_die->offset, (long) offset, cu->objfile->name);
+
   return NULL;
 }
 
@@ -9138,6 +9326,26 @@ dwarf2_find_comp_unit (unsigned long off
   return this_cu;
 }
 
+/* Return the partial_symtab associated with the compilation unit at OFFSET
+   in OBJFILE.  Most compilation units will have this field initialized when
+   partial symbols are read, but any initial compilation units (before we
+   created the per_cu structures) will not.  */
+static struct partial_symtab *
+dwarf2_find_comp_unit_psymtab (unsigned int offset, struct objfile *objfile)
+{
+  struct partial_symtab *pst;
+
+  ALL_PSYMTABS (objfile, pst)
+    {
+      if (DWARF_INFO_OFFSET (pst) == offset)
+	return pst;
+    }
+
+  internal_error (__FILE__, __LINE__,
+		  "could not find partial symtab for offset %d",
+		  offset);
+}
+
 /* Release one cached compilation unit, CU.  We unlink it from the tree
    of compilation units, but we don't remove it from the read_in_chain;
    the caller is responsible for that.  */
@@ -9152,6 +9360,8 @@ free_one_comp_unit (void *data)
   cu->per_cu = NULL;
 
   obstack_free (&cu->comp_unit_obstack, NULL);
+  if (cu->dies)
+    free_die_list (cu->dies);
 
   xfree (cu);
 }
@@ -9332,8 +9542,6 @@ set_die_type (struct die_info *die, stru
   **slot = ofs;
 }
 
-#if 0
-
 /* Find the type for DIE in TYPE_HASH, or return NULL if DIE does not
    have a saved type.  */
 
@@ -9369,17 +9577,56 @@ reset_die_and_siblings_types (struct die
     }
 }
 
-#endif
+/* Set the mark field in CU and in every other compilation unit in the
+   cache that we must keep because we are keeping CU.  */
+
+/* Add a dependence relationship from CU to REF_PER_CU.  */
+
+static void
+dwarf2_add_dependence (struct dwarf2_cu *cu,
+		       struct dwarf2_per_cu_data *ref_per_cu)
+{
+  void **slot;
+
+  if (cu->dependencies == NULL)
+    cu->dependencies
+      = htab_create_alloc_ex (5, htab_hash_pointer, htab_eq_pointer,
+			      NULL, &cu->comp_unit_obstack,
+			      hashtab_obstack_allocate,
+			      dummy_obstack_deallocate);
+
+  slot = htab_find_slot (cu->dependencies, ref_per_cu, INSERT);
+  if (*slot == NULL)
+    *slot = ref_per_cu;
+}
 
 /* Set the mark field in CU and in every other compilation unit in the
    cache that we must keep because we are keeping CU.  */
 
+static int
+dwarf2_mark_helper (void **slot, void *data)
+{
+  struct dwarf2_per_cu_data *per_cu;
+
+  per_cu = (struct dwarf2_per_cu_data *) *slot;
+  if (per_cu->cu->mark)
+    return 1;
+  per_cu->cu->mark = 1;
+
+  if (per_cu->cu->dependencies != NULL)
+    htab_traverse (per_cu->cu->dependencies, dwarf2_mark_helper, NULL);
+
+  return 1;
+}
+
 static void
 dwarf2_mark (struct dwarf2_cu *cu)
 {
   if (cu->mark)
     return;
   cu->mark = 1;
+  if (cu->dependencies != NULL)
+    htab_traverse (cu->dependencies, dwarf2_mark_helper, NULL);
 }
 
 static void

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

I take it the reason we can't just fill in all the per_cu->psymtab
values when we build the psymtabs in the first place is that we don't
actually build the CU index until we see a CU that contains (or might
contain) inter-CU references.  So we'd have to go back and record the
psymtabs we'd already created anyway.  Is that right?

We're constructing a psymtab for every CU we see; does it save much
time or space to avoid creating a twenty-byte structure as well?  I
understand that this is time- and space-critical code, but if the hair
isn't a noticable win, I'd rather it not go in.

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

Jim Blandy <jimb@redhat.com> writes:
> I take it the reason we can't just fill in all the per_cu->psymtab
> values when we build the psymtabs in the first place is that we don't
> actually build the CU index until we see a CU that contains (or might
> contain) inter-CU references.  So we'd have to go back and record the
> psymtabs we'd already created anyway.  Is that right?
> 
> We're constructing a psymtab for every CU we see; does it save much
> time or space to avoid creating a twenty-byte structure as well?  I
> understand that this is time- and space-critical code, but if the hair
> isn't a noticable win, I'd rather it not go in.

And then load_full_comp_unit and process_full_comp_unit would only
need to take a single argument.

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

Jim Blandy <jimb@redhat.com> writes:
> I take it the reason we can't just fill in all the per_cu->psymtab
> values when we build the psymtabs in the first place is that we don't
> actually build the CU index until we see a CU that contains (or might
> contain) inter-CU references.  So we'd have to go back and record the
> psymtabs we'd already created anyway.  Is that right?
> 
> We're constructing a psymtab for every CU we see; does it save much
> time or space to avoid creating a twenty-byte structure as well?  I
> understand that this is time- and space-critical code, but if the hair
> isn't a noticable win, I'd rather it not go in.

And dwarf2_build_psymtabs_hard and psymtab_to_symtab_1 could be
simplified.

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

If you made your queue into a real queue, with head and tail pointers,
and had queue_comp_unit not re-queue things that were already queued,
then you could replace the two nested loops around load_full_comp_unit
with a single, straightforward linked list walk, couldn't you?

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

Since we never toss types anyway, would it make sense to move
type_hash to dwarf2_per_objfile?

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

Here's a entirely separate thought that shouldn't be addressed as part
of this patch, if it's even a good idea:

I wonder how much of a speed hit it would actually be to get rid of
die->type altogether, and always go through the hash table.  I suppose
that's three distinct areas of memory getting hit (die; hash table;
type) instead of just two.  But it'd be nice to get rid of the hair of
restoring the die types when we re-load a CU.  We've got two
representations of the same information, with different lifetimes, and
we're paying for the usual hair of keeping them in sync; it'd be nice
to just get rid of the more volatile one altogether.

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

> @@ -4610,6 +4821,9 @@ read_subrange_type (struct die_info *die
>        low = 1;
>      }
>  
> +  /* FIXME: For variable sized arrays either of these could be
> +     a variable rather than a constant value.  We'll allow it,
> +     but we don't know how to handle it.  */
>    attr = dwarf2_attr (die, DW_AT_lower_bound, cu);
>    if (attr)
>      low = dwarf2_get_attr_constant_value (attr, 0);

This should be committed separately.

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

Other than that, looks great!

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Daniel Jacobowitz]

On Wed, Sep 29, 2004 at 12:49:35PM -0500, Jim Blandy wrote:
> 
> Since we never toss types anyway, would it make sense to move
> type_hash to dwarf2_per_objfile?

I don't think so.  type_hash is used in two ways: individual items are
set, when we know which CU we ought to have, and a whole CU is
restored, when we know which CU we're restoring.  It's always more
efficient to have a lot of small hash tables if you know precisely
which one you'll need; fewer collisions.

-- 
Daniel Jacobowitz

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Daniel Jacobowitz]

On Wed, Sep 29, 2004 at 02:50:20PM -0500, Jim Blandy wrote:
> 
> > @@ -4610,6 +4821,9 @@ read_subrange_type (struct die_info *die
> >        low = 1;
> >      }
> >  
> > +  /* FIXME: For variable sized arrays either of these could be
> > +     a variable rather than a constant value.  We'll allow it,
> > +     but we don't know how to handle it.  */
> >    attr = dwarf2_attr (die, DW_AT_lower_bound, cu);
> >    if (attr)
> >      low = dwarf2_get_attr_constant_value (attr, 0);
> 
> This should be committed separately.

OK, done as below.

-- 
Daniel Jacobowitz

2004-10-03  Daniel Jacobowitz  <dan@debian.org>

	* dwarf2read.c (read_subrange_type): Add comment for variable
	sized arrays.

Index: dwarf2read.c
===================================================================
RCS file: /cvs/src/src/gdb/dwarf2read.c,v
retrieving revision 1.163
diff -u -p -r1.163 dwarf2read.c
--- dwarf2read.c	21 Sep 2004 15:04:41 -0000	1.163
+++ dwarf2read.c	3 Oct 2004 16:14:23 -0000
@@ -4610,6 +4610,9 @@ read_subrange_type (struct die_info *die
       low = 1;
     }
 
+  /* FIXME: For variable sized arrays either of these could be
+     a variable rather than a constant value.  We'll allow it,
+     but we don't know how to handle it.  */
   attr = dwarf2_attr (die, DW_AT_lower_bound, cu);
   if (attr)
     low = dwarf2_get_attr_constant_value (attr, 0);

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Daniel Jacobowitz]

On Tue, Sep 28, 2004 at 06:05:18PM -0500, Jim Blandy wrote:
> 
> If you made your queue into a real queue, with head and tail pointers,
> and had queue_comp_unit not re-queue things that were already queued,
> then you could replace the two nested loops around load_full_comp_unit
> with a single, straightforward linked list walk, couldn't you?

Yes, it looks like that will work.  Much nicer.

-- 
Daniel Jacobowitz

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Daniel Jacobowitz]

On Tue, Sep 28, 2004 at 05:40:25PM -0500, Jim Blandy wrote:
> 
> I take it the reason we can't just fill in all the per_cu->psymtab
> values when we build the psymtabs in the first place is that we don't
> actually build the CU index until we see a CU that contains (or might
> contain) inter-CU references.  So we'd have to go back and record the
> psymtabs we'd already created anyway.  Is that right?
> 
> We're constructing a psymtab for every CU we see; does it save much
> time or space to avoid creating a twenty-byte structure as well?  I
> understand that this is time- and space-critical code, but if the hair
> isn't a noticable win, I'd rather it not go in.

Hi Jim,

I think that my earlier messages this afternoon and this patch address
all of your comments.  Please let me know if I missed anything.  The
biggest change is that I removed struct dwarf2_pinfo; instead, we
create a per_cu structure for every psymtab, and use that as the
private data.  This makes a heck of a lot more sense anyway.  The
function used to search for the right per_cu for this psymtab goes
away.  The pointer back to the psymtab is now always set.  It could be
eliminated by passing psymtabs around, but I don't want to do that;
right now per_cu structures correspond one-to-one to symtabs, but
conceptually there can be compilation units that do not correspond to
symtabs (I think).  I also reworked process_queue; it's much clearer
now (and only two passes instead of three).

OK?

-- 
Daniel Jacobowitz

2004-10-03  Daniel Jacobowitz  <dan@debian.org>

	* dwarf2read.c (REF_HASH_SIZE): Move earlier.
	(die_ref_table): Remove.
	(struct dwarf2_cu): Add DIES, DEPENDENCIES, and DIE_REF_TABLE.
	(struct dwarf2_per_cu_data): Add PSYMTAB.  Add a comment describing
	the usage of this type.
	(struct dwarf2_pinfo, PST_PRIVATE, DWARF_INFO_OFFSET): Remove.
	(struct dwarf2_queue_item, dwarf2_queue, dwarf2_queue_tail): New.
	(create_all_comp_units): Take an OFFSET argument and return the per_cu
	data for the CU at that offset.  Handle already-created per_cu
	structures.
	(dwarf2_create_include_psymtab): Update comment.
	(find_partial_die): Remove third argument.  Remove unreachable call
	to error ().
	(dwarf2_find_containing_comp_unit): Update comments.  Change one
	assertion to an error.
	(dwarf2_find_comp_unit): Update comments.
	(type_at_offset): Remove dead code.
	(make_cleanup_free_die_list, dwarf2_empty_hash_tables): Remove.
	(store_in_ref_table): Add CU argument.
	(follow_die_ref): Take DIE, attribute, and CU arguments.  Handle
	inter-compilation-unit references.
	(load_full_comp_unit, process_full_comp_unit): New functions, based
	on psymtab_to_symtab_1.
	(psymtab_to_symtab_1): Use them.
	(dwarf2_add_dependence): New function.
	(dwarf2_build_psymtabs_hard): Set the psymtab in per_cu.  Always create
	a per_cu structure, and save it in READ_PSYMTAB_PRIVATE.
	(partial_die_parent_scope, guess_structure_name): Update for changes
	to find_partial_die.
	(dwarf2_psymtab_to_symtab): Initialize dwarf2_per_objfile here.
	(queue_comp_unit, process_queue, dwarf2_release_queue): New.
	(read_comp_unit): Don't call dwarf2_empty_hash_tables.
	(read_die_and_children): Update call to store_in_ref_table.
	(do_free_die_list_cleanup): Remove.
	(fixup_partial_die): Update for changes to find_partial_die.
	(read_full_die): Handle queueing absolute references.
	(read_attribute_value): Use DW_ADDR for all DW_FORM_ref* forms.
	(dwarf2_attr, die_specification, die_type)
	(die_containing_type, dwarf2_extension): Update calls to
	follow_die_ref.
	(dump_die): Update DW_FORM_ref* handling.
	(dwarf2_get_ref_die_offset): Likewise.
	(free_one_comp_unit): Release the dies list.
	(dwarf2_mark_helper): New function.
	(dwarf2_mark): Use it.

--- src/gdb/dwarf2read.c	2004-10-03 12:14:10.000000000 -0400
+++ src/gdb/dwarf2read.c	2004-10-03 13:32:36.000000000 -0400
@@ -258,6 +258,11 @@ struct comp_unit_head
     int base_known;
   };
 
+/* Fixed size for the DIE hash table.  */
+#ifndef REF_HASH_SIZE
+#define REF_HASH_SIZE 1021
+#endif
+
 /* Internal state when decoding a particular compilation unit.  */
 struct dwarf2_cu
 {
@@ -325,6 +330,17 @@ struct dwarf2_cu
   /* How many compilation units ago was this CU last referenced?  */
   int last_used;
 
+  /* A hash table of die offsets for following references.  */
+  struct die_info *die_ref_table[REF_HASH_SIZE];
+
+  /* Full DIEs if read in.  */
+  struct die_info *dies;
+
+  /* A set of pointers to dwarf2_per_cu_data objects for compilation
+     units referenced by this one.  Only set during full symbol processing;
+     partial symbol tables do not have dependencies.  */
+  htab_t dependencies;
+
   /* Mark used when releasing cached dies.  */
   unsigned int mark : 1;
 
@@ -339,6 +355,12 @@ struct dwarf2_cu
   unsigned int has_namespace_info : 1;
 };
 
+/* Persistent data held for a compilation unit, even when not
+   processing it.  We put a pointer to this structure in the
+   read_symtab_private field of the psymtab.  If we encounter
+   inter-compilation-unit references, we also maintain a sorted
+   list of all compilation units.  */
+
 struct dwarf2_per_cu_data
 {
   /* The start offset and length of this compilation unit.  2**31-1
@@ -358,8 +380,10 @@ struct dwarf2_per_cu_data
      holds a map of DIE offsets to types.  It isn't always possible
      to reconstruct this information later, so we have to preserve
      it.  */
-
   htab_t type_hash;
+
+  /* The partial symbol table associated with this compilation unit.  */
+  struct partial_symtab *psymtab;
 };
 
 /* The line number information for a compilation unit (found in the
@@ -551,13 +575,6 @@ struct dwarf_block
 #define ATTR_ALLOC_CHUNK 4
 #endif
 
-/* A hash table of die offsets for following references.  */
-#ifndef REF_HASH_SIZE
-#define REF_HASH_SIZE 1021
-#endif
-
-static struct die_info *die_ref_table[REF_HASH_SIZE];
-
 /* Allocate fields for structs, unions and enums in this size.  */
 #ifndef DW_FIELD_ALLOC_CHUNK
 #define DW_FIELD_ALLOC_CHUNK 4
@@ -575,19 +592,6 @@ static int isreg;		/* Object lives in re
 				   decode_locdesc's return value is
 				   the register number.  */
 
-/* We put a pointer to this structure in the read_symtab_private field
-   of the psymtab.  */
-
-struct dwarf2_pinfo
-  {
-    /* Offset in .debug_info for this compilation unit. */
-
-    unsigned long dwarf_info_offset;
-  };
-
-#define PST_PRIVATE(p) ((struct dwarf2_pinfo *)(p)->read_symtab_private)
-#define DWARF_INFO_OFFSET(p) (PST_PRIVATE(p)->dwarf_info_offset)
-
 /* FIXME: We might want to set this from BFD via bfd_arch_bits_per_byte,
    but this would require a corresponding change in unpack_field_as_long
    and friends.  */
@@ -641,6 +645,17 @@ struct field_info
     int nfnfields;
   };
 
+/* One item on the queue of compilation units to read in full symbols
+   for.  */
+struct dwarf2_queue_item
+{
+  struct dwarf2_per_cu_data *per_cu;
+  struct dwarf2_queue_item *next;
+};
+
+/* The current queue.  */
+static struct dwarf2_queue_item *dwarf2_queue, *dwarf2_queue_tail;
+
 /* Loaded secondary compilation units are kept in memory until they
    have not been referenced for the processing of this many
    compilation units.  Set this to zero to disable caching.  Cache
@@ -755,8 +770,7 @@ static char *read_partial_die (struct pa
 			       bfd *, char *, struct dwarf2_cu *);
 
 static struct partial_die_info *find_partial_die (unsigned long,
-						  struct dwarf2_cu *,
-						  struct dwarf2_cu **);
+						  struct dwarf2_cu *);
 
 static void fixup_partial_die (struct partial_die_info *,
 			       struct dwarf2_cu *);
@@ -844,10 +858,6 @@ static struct type *die_type (struct die
 static struct type *die_containing_type (struct die_info *,
 					 struct dwarf2_cu *);
 
-#if 0
-static struct type *type_at_offset (unsigned int, struct objfile *);
-#endif
-
 static struct type *tag_type_to_type (struct die_info *, struct dwarf2_cu *);
 
 static void read_type_die (struct die_info *, struct dwarf2_cu *);
@@ -944,8 +954,6 @@ static struct die_info *read_die_and_sib
 
 static void free_die_list (struct die_info *);
 
-static struct cleanup *make_cleanup_free_die_list (struct die_info *);
-
 static void process_die (struct die_info *, struct dwarf2_cu *);
 
 static char *dwarf2_linkage_name (struct die_info *, struct dwarf2_cu *);
@@ -979,16 +987,17 @@ static void dump_die (struct die_info *)
 
 static void dump_die_list (struct die_info *);
 
-static void store_in_ref_table (unsigned int, struct die_info *);
-
-static void dwarf2_empty_hash_tables (void);
+static void store_in_ref_table (unsigned int, struct die_info *,
+				struct dwarf2_cu *);
 
 static unsigned int dwarf2_get_ref_die_offset (struct attribute *,
 					       struct dwarf2_cu *);
 
 static int dwarf2_get_attr_constant_value (struct attribute *, int);
 
-static struct die_info *follow_die_ref (unsigned int);
+static struct die_info *follow_die_ref (struct die_info *,
+					struct attribute *,
+					struct dwarf2_cu *);
 
 static struct type *dwarf2_fundamental_type (struct objfile *, int,
 					     struct dwarf2_cu *);
@@ -1045,12 +1054,18 @@ static void free_one_cached_comp_unit (v
 static void set_die_type (struct die_info *, struct type *,
 			  struct dwarf2_cu *);
 
-#if 0
 static void reset_die_and_siblings_types (struct die_info *,
 					  struct dwarf2_cu *);
-#endif
 
-static void create_all_comp_units (struct objfile *);
+static struct dwarf2_per_cu_data *create_all_comp_units (struct objfile *,
+							 unsigned long);
+
+static struct dwarf2_cu *load_full_comp_unit (struct dwarf2_per_cu_data *);
+
+static void process_full_comp_unit (struct dwarf2_per_cu_data *);
+
+static void dwarf2_add_dependence (struct dwarf2_cu *,
+				   struct dwarf2_per_cu_data *);
 
 static void dwarf2_mark (struct dwarf2_cu *);
 
@@ -1336,9 +1351,7 @@ dwarf2_create_include_psymtab (char *nam
 
   /* No private part is necessary for include psymtabs.  This property
      can be used to differentiate between such include psymtabs and
-     the regular ones.  If it ever happens that a regular psymtab can
-     legitimally have a NULL private part, then we'll have to add a
-     dedicated field for that in the dwarf2_pinfo structure.  */
+     the regular ones.  */
   subpst->read_symtab_private = NULL;
 }
 
@@ -1436,7 +1449,17 @@ dwarf2_build_psymtabs_hard (struct objfi
       make_cleanup (dwarf2_free_abbrev_table, &cu);
 
       if (cu.has_form_ref_addr && dwarf2_per_objfile->all_comp_units == NULL)
-	create_all_comp_units (objfile);
+	this_cu = create_all_comp_units (objfile, cu.header.offset);
+      else if (dwarf2_per_objfile->all_comp_units != NULL)
+	this_cu = dwarf2_find_comp_unit (cu.header.offset, objfile);
+      else
+	{
+	  this_cu = obstack_alloc (&objfile->objfile_obstack,
+				   sizeof (struct dwarf2_per_cu_data));
+	  memset (this_cu, 0, sizeof (*this_cu));
+	  this_cu->offset = cu.header.offset;
+	  this_cu->length = cu.header.length + cu.header.initial_length_size;
+	}
 
       /* Read the compilation unit die */
       abbrev = peek_die_abbrev (info_ptr, &bytes_read, &cu);
@@ -1456,9 +1479,8 @@ dwarf2_build_psymtabs_hard (struct objfi
       if (comp_unit_die.dirname)
 	pst->dirname = xstrdup (comp_unit_die.dirname);
 
-      pst->read_symtab_private = (char *)
-	obstack_alloc (&objfile->objfile_obstack, sizeof (struct dwarf2_pinfo));
-      DWARF_INFO_OFFSET (pst) = beg_of_comp_unit - dwarf2_per_objfile->info_buffer;
+      pst->read_symtab_private = (char *) this_cu;
+
       baseaddr = ANOFFSET (objfile->section_offsets, SECT_OFF_TEXT (objfile));
 
       /* Store the function that reads in the rest of the symbol table */
@@ -1466,30 +1488,28 @@ dwarf2_build_psymtabs_hard (struct objfi
 
       if (dwarf2_per_objfile->all_comp_units != NULL)
 	{
-	  struct dwarf2_per_cu_data *per_cu;
-
-	  per_cu = dwarf2_find_comp_unit (cu.header.offset, objfile);
-
 	  /* If this compilation unit was already read in, free the
 	     cached copy in order to read it in again.  This is
 	     necessary because we skipped some symbols when we first
 	     read in the compilation unit (see load_partial_dies).
 	     This problem could be avoided, but the benefit is
 	     unclear.  */
-	  if (per_cu->cu != NULL)
-	    free_one_cached_comp_unit (per_cu->cu);
+	  if (this_cu->cu != NULL)
+	    free_one_cached_comp_unit (this_cu->cu);
 
-	  cu.per_cu = per_cu;
+	  cu.per_cu = this_cu;
 
 	  /* Note that this is a pointer to our stack frame, being
 	     added to a global data structure.  It will be cleaned up
 	     in free_stack_comp_unit when we finish with this
 	     compilation unit.  */
-	  per_cu->cu = &cu;
+	  this_cu->cu = &cu;
 	}
       else
 	cu.per_cu = NULL;
 
+      this_cu->psymtab = pst;
+
       /* Check if comp unit has_children.
          If so, read the rest of the partial symbols from this comp unit.
          If not, there's no more debug_info for this comp unit. */
@@ -1605,19 +1625,19 @@ load_comp_unit (struct dwarf2_per_cu_dat
    there will be many, and one will occur early in the .debug_info section.
    So there's no point in building this list incrementally.  */
 
-static void
-create_all_comp_units (struct objfile *objfile)
+static struct dwarf2_per_cu_data *
+create_all_comp_units (struct objfile *objfile, unsigned long cur_offset)
 {
   int n_allocated;
   int n_comp_units;
-  struct dwarf2_per_cu_data **all_comp_units;
+  struct dwarf2_per_cu_data **all_comp_units, *cur_cu = NULL;
   char *info_ptr = dwarf2_per_objfile->info_buffer;
 
   n_comp_units = 0;
   n_allocated = 10;
   all_comp_units = xmalloc (n_allocated
 			    * sizeof (struct dwarf2_per_cu_data *));
-  
+
   while (info_ptr < dwarf2_per_objfile->info_buffer + dwarf2_per_objfile->info_size)
     {
       struct comp_unit_head cu_header;
@@ -1628,18 +1648,48 @@ create_all_comp_units (struct objfile *o
 
       offset = info_ptr - dwarf2_per_objfile->info_buffer;
 
-      /* Read just enough information to find out where the next
-	 compilation unit is.  */
-      cu_header.length = read_initial_length (objfile->obfd, info_ptr,
-					      &cu_header, &bytes_read);
+      /* All compilation units strictly before CUR_OFFSET have already
+	 had per_cu structures (and partial symtabs) created.  Find
+	 the psymtab.  This is quadratic, but we expect that the number of
+	 symtabs not including references will be small.  */
+      if (offset < cur_offset)
+	{
+	  struct partial_symtab *pst;
+	  struct dwarf2_per_cu_data *tmp_cu;
 
-      /* Save the compilation unit for later lookup.  */
-      this_cu = obstack_alloc (&objfile->objfile_obstack,
-			       sizeof (struct dwarf2_per_cu_data));
-      memset (this_cu, 0, sizeof (*this_cu));
-      this_cu->offset = offset;
-      this_cu->length = cu_header.length + cu_header.initial_length_size;
+	  this_cu = NULL;
+	  ALL_OBJFILE_PSYMTABS (objfile, pst)
+	    {
+	      if (pst->read_symtab != dwarf2_psymtab_to_symtab)
+		continue;
+	      tmp_cu = (struct dwarf2_per_cu_data *) pst->read_symtab_private;
+	      if (tmp_cu && tmp_cu->offset == offset)
+		{
+		  this_cu = tmp_cu;
+		  break;
+		}
+	    }
+	  gdb_assert (this_cu != NULL);
+	}
+      else
+	{
+	  /* Otherwise, create a new per_cu structure.  Read just
+	     enough information to find out where the next compilation
+	     unit is.  */
+	  cu_header.length = read_initial_length (objfile->obfd, info_ptr,
+						  &cu_header, &bytes_read);
+
+	  this_cu = obstack_alloc (&objfile->objfile_obstack,
+				   sizeof (struct dwarf2_per_cu_data));
+	  memset (this_cu, 0, sizeof (*this_cu));
+	  this_cu->offset = offset;
+	  this_cu->length = cu_header.length + cu_header.initial_length_size;
+
+	  if (cur_offset == offset)
+	    cur_cu = this_cu;
+	}
 
+      /* Save the compilation unit for later lookup.  */
       if (n_comp_units == n_allocated)
 	{
 	  n_allocated *= 2;
@@ -1659,6 +1709,9 @@ create_all_comp_units (struct objfile *o
 	  n_comp_units * sizeof (struct dwarf2_per_cu_data *));
   xfree (all_comp_units);
   dwarf2_per_objfile->n_comp_units = n_comp_units;
+
+  gdb_assert (cur_cu != NULL);
+  return cur_cu;
 }
 
 /* Process all loaded DIEs for compilation unit CU, starting at FIRST_DIE.
@@ -1778,15 +1831,13 @@ partial_die_parent_scope (struct partial
 {
   char *grandparent_scope;
   struct partial_die_info *parent, *real_pdi;
-  struct dwarf2_cu *spec_cu;
 
   /* We need to look at our parent DIE; if we have a DW_AT_specification,
      then this means the parent of the specification DIE.  */
 
   real_pdi = pdi;
-  spec_cu = cu;
   while (real_pdi->has_specification)
-    real_pdi = find_partial_die (real_pdi->spec_offset, spec_cu, &spec_cu);
+    real_pdi = find_partial_die (real_pdi->spec_offset, cu);
 
   parent = real_pdi->die_parent;
   if (parent == NULL)
@@ -1797,7 +1848,7 @@ partial_die_parent_scope (struct partial
 
   fixup_partial_die (parent, cu);
 
-  grandparent_scope = partial_die_parent_scope (parent, spec_cu);
+  grandparent_scope = partial_die_parent_scope (parent, cu);
 
   if (parent->tag == DW_TAG_namespace
       || parent->tag == DW_TAG_structure_type
@@ -2071,16 +2122,14 @@ guess_structure_name (struct partial_die
 
       struct partial_die_info *child_pdi = struct_pdi->die_child;
       struct partial_die_info *real_pdi;
-      struct dwarf2_cu *spec_cu;
 
       /* If this DIE (this DIE's specification, if any) has a parent, then
 	 we should not do this.  We'll prepend the parent's fully qualified
          name when we create the partial symbol.  */
 
       real_pdi = struct_pdi;
-      spec_cu = cu;
       while (real_pdi->has_specification)
-	real_pdi = find_partial_die (real_pdi->spec_offset, spec_cu, &spec_cu);
+	real_pdi = find_partial_die (real_pdi->spec_offset, cu);
 
       if (real_pdi->die_parent != NULL)
 	return;
@@ -2321,6 +2370,10 @@ dwarf2_psymtab_to_symtab (struct partial
 	      gdb_flush (gdb_stdout);
 	    }
 
+	  /* Restore our global data.  */
+	  dwarf2_per_objfile = objfile_data (pst->objfile,
+					     dwarf2_objfile_data_key);
+
 	  psymtab_to_symtab_1 (pst);
 
 	  /* Finish up the debug error message.  */
@@ -2330,21 +2383,104 @@ dwarf2_psymtab_to_symtab (struct partial
     }
 }
 
+/* Add PER_CU to the queue.  */
+
+static void
+queue_comp_unit (struct dwarf2_per_cu_data *per_cu)
+{
+  struct dwarf2_queue_item *item;
+
+  per_cu->queued = 1;
+  item = xmalloc (sizeof (*item));
+  item->per_cu = per_cu;
+  item->next = NULL;
+
+  if (dwarf2_queue == NULL)
+    dwarf2_queue = item;
+  else
+    dwarf2_queue_tail->next = item;
+
+  dwarf2_queue_tail = item;
+}
+
+/* Process the queue.  */
+
+static void
+process_queue (struct objfile *objfile)
+{
+  struct dwarf2_queue_item *item, *next_item;
+
+  /* Initially, there is just one item on the queue.  Load its DIEs,
+     and the DIEs of any other compilation units it requires,
+     transitively.  */
+
+  for (item = dwarf2_queue; item != NULL; item = item->next)
+    {
+      /* Read in this compilation unit.  This may add new items to
+	 the end of the queue.  */
+      load_full_comp_unit (item->per_cu);
+
+      item->per_cu->cu->read_in_chain = dwarf2_per_objfile->read_in_chain;
+      dwarf2_per_objfile->read_in_chain = item->per_cu;
+
+      /* If this compilation unit has already had full symbols created,
+	 reset the TYPE fields in each DIE.  */
+      if (item->per_cu->psymtab->readin)
+	reset_die_and_siblings_types (item->per_cu->cu->dies,
+				      item->per_cu->cu);
+    }
+
+  /* Now everything left on the queue needs to be read in.  Process
+     them, one at a time, removing from the queue as we finish.  */
+  for (item = dwarf2_queue; item != NULL; dwarf2_queue = item = next_item)
+    {
+      if (!item->per_cu->psymtab->readin)
+	process_full_comp_unit (item->per_cu);
+
+      item->per_cu->queued = 0;
+      next_item = item->next;
+      xfree (item);
+    }
+
+  dwarf2_queue_tail = NULL;
+}
+
+/* Free all allocated queue entries.  This function only releases anything if
+   an error was thrown; if the queue was processed then it would have been
+   freed as we went along.  */
+
+static void
+dwarf2_release_queue (void *dummy)
+{
+  struct dwarf2_queue_item *item, *last;
+
+  item = dwarf2_queue;
+  while (item)
+    {
+      /* Anything still marked queued is likely to be in an
+	 inconsistent state, so discard it.  */
+      if (item->per_cu->queued)
+	{
+	  if (item->per_cu->cu != NULL)
+	    free_one_cached_comp_unit (item->per_cu->cu);
+	  item->per_cu->queued = 0;
+	}
+
+      last = item;
+      item = item->next;
+      xfree (last);
+    }
+
+  dwarf2_queue = dwarf2_queue_tail = NULL;
+}
+
+/* Read in full symbols for PST, and anything it depends on.  */
+
 static void
 psymtab_to_symtab_1 (struct partial_symtab *pst)
 {
-  struct objfile *objfile = pst->objfile;
-  bfd *abfd = objfile->obfd;
-  struct dwarf2_cu cu;
-  struct die_info *dies;
-  unsigned long offset;
-  CORE_ADDR lowpc, highpc;
-  struct die_info *child_die;
-  char *info_ptr;
-  struct symtab *symtab;
+  struct dwarf2_per_cu_data *per_cu;
   struct cleanup *back_to;
-  struct attribute *attr;
-  CORE_ADDR baseaddr;
   int i;
 
   for (i = 0; i < pst->number_of_dependencies; i++)
@@ -2364,7 +2500,9 @@ psymtab_to_symtab_1 (struct partial_symt
         psymtab_to_symtab_1 (pst->dependencies[i]);
       }
 
-  if (pst->read_symtab_private == NULL)
+  per_cu = (struct dwarf2_per_cu_data *) pst->read_symtab_private;
+
+  if (per_cu == NULL)
     {
       /* It's an include file, no symbols to read for it.
          Everything is in the parent symtab.  */
@@ -2372,40 +2510,119 @@ psymtab_to_symtab_1 (struct partial_symt
       return;
     }
 
-  dwarf2_per_objfile = objfile_data (pst->objfile, dwarf2_objfile_data_key);
+  back_to = make_cleanup (null_cleanup, NULL);
+
+  if (dwarf2_per_objfile->all_comp_units == NULL)
+    {
+      struct dwarf2_cu *cu;
+      cu = load_full_comp_unit (per_cu);
+      make_cleanup (free_one_comp_unit, cu);
+      process_full_comp_unit (per_cu);
+    }
+  else
+    {
+      make_cleanup (dwarf2_release_queue, NULL);
+
+      queue_comp_unit (per_cu);
+
+      process_queue (pst->objfile);
+
+      /* Age the cache, releasing compilation units that have not
+	 been used recently.  */
+      age_cached_comp_units ();
+    }
+
+  do_cleanups (back_to);
+}
+
+/* Load the DIEs associated with PST and PER_CU into memory.  */
+
+static struct dwarf2_cu *
+load_full_comp_unit (struct dwarf2_per_cu_data *per_cu)
+{
+  struct partial_symtab *pst = per_cu->psymtab;
+  bfd *abfd = pst->objfile->obfd;
+  struct dwarf2_cu *cu;
+  unsigned long offset;
+  char *info_ptr;
+  struct cleanup *back_to, *free_cu_cleanup;
+  struct attribute *attr;
+  CORE_ADDR baseaddr;
 
   /* Set local variables from the partial symbol table info.  */
-  offset = DWARF_INFO_OFFSET (pst);
+  offset = per_cu->offset;
 
   info_ptr = dwarf2_per_objfile->info_buffer + offset;
-  baseaddr = ANOFFSET (objfile->section_offsets, SECT_OFF_TEXT (objfile));
-
-  /* We're in the global namespace.  */
-  processing_current_prefix = "";
 
-  memset (&cu, 0, sizeof (struct dwarf2_cu));
-  obstack_init (&cu.comp_unit_obstack);
-  back_to = make_cleanup (free_stack_comp_unit, &cu);
+  cu = xmalloc (sizeof (struct dwarf2_cu));
+  memset (cu, 0, sizeof (struct dwarf2_cu));
 
-  buildsym_init ();
-  make_cleanup (really_free_pendings, NULL);
+  /* If an error occurs while loading, release our storage.  */
+  free_cu_cleanup = make_cleanup (free_one_comp_unit, cu);
 
-  cu.objfile = objfile;
+  cu->objfile = pst->objfile;
 
   /* read in the comp_unit header  */
-  info_ptr = read_comp_unit_head (&cu.header, info_ptr, abfd);
+  info_ptr = read_comp_unit_head (&cu->header, info_ptr, abfd);
 
   /* Read the abbrevs for this compilation unit  */
-  dwarf2_read_abbrevs (abfd, &cu);
-  make_cleanup (dwarf2_free_abbrev_table, &cu);
+  dwarf2_read_abbrevs (abfd, cu);
+  back_to = make_cleanup (dwarf2_free_abbrev_table, cu);
+
+  cu->header.offset = offset;
+
+  cu->per_cu = per_cu;
+  per_cu->cu = cu;
+
+  /* We use this obstack for block values in dwarf_alloc_block.  */
+  obstack_init (&cu->comp_unit_obstack);
+
+  cu->dies = read_comp_unit (info_ptr, abfd, cu);
+
+  /* We try not to read any attributes in this function, because not
+     all objfiles needed for references have been loaded yet, and symbol
+     table processing isn't initialized.  But we have to set the CU language,
+     or we won't be able to build types correctly.  */
+  attr = dwarf2_attr (cu->dies, DW_AT_language, cu);
+  if (attr)
+    set_cu_language (DW_UNSND (attr), cu);
+  else
+    set_cu_language (language_minimal, cu);
+
+  do_cleanups (back_to);
+
+  /* We've successfully allocated this compilation unit.  Let our caller
+     clean it up when finished with it.  */
+  discard_cleanups (free_cu_cleanup);
 
-  cu.header.offset = offset;
+  return cu;
+}
+
+/* Generate full symbol information for PST and CU, whose DIEs have
+   already been loaded into memory.  */
+
+static void
+process_full_comp_unit (struct dwarf2_per_cu_data *per_cu)
+{
+  struct partial_symtab *pst = per_cu->psymtab;
+  struct dwarf2_cu *cu = per_cu->cu;
+  struct objfile *objfile = pst->objfile;
+  bfd *abfd = objfile->obfd;
+  CORE_ADDR lowpc, highpc;
+  struct symtab *symtab;
+  struct cleanup *back_to;
+  struct attribute *attr;
+  CORE_ADDR baseaddr;
 
-  cu.list_in_scope = &file_symbols;
+  baseaddr = ANOFFSET (objfile->section_offsets, SECT_OFF_TEXT (objfile));
 
-  dies = read_comp_unit (info_ptr, abfd, &cu);
+  /* We're in the global namespace.  */
+  processing_current_prefix = "";
+
+  buildsym_init ();
+  back_to = make_cleanup (really_free_pendings, NULL);
 
-  make_cleanup_free_die_list (dies);
+  cu->list_in_scope = &file_symbols;
 
   /* Find the base address of the compilation unit for range lists and
      location lists.  It will normally be specified by DW_AT_low_pc.
@@ -2413,32 +2630,32 @@ psymtab_to_symtab_1 (struct partial_symt
      DW_AT_entry_pc.  It's been removed, but GCC still uses this for
      compilation units with discontinuous ranges.  */
 
-  cu.header.base_known = 0;
-  cu.header.base_address = 0;
+  cu->header.base_known = 0;
+  cu->header.base_address = 0;
 
-  attr = dwarf2_attr (dies, DW_AT_entry_pc, &cu);
+  attr = dwarf2_attr (cu->dies, DW_AT_entry_pc, cu);
   if (attr)
     {
-      cu.header.base_address = DW_ADDR (attr);
-      cu.header.base_known = 1;
+      cu->header.base_address = DW_ADDR (attr);
+      cu->header.base_known = 1;
     }
   else
     {
-      attr = dwarf2_attr (dies, DW_AT_low_pc, &cu);
+      attr = dwarf2_attr (cu->dies, DW_AT_low_pc, cu);
       if (attr)
 	{
-	  cu.header.base_address = DW_ADDR (attr);
-	  cu.header.base_known = 1;
+	  cu->header.base_address = DW_ADDR (attr);
+	  cu->header.base_known = 1;
 	}
     }
 
   /* Do line number decoding in read_file_scope () */
-  process_die (dies, &cu);
+  process_die (cu->dies, cu);
 
   /* Some compilers don't define a DW_AT_high_pc attribute for the
      compilation unit.  If the DW_AT_high_pc is missing, synthesize
      it, by scanning the DIE's below the compilation unit.  */
-  get_scope_pc_bounds (dies, &lowpc, &highpc, &cu);
+  get_scope_pc_bounds (cu->dies, &lowpc, &highpc, cu);
 
   symtab = end_symtab (highpc + baseaddr, objfile, SECT_OFF_TEXT (objfile));
 
@@ -2446,9 +2663,9 @@ psymtab_to_symtab_1 (struct partial_symt
      If the compilation is from a C file generated by language preprocessors,
      do not set the language if it was already deduced by start_subfile.  */
   if (symtab != NULL
-      && !(cu.language == language_c && symtab->language != language_c))
+      && !(cu->language == language_c && symtab->language != language_c))
     {
-      symtab->language = cu.language;
+      symtab->language = cu->language;
     }
   pst->symtab = symtab;
   pst->readin = 1;
@@ -4658,10 +4875,6 @@ read_subrange_type (struct die_info *die
 static struct die_info *
 read_comp_unit (char *info_ptr, bfd *abfd, struct dwarf2_cu *cu)
 {
-  /* Reset die reference table; we are
-     building new ones now.  */
-  dwarf2_empty_hash_tables ();
-
   return read_die_and_children (info_ptr, abfd, cu, &info_ptr, NULL);
 }
 
@@ -4682,7 +4895,7 @@ read_die_and_children (char *info_ptr, b
   int has_children;
 
   cur_ptr = read_full_die (&die, abfd, info_ptr, cu, &has_children);
-  store_in_ref_table (die->offset, die);
+  store_in_ref_table (die->offset, die, cu);
 
   if (has_children)
     {
@@ -4761,19 +4974,6 @@ free_die_list (struct die_info *dies)
     }
 }
 
-static void
-do_free_die_list_cleanup (void *dies)
-{
-  free_die_list (dies);
-}
-
-static struct cleanup *
-make_cleanup_free_die_list (struct die_info *dies)
-{
-  return make_cleanup (do_free_die_list_cleanup, dies);
-}
-
-
 /* Read the contents of the section at OFFSET and of size SIZE from the
    object file specified by OBJFILE into the objfile_obstack and return it.  */
 
@@ -5313,27 +5513,16 @@ find_partial_die_in_comp_unit (unsigned 
 /* Find a partial DIE at OFFSET, which may or may not be in CU.  */
 
 static struct partial_die_info *
-find_partial_die (unsigned long offset, struct dwarf2_cu *cu,
-		  struct dwarf2_cu **target_cu)
+find_partial_die (unsigned long offset, struct dwarf2_cu *cu)
 {
   struct dwarf2_per_cu_data *per_cu;
 
   if (offset >= cu->header.offset
       && offset < cu->header.offset + cu->header.length)
-    {
-      *target_cu = cu;
-      return find_partial_die_in_comp_unit (offset, cu);
-    }
+    return find_partial_die_in_comp_unit (offset, cu);
 
   per_cu = dwarf2_find_containing_comp_unit (offset, cu->objfile);
 
-  /* If this offset isn't pointing into a known compilation unit,
-     the debug information is probably corrupted.  */
-  if (per_cu == NULL)
-    error ("Dwarf Error: could not find partial DIE containing "
-	   "offset 0x%lx [in module %s]",
-	   (long) offset, bfd_get_filename (cu->objfile->obfd));
-
   if (per_cu->cu == NULL)
     {
       load_comp_unit (per_cu, cu->objfile);
@@ -5342,7 +5531,6 @@ find_partial_die (unsigned long offset, 
     }
 
   per_cu->cu->last_used = 0;
-  *target_cu = per_cu->cu;
   return find_partial_die_in_comp_unit (offset, per_cu->cu);
 }
 
@@ -5359,11 +5547,10 @@ fixup_partial_die (struct partial_die_in
   if (part_die->name == NULL && part_die->has_specification)
     {
       struct partial_die_info *spec_die;
-      struct dwarf2_cu *spec_cu;
 
-      spec_die = find_partial_die (part_die->spec_offset, cu, &spec_cu);
+      spec_die = find_partial_die (part_die->spec_offset, cu);
 
-      fixup_partial_die (spec_die, spec_cu);
+      fixup_partial_die (spec_die, cu);
 
       if (spec_die->name)
 	{
@@ -5437,6 +5624,38 @@ read_full_die (struct die_info **diep, b
     {
       info_ptr = read_attribute (&die->attrs[i], &abbrev->attrs[i],
 				 abfd, info_ptr, cu);
+
+      /* If this attribute is an absolute reference to a different
+	 compilation unit, make sure that compilation unit is loaded
+	 also.  */
+      if (die->attrs[i].form == DW_FORM_ref_addr
+	  && (DW_ADDR (&die->attrs[i]) < cu->header.offset
+	      || (DW_ADDR (&die->attrs[i])
+		  >= cu->header.offset + cu->header.length)))
+	{
+	  struct dwarf2_per_cu_data *per_cu;
+	  per_cu = dwarf2_find_containing_comp_unit (DW_ADDR (&die->attrs[i]),
+						     cu->objfile);
+
+	  /* Mark the dependence relation so that we don't flush PER_CU
+	     too early.  */
+	  dwarf2_add_dependence (cu, per_cu);
+
+	  /* If it's already on the queue, we have nothing to do.  */
+	  if (per_cu->queued)
+	    continue;
+
+	  /* If the compilation unit is already loaded, just mark it as
+	     used.  */
+	  if (per_cu->cu != NULL)
+	    {
+	      per_cu->cu->last_used = 0;
+	      continue;
+	    }
+
+	  /* Add it to the queue.  */
+	  queue_comp_unit (per_cu);
+       }
     }
 
   *diep = die;
@@ -5533,23 +5752,24 @@ read_attribute_value (struct attribute *
       info_ptr += bytes_read;
       break;
     case DW_FORM_ref1:
-      DW_UNSND (attr) = read_1_byte (abfd, info_ptr);
+      DW_ADDR (attr) = cu->header.offset + read_1_byte (abfd, info_ptr);
       info_ptr += 1;
       break;
     case DW_FORM_ref2:
-      DW_UNSND (attr) = read_2_bytes (abfd, info_ptr);
+      DW_ADDR (attr) = cu->header.offset + read_2_bytes (abfd, info_ptr);
       info_ptr += 2;
       break;
     case DW_FORM_ref4:
-      DW_UNSND (attr) = read_4_bytes (abfd, info_ptr);
+      DW_ADDR (attr) = cu->header.offset + read_4_bytes (abfd, info_ptr);
       info_ptr += 4;
       break;
     case DW_FORM_ref8:
-      DW_UNSND (attr) = read_8_bytes (abfd, info_ptr);
+      DW_ADDR (attr) = cu->header.offset + read_8_bytes (abfd, info_ptr);
       info_ptr += 8;
       break;
     case DW_FORM_ref_udata:
-      DW_UNSND (attr) = read_unsigned_leb128 (abfd, info_ptr, &bytes_read);
+      DW_ADDR (attr) = (cu->header.offset
+			+ read_unsigned_leb128 (abfd, info_ptr, &bytes_read));
       info_ptr += bytes_read;
       break;
     case DW_FORM_indirect:
@@ -5957,21 +6177,14 @@ dwarf2_attr (struct die_info *die, unsig
   for (i = 0; i < die->num_attrs; ++i)
     {
       if (die->attrs[i].name == name)
-	{
-	  return &die->attrs[i];
-	}
+	return &die->attrs[i];
       if (die->attrs[i].name == DW_AT_specification
 	  || die->attrs[i].name == DW_AT_abstract_origin)
 	spec = &die->attrs[i];
     }
-  if (spec)
-    {
-      struct die_info *ref_die =
-      follow_die_ref (dwarf2_get_ref_die_offset (spec, cu));
 
-      if (ref_die)
-	return dwarf2_attr (ref_die, name, cu);
-    }
+  if (spec)
+    return dwarf2_attr (follow_die_ref (die, spec, cu), name, cu);
 
   return NULL;
 }
@@ -6013,7 +6226,7 @@ die_specification (struct die_info *die,
   if (spec_attr == NULL)
     return NULL;
   else
-    return follow_die_ref (dwarf2_get_ref_die_offset (spec_attr, cu));
+    return follow_die_ref (die, spec_attr, cu);
 }
 
 /* Free the line_header structure *LH, and any arrays and strings it
@@ -6944,7 +7157,6 @@ die_type (struct die_info *die, struct d
   struct type *type;
   struct attribute *type_attr;
   struct die_info *type_die;
-  unsigned int ref;
 
   type_attr = dwarf2_attr (die, DW_AT_type, cu);
   if (!type_attr)
@@ -6953,16 +7165,8 @@ die_type (struct die_info *die, struct d
       return dwarf2_fundamental_type (cu->objfile, FT_VOID, cu);
     }
   else
-    {
-      ref = dwarf2_get_ref_die_offset (type_attr, cu);
-      type_die = follow_die_ref (ref);
-      if (!type_die)
-	{
-	  error ("Dwarf Error: Cannot find referent at offset %d [in module %s]", 
-			  ref, cu->objfile->name);
-	  return NULL;
-	}
-    }
+    type_die = follow_die_ref (die, type_attr, cu);
+
   type = tag_type_to_type (type_die, cu);
   if (!type)
     {
@@ -6982,19 +7186,11 @@ die_containing_type (struct die_info *di
   struct type *type = NULL;
   struct attribute *type_attr;
   struct die_info *type_die = NULL;
-  unsigned int ref;
 
   type_attr = dwarf2_attr (die, DW_AT_containing_type, cu);
   if (type_attr)
     {
-      ref = dwarf2_get_ref_die_offset (type_attr, cu);
-      type_die = follow_die_ref (ref);
-      if (!type_die)
-	{
-	  error ("Dwarf Error: Cannot find referent at offset %d [in module %s]", ref, 
-			  cu->objfile->name);
-	  return NULL;
-	}
+      type_die = follow_die_ref (die, type_attr, cu);
       type = tag_type_to_type (type_die, cu);
     }
   if (!type)
@@ -7007,24 +7203,6 @@ die_containing_type (struct die_info *di
   return type;
 }
 
-#if 0
-static struct type *
-type_at_offset (unsigned int offset, struct dwarf2_cu *cu)
-{
-  struct die_info *die;
-  struct type *type;
-
-  die = follow_die_ref (offset);
-  if (!die)
-    {
-      error ("Dwarf Error: Cannot find type referent at offset %d.", offset);
-      return NULL;
-    }
-  type = tag_type_to_type (die, cu);
-  return type;
-}
-#endif
-
 static struct type *
 tag_type_to_type (struct die_info *die, struct dwarf2_cu *cu)
 {
@@ -7378,21 +7556,12 @@ static struct die_info *
 dwarf2_extension (struct die_info *die, struct dwarf2_cu *cu)
 {
   struct attribute *attr;
-  struct die_info *extension_die;
-  unsigned int ref;
 
   attr = dwarf2_attr (die, DW_AT_extension, cu);
   if (attr == NULL)
     return NULL;
 
-  ref = dwarf2_get_ref_die_offset (attr, cu);
-  extension_die = follow_die_ref (ref);
-  if (!extension_die)
-    {
-      error ("Dwarf Error: Cannot find referent at offset %d.", ref);
-    }
-
-  return extension_die;
+  return follow_die_ref (die, attr, cu);
 }
 
 /* Convert a DIE tag into its string name.  */
@@ -8229,13 +8398,16 @@ dump_die (struct die_info *die)
 	case DW_FORM_block1:
 	  fprintf_unfiltered (gdb_stderr, "block: size %d", DW_BLOCK (&die->attrs[i])->size);
 	  break;
+	case DW_FORM_ref1:
+	case DW_FORM_ref2:
+	case DW_FORM_ref4:
+	  fprintf_unfiltered (gdb_stderr, "constant ref: %ld (adjusted)",
+			      (long) (DW_ADDR (&die->attrs[i])));
+	  break;
 	case DW_FORM_data1:
 	case DW_FORM_data2:
 	case DW_FORM_data4:
 	case DW_FORM_data8:
-	case DW_FORM_ref1:
-	case DW_FORM_ref2:
-	case DW_FORM_ref4:
 	case DW_FORM_udata:
 	case DW_FORM_sdata:
 	  fprintf_unfiltered (gdb_stderr, "constant: %ld", DW_UNSND (&die->attrs[i]));
@@ -8279,22 +8451,16 @@ dump_die_list (struct die_info *die)
 }
 
 static void
-store_in_ref_table (unsigned int offset, struct die_info *die)
+store_in_ref_table (unsigned int offset, struct die_info *die,
+		    struct dwarf2_cu *cu)
 {
   int h;
   struct die_info *old;
 
   h = (offset % REF_HASH_SIZE);
-  old = die_ref_table[h];
+  old = cu->die_ref_table[h];
   die->next_ref = old;
-  die_ref_table[h] = die;
-}
-
-
-static void
-dwarf2_empty_hash_tables (void)
-{
-  memset (die_ref_table, 0, sizeof (die_ref_table));
+  cu->die_ref_table[h] = die;
 }
 
 static unsigned int
@@ -8305,14 +8471,12 @@ dwarf2_get_ref_die_offset (struct attrib
   switch (attr->form)
     {
     case DW_FORM_ref_addr:
-      result = DW_ADDR (attr);
-      break;
     case DW_FORM_ref1:
     case DW_FORM_ref2:
     case DW_FORM_ref4:
     case DW_FORM_ref8:
     case DW_FORM_ref_udata:
-      result = cu->header.offset + DW_UNSND (attr);
+      result = DW_ADDR (attr);
       break;
     default:
       complaint (&symfile_complaints,
@@ -8345,21 +8509,41 @@ dwarf2_get_attr_constant_value (struct a
 }
 
 static struct die_info *
-follow_die_ref (unsigned int offset)
+follow_die_ref (struct die_info *src_die, struct attribute *attr,
+		struct dwarf2_cu *cu)
 {
   struct die_info *die;
+  unsigned int offset;
   int h;
+  struct die_info temp_die;
+  struct dwarf2_cu *target_cu;
+
+  offset = dwarf2_get_ref_die_offset (attr, cu);
+
+  if (DW_ADDR (attr) < cu->header.offset
+      || DW_ADDR (attr) >= cu->header.offset + cu->header.length)
+    {
+      struct dwarf2_per_cu_data *per_cu;
+      per_cu = dwarf2_find_containing_comp_unit (DW_ADDR (attr),
+						 cu->objfile);
+      target_cu = per_cu->cu;
+    }
+  else
+    target_cu = cu;
 
   h = (offset % REF_HASH_SIZE);
-  die = die_ref_table[h];
+  die = target_cu->die_ref_table[h];
   while (die)
     {
       if (die->offset == offset)
-	{
-	  return die;
-	}
+	return die;
       die = die->next_ref;
     }
+
+  error ("Dwarf Error: Cannot find DIE at 0x%lx referenced from DIE "
+	 "at 0x%lx [in module %s]",
+	 (long) src_die->offset, (long) offset, cu->objfile->name);
+
   return NULL;
 }
 
@@ -9089,7 +9273,7 @@ dwarf2_symbol_mark_computed (struct attr
 }
 
 /* Locate the compilation unit from CU's objfile which contains the
-   DIE at OFFSET.  Returns NULL on failure.  */
+   DIE at OFFSET.  Raises an error on failure.  */
 
 static struct dwarf2_per_cu_data *
 dwarf2_find_containing_comp_unit (unsigned long offset,
@@ -9116,7 +9300,11 @@ dwarf2_find_containing_comp_unit (unsign
   gdb_assert (low == high);
   if (dwarf2_per_objfile->all_comp_units[low]->offset > offset)
     {
-      gdb_assert (low > 0);
+      if (low == 0)
+	error ("Dwarf Error: could not find partial DIE containing "
+	       "offset 0x%lx [in module %s]",
+	       (long) offset, bfd_get_filename (objfile->obfd));
+
       gdb_assert (dwarf2_per_objfile->all_comp_units[low-1]->offset <= offset);
       return dwarf2_per_objfile->all_comp_units[low-1];
     }
@@ -9131,6 +9319,9 @@ dwarf2_find_containing_comp_unit (unsign
     }
 }
 
+/* Locate the compilation unit from OBJFILE which is located at exactly
+   OFFSET.  Raises an error on failure.  */
+
 static struct dwarf2_per_cu_data *
 dwarf2_find_comp_unit (unsigned long offset, struct objfile *objfile)
 {
@@ -9155,6 +9346,8 @@ free_one_comp_unit (void *data)
   cu->per_cu = NULL;
 
   obstack_free (&cu->comp_unit_obstack, NULL);
+  if (cu->dies)
+    free_die_list (cu->dies);
 
   xfree (cu);
 }
@@ -9335,8 +9528,6 @@ set_die_type (struct die_info *die, stru
   **slot = ofs;
 }
 
-#if 0
-
 /* Find the type for DIE in TYPE_HASH, or return NULL if DIE does not
    have a saved type.  */
 
@@ -9372,17 +9563,56 @@ reset_die_and_siblings_types (struct die
     }
 }
 
-#endif
+/* Set the mark field in CU and in every other compilation unit in the
+   cache that we must keep because we are keeping CU.  */
+
+/* Add a dependence relationship from CU to REF_PER_CU.  */
+
+static void
+dwarf2_add_dependence (struct dwarf2_cu *cu,
+		       struct dwarf2_per_cu_data *ref_per_cu)
+{
+  void **slot;
+
+  if (cu->dependencies == NULL)
+    cu->dependencies
+      = htab_create_alloc_ex (5, htab_hash_pointer, htab_eq_pointer,
+			      NULL, &cu->comp_unit_obstack,
+			      hashtab_obstack_allocate,
+			      dummy_obstack_deallocate);
+
+  slot = htab_find_slot (cu->dependencies, ref_per_cu, INSERT);
+  if (*slot == NULL)
+    *slot = ref_per_cu;
+}
 
 /* Set the mark field in CU and in every other compilation unit in the
    cache that we must keep because we are keeping CU.  */
 
+static int
+dwarf2_mark_helper (void **slot, void *data)
+{
+  struct dwarf2_per_cu_data *per_cu;
+
+  per_cu = (struct dwarf2_per_cu_data *) *slot;
+  if (per_cu->cu->mark)
+    return 1;
+  per_cu->cu->mark = 1;
+
+  if (per_cu->cu->dependencies != NULL)
+    htab_traverse (per_cu->cu->dependencies, dwarf2_mark_helper, NULL);
+
+  return 1;
+}
+
 static void
 dwarf2_mark (struct dwarf2_cu *cu)
 {
   if (cu->mark)
     return;
   cu->mark = 1;
+  if (cu->dependencies != NULL)
+    htab_traverse (cu->dependencies, dwarf2_mark_helper, NULL);
 }
 
 static void

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

Daniel Jacobowitz <drow@false.org> writes:
> On Wed, Sep 29, 2004 at 12:49:35PM -0500, Jim Blandy wrote:
> > Since we never toss types anyway, would it make sense to move
> > type_hash to dwarf2_per_objfile?
> 
> I don't think so.  type_hash is used in two ways: individual items are
> set, when we know which CU we ought to have, and a whole CU is
> restored, when we know which CU we're restoring.  It's always more
> efficient to have a lot of small hash tables if you know precisely
> which one you'll need; fewer collisions.

Really?  libiberty/hashtab.c is a resizing hash table; I thought hash
table resizing was supposed to keep the collision rate roughly
constant (modulo hysteresis) regardless of the number of elements.  If
that's not so, doesn't that mean your hash function isn't doing its
job spreading the elements across the (adequately sized) table?

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Daniel Jacobowitz]

On Mon, Oct 04, 2004 at 04:14:24PM -0500, Jim Blandy wrote:
> 
> Daniel Jacobowitz <drow@false.org> writes:
> > On Wed, Sep 29, 2004 at 12:49:35PM -0500, Jim Blandy wrote:
> > > Since we never toss types anyway, would it make sense to move
> > > type_hash to dwarf2_per_objfile?
> > 
> > I don't think so.  type_hash is used in two ways: individual items are
> > set, when we know which CU we ought to have, and a whole CU is
> > restored, when we know which CU we're restoring.  It's always more
> > efficient to have a lot of small hash tables if you know precisely
> > which one you'll need; fewer collisions.
> 
> Really?  libiberty/hashtab.c is a resizing hash table; I thought hash
> table resizing was supposed to keep the collision rate roughly
> constant (modulo hysteresis) regardless of the number of elements.  If
> that's not so, doesn't that mean your hash function isn't doing its
> job spreading the elements across the (adequately sized) table?

Poor job of thinking on my part, there.  The rest of the paragraph
still makes sense to me, though.  For instance, in a resizing hash
table, I suspect that there is more copying to have one large
expandable hash table than several small ones.

I haven't done the math for that, of course.  Maybe I've got it
backwards.  Do you think there would be any advantage to doing it the
other way round?

-- 
Daniel Jacobowitz

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

This patch still avoids building the all_comp_units table if it never
finds a CU whose abbrev table suggests it might contain inter-CU
references.  What I was actually suggesting was building
all_comp_units for every objfile, unconditionally --- even for
objfiles with no inter-CU references, and which will thus never use
the table.

The win would be to remove the case of an absent all_comp_units table
from the code.  I want to keep the number of different cases one needs
to think about down where doing so doesn't provide a noticeable
performance hit.

On an 8000-CU objfile (what you estimated glibc compiled with
duplicate elimination would have), that would be a memory overhead of
160k, which doesn't seem bad.  It's true that that would be a separate
pass over the .debug_info section, but the latest patch will do two
passes anyway.

The quadratic bit in create_all_comp_units to recover the psymtabs
we'd already created wouldn't be needed any more.

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Daniel Jacobowitz]

On Mon, Oct 04, 2004 at 05:08:36PM -0500, Jim Blandy wrote:
> 
> This patch still avoids building the all_comp_units table if it never
> finds a CU whose abbrev table suggests it might contain inter-CU
> references.  What I was actually suggesting was building
> all_comp_units for every objfile, unconditionally --- even for
> objfiles with no inter-CU references, and which will thus never use
> the table.
> 
> The win would be to remove the case of an absent all_comp_units table
> from the code.  I want to keep the number of different cases one needs
> to think about down where doing so doesn't provide a noticeable
> performance hit.
> 
> On an 8000-CU objfile (what you estimated glibc compiled with
> duplicate elimination would have), that would be a memory overhead of
> 160k, which doesn't seem bad.  It's true that that would be a separate
> pass over the .debug_info section, but the latest patch will do two
> passes anyway.

I don't think it will do two passes... but:

> The quadratic bit in create_all_comp_units to recover the psymtabs
> we'd already created wouldn't be needed any more.

True.

The performance impact is negligibly in favor of lazy allocation on an
inter-CU object, about 1% in favor of it on an old object, and about 1%
against it (!!!) on an old object but reading in full symbols. 
Apparently:
  - it's a wash, so we may as well use inter-cu-aware mode always
  - I ought to get around to cleaning up the releasing of full DIEs;
    that's why it's faster, I think, because the last five compilation
    units (in the cache) don't get freed.  This could be obstacked
    easily enough.

Here's a revision which always creates the CU tree.  Retested on
i686-pc-linux-gnu; no changes.

-- 
Daniel Jacobowitz

2004-10-04  Daniel Jacobowitz  <dan@debian.org>

	* dwarf2read.c (REF_HASH_SIZE): Move earlier.
	(die_ref_table): Remove.
	(struct dwarf2_cu): Add DIES, DEPENDENCIES, and DIE_REF_TABLE.
	(struct dwarf2_per_cu_data): Add PSYMTAB.  Add a comment describing
	the usage of this type.
	(struct dwarf2_per_objfile): Update comment for ALL_COMP_UNITS.
	(struct dwarf2_pinfo, PST_PRIVATE, DWARF_INFO_OFFSET): Remove.
	(struct dwarf2_queue_item, dwarf2_queue, dwarf2_queue_tail): New.
	(dwarf2_create_include_psymtab): Update comment.
	(find_partial_die): Remove third argument.  Remove unreachable call
	to error ().
	(dwarf2_find_containing_comp_unit): Update comments.  Change one
	assertion to an error.  Remove an unreachable error.
	(dwarf2_find_comp_unit): Update comments.
	(type_at_offset): Remove dead code.
	(make_cleanup_free_die_list, dwarf2_empty_hash_tables): Remove.
	(store_in_ref_table): Add CU argument.
	(follow_die_ref): Take DIE, attribute, and CU arguments.  Handle
	inter-compilation-unit references.
	(load_full_comp_unit, process_full_comp_unit): New functions, based
	on psymtab_to_symtab_1.
	(psymtab_to_symtab_1): Use them.
	(dwarf2_add_dependence): New function.
	(dwarf2_build_psymtabs_hard): Set the psymtab in per_cu.  Always create
	a per_cu structure, and save it in READ_PSYMTAB_PRIVATE.
	(partial_die_parent_scope, guess_structure_name): Update for changes
	to find_partial_die.
	(dwarf2_psymtab_to_symtab): Initialize dwarf2_per_objfile here.
	(queue_comp_unit, process_queue, dwarf2_release_queue): New.
	(read_comp_unit): Don't call dwarf2_empty_hash_tables.
	(read_die_and_children): Update call to store_in_ref_table.
	(do_free_die_list_cleanup): Remove.
	(fixup_partial_die): Update for changes to find_partial_die.
	(read_full_die): Handle queueing absolute references.
	(read_attribute_value): Use DW_ADDR for all DW_FORM_ref* forms.
	(dwarf2_attr, die_specification, die_type)
	(die_containing_type, dwarf2_extension): Update calls to
	follow_die_ref.
	(dump_die): Update DW_FORM_ref* handling.
	(dwarf2_get_ref_die_offset): Likewise.
	(free_one_comp_unit): Release the dies list.
	(dwarf2_mark_helper): New function.
	(dwarf2_mark): Use it.

--- src/gdb/dwarf2read.c	2004-10-03 12:14:10.000000000 -0400
+++ src/gdb/dwarf2read.c	2004-10-04 18:58:48.000000000 -0400
@@ -169,9 +169,8 @@ struct dwarf2_per_objfile
   char *ranges_buffer;
   char *loc_buffer;
 
-  /* A list of all the compilation units.  This will be set if and
-     only if we have encountered a compilation unit with inter-CU
-     references.  */
+  /* A list of all the compilation units.  This is used to locate
+     the target compilation unit of a particular reference.  */
   struct dwarf2_per_cu_data **all_comp_units;
 
   /* The number of compilation units in ALL_COMP_UNITS.  */
@@ -258,6 +257,11 @@ struct comp_unit_head
     int base_known;
   };
 
+/* Fixed size for the DIE hash table.  */
+#ifndef REF_HASH_SIZE
+#define REF_HASH_SIZE 1021
+#endif
+
 /* Internal state when decoding a particular compilation unit.  */
 struct dwarf2_cu
 {
@@ -325,6 +329,17 @@ struct dwarf2_cu
   /* How many compilation units ago was this CU last referenced?  */
   int last_used;
 
+  /* A hash table of die offsets for following references.  */
+  struct die_info *die_ref_table[REF_HASH_SIZE];
+
+  /* Full DIEs if read in.  */
+  struct die_info *dies;
+
+  /* A set of pointers to dwarf2_per_cu_data objects for compilation
+     units referenced by this one.  Only set during full symbol processing;
+     partial symbol tables do not have dependencies.  */
+  htab_t dependencies;
+
   /* Mark used when releasing cached dies.  */
   unsigned int mark : 1;
 
@@ -339,6 +354,12 @@ struct dwarf2_cu
   unsigned int has_namespace_info : 1;
 };
 
+/* Persistent data held for a compilation unit, even when not
+   processing it.  We put a pointer to this structure in the
+   read_symtab_private field of the psymtab.  If we encounter
+   inter-compilation-unit references, we also maintain a sorted
+   list of all compilation units.  */
+
 struct dwarf2_per_cu_data
 {
   /* The start offset and length of this compilation unit.  2**31-1
@@ -358,8 +379,10 @@ struct dwarf2_per_cu_data
      holds a map of DIE offsets to types.  It isn't always possible
      to reconstruct this information later, so we have to preserve
      it.  */
-
   htab_t type_hash;
+
+  /* The partial symbol table associated with this compilation unit.  */
+  struct partial_symtab *psymtab;
 };
 
 /* The line number information for a compilation unit (found in the
@@ -551,13 +574,6 @@ struct dwarf_block
 #define ATTR_ALLOC_CHUNK 4
 #endif
 
-/* A hash table of die offsets for following references.  */
-#ifndef REF_HASH_SIZE
-#define REF_HASH_SIZE 1021
-#endif
-
-static struct die_info *die_ref_table[REF_HASH_SIZE];
-
 /* Allocate fields for structs, unions and enums in this size.  */
 #ifndef DW_FIELD_ALLOC_CHUNK
 #define DW_FIELD_ALLOC_CHUNK 4
@@ -575,19 +591,6 @@ static int isreg;		/* Object lives in re
 				   decode_locdesc's return value is
 				   the register number.  */
 
-/* We put a pointer to this structure in the read_symtab_private field
-   of the psymtab.  */
-
-struct dwarf2_pinfo
-  {
-    /* Offset in .debug_info for this compilation unit. */
-
-    unsigned long dwarf_info_offset;
-  };
-
-#define PST_PRIVATE(p) ((struct dwarf2_pinfo *)(p)->read_symtab_private)
-#define DWARF_INFO_OFFSET(p) (PST_PRIVATE(p)->dwarf_info_offset)
-
 /* FIXME: We might want to set this from BFD via bfd_arch_bits_per_byte,
    but this would require a corresponding change in unpack_field_as_long
    and friends.  */
@@ -641,6 +644,17 @@ struct field_info
     int nfnfields;
   };
 
+/* One item on the queue of compilation units to read in full symbols
+   for.  */
+struct dwarf2_queue_item
+{
+  struct dwarf2_per_cu_data *per_cu;
+  struct dwarf2_queue_item *next;
+};
+
+/* The current queue.  */
+static struct dwarf2_queue_item *dwarf2_queue, *dwarf2_queue_tail;
+
 /* Loaded secondary compilation units are kept in memory until they
    have not been referenced for the processing of this many
    compilation units.  Set this to zero to disable caching.  Cache
@@ -755,8 +769,7 @@ static char *read_partial_die (struct pa
 			       bfd *, char *, struct dwarf2_cu *);
 
 static struct partial_die_info *find_partial_die (unsigned long,
-						  struct dwarf2_cu *,
-						  struct dwarf2_cu **);
+						  struct dwarf2_cu *);
 
 static void fixup_partial_die (struct partial_die_info *,
 			       struct dwarf2_cu *);
@@ -844,10 +857,6 @@ static struct type *die_type (struct die
 static struct type *die_containing_type (struct die_info *,
 					 struct dwarf2_cu *);
 
-#if 0
-static struct type *type_at_offset (unsigned int, struct objfile *);
-#endif
-
 static struct type *tag_type_to_type (struct die_info *, struct dwarf2_cu *);
 
 static void read_type_die (struct die_info *, struct dwarf2_cu *);
@@ -944,8 +953,6 @@ static struct die_info *read_die_and_sib
 
 static void free_die_list (struct die_info *);
 
-static struct cleanup *make_cleanup_free_die_list (struct die_info *);
-
 static void process_die (struct die_info *, struct dwarf2_cu *);
 
 static char *dwarf2_linkage_name (struct die_info *, struct dwarf2_cu *);
@@ -979,16 +986,17 @@ static void dump_die (struct die_info *)
 
 static void dump_die_list (struct die_info *);
 
-static void store_in_ref_table (unsigned int, struct die_info *);
-
-static void dwarf2_empty_hash_tables (void);
+static void store_in_ref_table (unsigned int, struct die_info *,
+				struct dwarf2_cu *);
 
 static unsigned int dwarf2_get_ref_die_offset (struct attribute *,
 					       struct dwarf2_cu *);
 
 static int dwarf2_get_attr_constant_value (struct attribute *, int);
 
-static struct die_info *follow_die_ref (unsigned int);
+static struct die_info *follow_die_ref (struct die_info *,
+					struct attribute *,
+					struct dwarf2_cu *);
 
 static struct type *dwarf2_fundamental_type (struct objfile *, int,
 					     struct dwarf2_cu *);
@@ -1045,13 +1053,18 @@ static void free_one_cached_comp_unit (v
 static void set_die_type (struct die_info *, struct type *,
 			  struct dwarf2_cu *);
 
-#if 0
 static void reset_die_and_siblings_types (struct die_info *,
 					  struct dwarf2_cu *);
-#endif
 
 static void create_all_comp_units (struct objfile *);
 
+static struct dwarf2_cu *load_full_comp_unit (struct dwarf2_per_cu_data *);
+
+static void process_full_comp_unit (struct dwarf2_per_cu_data *);
+
+static void dwarf2_add_dependence (struct dwarf2_cu *,
+				   struct dwarf2_per_cu_data *);
+
 static void dwarf2_mark (struct dwarf2_cu *);
 
 static void dwarf2_clear_marks (struct dwarf2_per_cu_data *);
@@ -1336,9 +1349,7 @@ dwarf2_create_include_psymtab (char *nam
 
   /* No private part is necessary for include psymtabs.  This property
      can be used to differentiate between such include psymtabs and
-     the regular ones.  If it ever happens that a regular psymtab can
-     legitimally have a NULL private part, then we'll have to add a
-     dedicated field for that in the dwarf2_pinfo structure.  */
+     the regular ones.  */
   subpst->read_symtab_private = NULL;
 }
 
@@ -1391,6 +1402,8 @@ dwarf2_build_psymtabs_hard (struct objfi
      read_in_chain.  Make sure to free them when we're done.  */
   back_to = make_cleanup (free_cached_comp_units, NULL);
 
+  create_all_comp_units (objfile);
+
   /* Since the objects we're extracting from .debug_info vary in
      length, only the individual functions to extract them (like
      read_comp_unit_head and load_partial_die) can really know whether
@@ -1435,8 +1448,7 @@ dwarf2_build_psymtabs_hard (struct objfi
       dwarf2_read_abbrevs (abfd, &cu);
       make_cleanup (dwarf2_free_abbrev_table, &cu);
 
-      if (cu.has_form_ref_addr && dwarf2_per_objfile->all_comp_units == NULL)
-	create_all_comp_units (objfile);
+      this_cu = dwarf2_find_comp_unit (cu.header.offset, objfile);
 
       /* Read the compilation unit die */
       abbrev = peek_die_abbrev (info_ptr, &bytes_read, &cu);
@@ -1456,39 +1468,31 @@ dwarf2_build_psymtabs_hard (struct objfi
       if (comp_unit_die.dirname)
 	pst->dirname = xstrdup (comp_unit_die.dirname);
 
-      pst->read_symtab_private = (char *)
-	obstack_alloc (&objfile->objfile_obstack, sizeof (struct dwarf2_pinfo));
-      DWARF_INFO_OFFSET (pst) = beg_of_comp_unit - dwarf2_per_objfile->info_buffer;
+      pst->read_symtab_private = (char *) this_cu;
+
       baseaddr = ANOFFSET (objfile->section_offsets, SECT_OFF_TEXT (objfile));
 
       /* Store the function that reads in the rest of the symbol table */
       pst->read_symtab = dwarf2_psymtab_to_symtab;
 
-      if (dwarf2_per_objfile->all_comp_units != NULL)
-	{
-	  struct dwarf2_per_cu_data *per_cu;
-
-	  per_cu = dwarf2_find_comp_unit (cu.header.offset, objfile);
-
-	  /* If this compilation unit was already read in, free the
-	     cached copy in order to read it in again.  This is
-	     necessary because we skipped some symbols when we first
-	     read in the compilation unit (see load_partial_dies).
-	     This problem could be avoided, but the benefit is
-	     unclear.  */
-	  if (per_cu->cu != NULL)
-	    free_one_cached_comp_unit (per_cu->cu);
-
-	  cu.per_cu = per_cu;
+      /* If this compilation unit was already read in, free the
+	 cached copy in order to read it in again.  This is
+	 necessary because we skipped some symbols when we first
+	 read in the compilation unit (see load_partial_dies).
+	 This problem could be avoided, but the benefit is
+	 unclear.  */
+      if (this_cu->cu != NULL)
+	free_one_cached_comp_unit (this_cu->cu);
+
+      cu.per_cu = this_cu;
+
+      /* Note that this is a pointer to our stack frame, being
+	 added to a global data structure.  It will be cleaned up
+	 in free_stack_comp_unit when we finish with this
+	 compilation unit.  */
+      this_cu->cu = &cu;
 
-	  /* Note that this is a pointer to our stack frame, being
-	     added to a global data structure.  It will be cleaned up
-	     in free_stack_comp_unit when we finish with this
-	     compilation unit.  */
-	  per_cu->cu = &cu;
-	}
-      else
-	cu.per_cu = NULL;
+      this_cu->psymtab = pst;
 
       /* Check if comp unit has_children.
          If so, read the rest of the partial symbols from this comp unit.
@@ -1778,15 +1782,13 @@ partial_die_parent_scope (struct partial
 {
   char *grandparent_scope;
   struct partial_die_info *parent, *real_pdi;
-  struct dwarf2_cu *spec_cu;
 
   /* We need to look at our parent DIE; if we have a DW_AT_specification,
      then this means the parent of the specification DIE.  */
 
   real_pdi = pdi;
-  spec_cu = cu;
   while (real_pdi->has_specification)
-    real_pdi = find_partial_die (real_pdi->spec_offset, spec_cu, &spec_cu);
+    real_pdi = find_partial_die (real_pdi->spec_offset, cu);
 
   parent = real_pdi->die_parent;
   if (parent == NULL)
@@ -1797,7 +1799,7 @@ partial_die_parent_scope (struct partial
 
   fixup_partial_die (parent, cu);
 
-  grandparent_scope = partial_die_parent_scope (parent, spec_cu);
+  grandparent_scope = partial_die_parent_scope (parent, cu);
 
   if (parent->tag == DW_TAG_namespace
       || parent->tag == DW_TAG_structure_type
@@ -2071,16 +2073,14 @@ guess_structure_name (struct partial_die
 
       struct partial_die_info *child_pdi = struct_pdi->die_child;
       struct partial_die_info *real_pdi;
-      struct dwarf2_cu *spec_cu;
 
       /* If this DIE (this DIE's specification, if any) has a parent, then
 	 we should not do this.  We'll prepend the parent's fully qualified
          name when we create the partial symbol.  */
 
       real_pdi = struct_pdi;
-      spec_cu = cu;
       while (real_pdi->has_specification)
-	real_pdi = find_partial_die (real_pdi->spec_offset, spec_cu, &spec_cu);
+	real_pdi = find_partial_die (real_pdi->spec_offset, cu);
 
       if (real_pdi->die_parent != NULL)
 	return;
@@ -2321,6 +2321,10 @@ dwarf2_psymtab_to_symtab (struct partial
 	      gdb_flush (gdb_stdout);
 	    }
 
+	  /* Restore our global data.  */
+	  dwarf2_per_objfile = objfile_data (pst->objfile,
+					     dwarf2_objfile_data_key);
+
 	  psymtab_to_symtab_1 (pst);
 
 	  /* Finish up the debug error message.  */
@@ -2330,21 +2334,104 @@ dwarf2_psymtab_to_symtab (struct partial
     }
 }
 
+/* Add PER_CU to the queue.  */
+
+static void
+queue_comp_unit (struct dwarf2_per_cu_data *per_cu)
+{
+  struct dwarf2_queue_item *item;
+
+  per_cu->queued = 1;
+  item = xmalloc (sizeof (*item));
+  item->per_cu = per_cu;
+  item->next = NULL;
+
+  if (dwarf2_queue == NULL)
+    dwarf2_queue = item;
+  else
+    dwarf2_queue_tail->next = item;
+
+  dwarf2_queue_tail = item;
+}
+
+/* Process the queue.  */
+
+static void
+process_queue (struct objfile *objfile)
+{
+  struct dwarf2_queue_item *item, *next_item;
+
+  /* Initially, there is just one item on the queue.  Load its DIEs,
+     and the DIEs of any other compilation units it requires,
+     transitively.  */
+
+  for (item = dwarf2_queue; item != NULL; item = item->next)
+    {
+      /* Read in this compilation unit.  This may add new items to
+	 the end of the queue.  */
+      load_full_comp_unit (item->per_cu);
+
+      item->per_cu->cu->read_in_chain = dwarf2_per_objfile->read_in_chain;
+      dwarf2_per_objfile->read_in_chain = item->per_cu;
+
+      /* If this compilation unit has already had full symbols created,
+	 reset the TYPE fields in each DIE.  */
+      if (item->per_cu->psymtab->readin)
+	reset_die_and_siblings_types (item->per_cu->cu->dies,
+				      item->per_cu->cu);
+    }
+
+  /* Now everything left on the queue needs to be read in.  Process
+     them, one at a time, removing from the queue as we finish.  */
+  for (item = dwarf2_queue; item != NULL; dwarf2_queue = item = next_item)
+    {
+      if (!item->per_cu->psymtab->readin)
+	process_full_comp_unit (item->per_cu);
+
+      item->per_cu->queued = 0;
+      next_item = item->next;
+      xfree (item);
+    }
+
+  dwarf2_queue_tail = NULL;
+}
+
+/* Free all allocated queue entries.  This function only releases anything if
+   an error was thrown; if the queue was processed then it would have been
+   freed as we went along.  */
+
+static void
+dwarf2_release_queue (void *dummy)
+{
+  struct dwarf2_queue_item *item, *last;
+
+  item = dwarf2_queue;
+  while (item)
+    {
+      /* Anything still marked queued is likely to be in an
+	 inconsistent state, so discard it.  */
+      if (item->per_cu->queued)
+	{
+	  if (item->per_cu->cu != NULL)
+	    free_one_cached_comp_unit (item->per_cu->cu);
+	  item->per_cu->queued = 0;
+	}
+
+      last = item;
+      item = item->next;
+      xfree (last);
+    }
+
+  dwarf2_queue = dwarf2_queue_tail = NULL;
+}
+
+/* Read in full symbols for PST, and anything it depends on.  */
+
 static void
 psymtab_to_symtab_1 (struct partial_symtab *pst)
 {
-  struct objfile *objfile = pst->objfile;
-  bfd *abfd = objfile->obfd;
-  struct dwarf2_cu cu;
-  struct die_info *dies;
-  unsigned long offset;
-  CORE_ADDR lowpc, highpc;
-  struct die_info *child_die;
-  char *info_ptr;
-  struct symtab *symtab;
+  struct dwarf2_per_cu_data *per_cu;
   struct cleanup *back_to;
-  struct attribute *attr;
-  CORE_ADDR baseaddr;
   int i;
 
   for (i = 0; i < pst->number_of_dependencies; i++)
@@ -2364,7 +2451,9 @@ psymtab_to_symtab_1 (struct partial_symt
         psymtab_to_symtab_1 (pst->dependencies[i]);
       }
 
-  if (pst->read_symtab_private == NULL)
+  per_cu = (struct dwarf2_per_cu_data *) pst->read_symtab_private;
+
+  if (per_cu == NULL)
     {
       /* It's an include file, no symbols to read for it.
          Everything is in the parent symtab.  */
@@ -2372,40 +2461,107 @@ psymtab_to_symtab_1 (struct partial_symt
       return;
     }
 
-  dwarf2_per_objfile = objfile_data (pst->objfile, dwarf2_objfile_data_key);
+  back_to = make_cleanup (dwarf2_release_queue, NULL);
+
+  queue_comp_unit (per_cu);
+
+  process_queue (pst->objfile);
+
+  /* Age the cache, releasing compilation units that have not
+     been used recently.  */
+  age_cached_comp_units ();
+
+  do_cleanups (back_to);
+}
+
+/* Load the DIEs associated with PST and PER_CU into memory.  */
+
+static struct dwarf2_cu *
+load_full_comp_unit (struct dwarf2_per_cu_data *per_cu)
+{
+  struct partial_symtab *pst = per_cu->psymtab;
+  bfd *abfd = pst->objfile->obfd;
+  struct dwarf2_cu *cu;
+  unsigned long offset;
+  char *info_ptr;
+  struct cleanup *back_to, *free_cu_cleanup;
+  struct attribute *attr;
+  CORE_ADDR baseaddr;
 
   /* Set local variables from the partial symbol table info.  */
-  offset = DWARF_INFO_OFFSET (pst);
+  offset = per_cu->offset;
 
   info_ptr = dwarf2_per_objfile->info_buffer + offset;
-  baseaddr = ANOFFSET (objfile->section_offsets, SECT_OFF_TEXT (objfile));
-
-  /* We're in the global namespace.  */
-  processing_current_prefix = "";
 
-  memset (&cu, 0, sizeof (struct dwarf2_cu));
-  obstack_init (&cu.comp_unit_obstack);
-  back_to = make_cleanup (free_stack_comp_unit, &cu);
+  cu = xmalloc (sizeof (struct dwarf2_cu));
+  memset (cu, 0, sizeof (struct dwarf2_cu));
 
-  buildsym_init ();
-  make_cleanup (really_free_pendings, NULL);
+  /* If an error occurs while loading, release our storage.  */
+  free_cu_cleanup = make_cleanup (free_one_comp_unit, cu);
 
-  cu.objfile = objfile;
+  cu->objfile = pst->objfile;
 
   /* read in the comp_unit header  */
-  info_ptr = read_comp_unit_head (&cu.header, info_ptr, abfd);
+  info_ptr = read_comp_unit_head (&cu->header, info_ptr, abfd);
 
   /* Read the abbrevs for this compilation unit  */
-  dwarf2_read_abbrevs (abfd, &cu);
-  make_cleanup (dwarf2_free_abbrev_table, &cu);
+  dwarf2_read_abbrevs (abfd, cu);
+  back_to = make_cleanup (dwarf2_free_abbrev_table, cu);
+
+  cu->header.offset = offset;
+
+  cu->per_cu = per_cu;
+  per_cu->cu = cu;
+
+  /* We use this obstack for block values in dwarf_alloc_block.  */
+  obstack_init (&cu->comp_unit_obstack);
 
-  cu.header.offset = offset;
+  cu->dies = read_comp_unit (info_ptr, abfd, cu);
 
-  cu.list_in_scope = &file_symbols;
+  /* We try not to read any attributes in this function, because not
+     all objfiles needed for references have been loaded yet, and symbol
+     table processing isn't initialized.  But we have to set the CU language,
+     or we won't be able to build types correctly.  */
+  attr = dwarf2_attr (cu->dies, DW_AT_language, cu);
+  if (attr)
+    set_cu_language (DW_UNSND (attr), cu);
+  else
+    set_cu_language (language_minimal, cu);
 
-  dies = read_comp_unit (info_ptr, abfd, &cu);
+  do_cleanups (back_to);
 
-  make_cleanup_free_die_list (dies);
+  /* We've successfully allocated this compilation unit.  Let our caller
+     clean it up when finished with it.  */
+  discard_cleanups (free_cu_cleanup);
+
+  return cu;
+}
+
+/* Generate full symbol information for PST and CU, whose DIEs have
+   already been loaded into memory.  */
+
+static void
+process_full_comp_unit (struct dwarf2_per_cu_data *per_cu)
+{
+  struct partial_symtab *pst = per_cu->psymtab;
+  struct dwarf2_cu *cu = per_cu->cu;
+  struct objfile *objfile = pst->objfile;
+  bfd *abfd = objfile->obfd;
+  CORE_ADDR lowpc, highpc;
+  struct symtab *symtab;
+  struct cleanup *back_to;
+  struct attribute *attr;
+  CORE_ADDR baseaddr;
+
+  baseaddr = ANOFFSET (objfile->section_offsets, SECT_OFF_TEXT (objfile));
+
+  /* We're in the global namespace.  */
+  processing_current_prefix = "";
+
+  buildsym_init ();
+  back_to = make_cleanup (really_free_pendings, NULL);
+
+  cu->list_in_scope = &file_symbols;
 
   /* Find the base address of the compilation unit for range lists and
      location lists.  It will normally be specified by DW_AT_low_pc.
@@ -2413,32 +2569,32 @@ psymtab_to_symtab_1 (struct partial_symt
      DW_AT_entry_pc.  It's been removed, but GCC still uses this for
      compilation units with discontinuous ranges.  */
 
-  cu.header.base_known = 0;
-  cu.header.base_address = 0;
+  cu->header.base_known = 0;
+  cu->header.base_address = 0;
 
-  attr = dwarf2_attr (dies, DW_AT_entry_pc, &cu);
+  attr = dwarf2_attr (cu->dies, DW_AT_entry_pc, cu);
   if (attr)
     {
-      cu.header.base_address = DW_ADDR (attr);
-      cu.header.base_known = 1;
+      cu->header.base_address = DW_ADDR (attr);
+      cu->header.base_known = 1;
     }
   else
     {
-      attr = dwarf2_attr (dies, DW_AT_low_pc, &cu);
+      attr = dwarf2_attr (cu->dies, DW_AT_low_pc, cu);
       if (attr)
 	{
-	  cu.header.base_address = DW_ADDR (attr);
-	  cu.header.base_known = 1;
+	  cu->header.base_address = DW_ADDR (attr);
+	  cu->header.base_known = 1;
 	}
     }
 
   /* Do line number decoding in read_file_scope () */
-  process_die (dies, &cu);
+  process_die (cu->dies, cu);
 
   /* Some compilers don't define a DW_AT_high_pc attribute for the
      compilation unit.  If the DW_AT_high_pc is missing, synthesize
      it, by scanning the DIE's below the compilation unit.  */
-  get_scope_pc_bounds (dies, &lowpc, &highpc, &cu);
+  get_scope_pc_bounds (cu->dies, &lowpc, &highpc, cu);
 
   symtab = end_symtab (highpc + baseaddr, objfile, SECT_OFF_TEXT (objfile));
 
@@ -2446,9 +2602,9 @@ psymtab_to_symtab_1 (struct partial_symt
      If the compilation is from a C file generated by language preprocessors,
      do not set the language if it was already deduced by start_subfile.  */
   if (symtab != NULL
-      && !(cu.language == language_c && symtab->language != language_c))
+      && !(cu->language == language_c && symtab->language != language_c))
     {
-      symtab->language = cu.language;
+      symtab->language = cu->language;
     }
   pst->symtab = symtab;
   pst->readin = 1;
@@ -4658,10 +4814,6 @@ read_subrange_type (struct die_info *die
 static struct die_info *
 read_comp_unit (char *info_ptr, bfd *abfd, struct dwarf2_cu *cu)
 {
-  /* Reset die reference table; we are
-     building new ones now.  */
-  dwarf2_empty_hash_tables ();
-
   return read_die_and_children (info_ptr, abfd, cu, &info_ptr, NULL);
 }
 
@@ -4682,7 +4834,7 @@ read_die_and_children (char *info_ptr, b
   int has_children;
 
   cur_ptr = read_full_die (&die, abfd, info_ptr, cu, &has_children);
-  store_in_ref_table (die->offset, die);
+  store_in_ref_table (die->offset, die, cu);
 
   if (has_children)
     {
@@ -4761,19 +4913,6 @@ free_die_list (struct die_info *dies)
     }
 }
 
-static void
-do_free_die_list_cleanup (void *dies)
-{
-  free_die_list (dies);
-}
-
-static struct cleanup *
-make_cleanup_free_die_list (struct die_info *dies)
-{
-  return make_cleanup (do_free_die_list_cleanup, dies);
-}
-
-
 /* Read the contents of the section at OFFSET and of size SIZE from the
    object file specified by OBJFILE into the objfile_obstack and return it.  */
 
@@ -5313,27 +5452,16 @@ find_partial_die_in_comp_unit (unsigned 
 /* Find a partial DIE at OFFSET, which may or may not be in CU.  */
 
 static struct partial_die_info *
-find_partial_die (unsigned long offset, struct dwarf2_cu *cu,
-		  struct dwarf2_cu **target_cu)
+find_partial_die (unsigned long offset, struct dwarf2_cu *cu)
 {
   struct dwarf2_per_cu_data *per_cu;
 
   if (offset >= cu->header.offset
       && offset < cu->header.offset + cu->header.length)
-    {
-      *target_cu = cu;
-      return find_partial_die_in_comp_unit (offset, cu);
-    }
+    return find_partial_die_in_comp_unit (offset, cu);
 
   per_cu = dwarf2_find_containing_comp_unit (offset, cu->objfile);
 
-  /* If this offset isn't pointing into a known compilation unit,
-     the debug information is probably corrupted.  */
-  if (per_cu == NULL)
-    error ("Dwarf Error: could not find partial DIE containing "
-	   "offset 0x%lx [in module %s]",
-	   (long) offset, bfd_get_filename (cu->objfile->obfd));
-
   if (per_cu->cu == NULL)
     {
       load_comp_unit (per_cu, cu->objfile);
@@ -5342,7 +5470,6 @@ find_partial_die (unsigned long offset, 
     }
 
   per_cu->cu->last_used = 0;
-  *target_cu = per_cu->cu;
   return find_partial_die_in_comp_unit (offset, per_cu->cu);
 }
 
@@ -5359,11 +5486,10 @@ fixup_partial_die (struct partial_die_in
   if (part_die->name == NULL && part_die->has_specification)
     {
       struct partial_die_info *spec_die;
-      struct dwarf2_cu *spec_cu;
 
-      spec_die = find_partial_die (part_die->spec_offset, cu, &spec_cu);
+      spec_die = find_partial_die (part_die->spec_offset, cu);
 
-      fixup_partial_die (spec_die, spec_cu);
+      fixup_partial_die (spec_die, cu);
 
       if (spec_die->name)
 	{
@@ -5437,6 +5563,38 @@ read_full_die (struct die_info **diep, b
     {
       info_ptr = read_attribute (&die->attrs[i], &abbrev->attrs[i],
 				 abfd, info_ptr, cu);
+
+      /* If this attribute is an absolute reference to a different
+	 compilation unit, make sure that compilation unit is loaded
+	 also.  */
+      if (die->attrs[i].form == DW_FORM_ref_addr
+	  && (DW_ADDR (&die->attrs[i]) < cu->header.offset
+	      || (DW_ADDR (&die->attrs[i])
+		  >= cu->header.offset + cu->header.length)))
+	{
+	  struct dwarf2_per_cu_data *per_cu;
+	  per_cu = dwarf2_find_containing_comp_unit (DW_ADDR (&die->attrs[i]),
+						     cu->objfile);
+
+	  /* Mark the dependence relation so that we don't flush PER_CU
+	     too early.  */
+	  dwarf2_add_dependence (cu, per_cu);
+
+	  /* If it's already on the queue, we have nothing to do.  */
+	  if (per_cu->queued)
+	    continue;
+
+	  /* If the compilation unit is already loaded, just mark it as
+	     used.  */
+	  if (per_cu->cu != NULL)
+	    {
+	      per_cu->cu->last_used = 0;
+	      continue;
+	    }
+
+	  /* Add it to the queue.  */
+	  queue_comp_unit (per_cu);
+       }
     }
 
   *diep = die;
@@ -5533,23 +5691,24 @@ read_attribute_value (struct attribute *
       info_ptr += bytes_read;
       break;
     case DW_FORM_ref1:
-      DW_UNSND (attr) = read_1_byte (abfd, info_ptr);
+      DW_ADDR (attr) = cu->header.offset + read_1_byte (abfd, info_ptr);
       info_ptr += 1;
       break;
     case DW_FORM_ref2:
-      DW_UNSND (attr) = read_2_bytes (abfd, info_ptr);
+      DW_ADDR (attr) = cu->header.offset + read_2_bytes (abfd, info_ptr);
       info_ptr += 2;
       break;
     case DW_FORM_ref4:
-      DW_UNSND (attr) = read_4_bytes (abfd, info_ptr);
+      DW_ADDR (attr) = cu->header.offset + read_4_bytes (abfd, info_ptr);
       info_ptr += 4;
       break;
     case DW_FORM_ref8:
-      DW_UNSND (attr) = read_8_bytes (abfd, info_ptr);
+      DW_ADDR (attr) = cu->header.offset + read_8_bytes (abfd, info_ptr);
       info_ptr += 8;
       break;
     case DW_FORM_ref_udata:
-      DW_UNSND (attr) = read_unsigned_leb128 (abfd, info_ptr, &bytes_read);
+      DW_ADDR (attr) = (cu->header.offset
+			+ read_unsigned_leb128 (abfd, info_ptr, &bytes_read));
       info_ptr += bytes_read;
       break;
     case DW_FORM_indirect:
@@ -5957,21 +6116,14 @@ dwarf2_attr (struct die_info *die, unsig
   for (i = 0; i < die->num_attrs; ++i)
     {
       if (die->attrs[i].name == name)
-	{
-	  return &die->attrs[i];
-	}
+	return &die->attrs[i];
       if (die->attrs[i].name == DW_AT_specification
 	  || die->attrs[i].name == DW_AT_abstract_origin)
 	spec = &die->attrs[i];
     }
-  if (spec)
-    {
-      struct die_info *ref_die =
-      follow_die_ref (dwarf2_get_ref_die_offset (spec, cu));
 
-      if (ref_die)
-	return dwarf2_attr (ref_die, name, cu);
-    }
+  if (spec)
+    return dwarf2_attr (follow_die_ref (die, spec, cu), name, cu);
 
   return NULL;
 }
@@ -6013,7 +6165,7 @@ die_specification (struct die_info *die,
   if (spec_attr == NULL)
     return NULL;
   else
-    return follow_die_ref (dwarf2_get_ref_die_offset (spec_attr, cu));
+    return follow_die_ref (die, spec_attr, cu);
 }
 
 /* Free the line_header structure *LH, and any arrays and strings it
@@ -6944,7 +7096,6 @@ die_type (struct die_info *die, struct d
   struct type *type;
   struct attribute *type_attr;
   struct die_info *type_die;
-  unsigned int ref;
 
   type_attr = dwarf2_attr (die, DW_AT_type, cu);
   if (!type_attr)
@@ -6953,16 +7104,8 @@ die_type (struct die_info *die, struct d
       return dwarf2_fundamental_type (cu->objfile, FT_VOID, cu);
     }
   else
-    {
-      ref = dwarf2_get_ref_die_offset (type_attr, cu);
-      type_die = follow_die_ref (ref);
-      if (!type_die)
-	{
-	  error ("Dwarf Error: Cannot find referent at offset %d [in module %s]", 
-			  ref, cu->objfile->name);
-	  return NULL;
-	}
-    }
+    type_die = follow_die_ref (die, type_attr, cu);
+
   type = tag_type_to_type (type_die, cu);
   if (!type)
     {
@@ -6982,19 +7125,11 @@ die_containing_type (struct die_info *di
   struct type *type = NULL;
   struct attribute *type_attr;
   struct die_info *type_die = NULL;
-  unsigned int ref;
 
   type_attr = dwarf2_attr (die, DW_AT_containing_type, cu);
   if (type_attr)
     {
-      ref = dwarf2_get_ref_die_offset (type_attr, cu);
-      type_die = follow_die_ref (ref);
-      if (!type_die)
-	{
-	  error ("Dwarf Error: Cannot find referent at offset %d [in module %s]", ref, 
-			  cu->objfile->name);
-	  return NULL;
-	}
+      type_die = follow_die_ref (die, type_attr, cu);
       type = tag_type_to_type (type_die, cu);
     }
   if (!type)
@@ -7007,24 +7142,6 @@ die_containing_type (struct die_info *di
   return type;
 }
 
-#if 0
-static struct type *
-type_at_offset (unsigned int offset, struct dwarf2_cu *cu)
-{
-  struct die_info *die;
-  struct type *type;
-
-  die = follow_die_ref (offset);
-  if (!die)
-    {
-      error ("Dwarf Error: Cannot find type referent at offset %d.", offset);
-      return NULL;
-    }
-  type = tag_type_to_type (die, cu);
-  return type;
-}
-#endif
-
 static struct type *
 tag_type_to_type (struct die_info *die, struct dwarf2_cu *cu)
 {
@@ -7378,21 +7495,12 @@ static struct die_info *
 dwarf2_extension (struct die_info *die, struct dwarf2_cu *cu)
 {
   struct attribute *attr;
-  struct die_info *extension_die;
-  unsigned int ref;
 
   attr = dwarf2_attr (die, DW_AT_extension, cu);
   if (attr == NULL)
     return NULL;
 
-  ref = dwarf2_get_ref_die_offset (attr, cu);
-  extension_die = follow_die_ref (ref);
-  if (!extension_die)
-    {
-      error ("Dwarf Error: Cannot find referent at offset %d.", ref);
-    }
-
-  return extension_die;
+  return follow_die_ref (die, attr, cu);
 }
 
 /* Convert a DIE tag into its string name.  */
@@ -8229,13 +8337,16 @@ dump_die (struct die_info *die)
 	case DW_FORM_block1:
 	  fprintf_unfiltered (gdb_stderr, "block: size %d", DW_BLOCK (&die->attrs[i])->size);
 	  break;
+	case DW_FORM_ref1:
+	case DW_FORM_ref2:
+	case DW_FORM_ref4:
+	  fprintf_unfiltered (gdb_stderr, "constant ref: %ld (adjusted)",
+			      (long) (DW_ADDR (&die->attrs[i])));
+	  break;
 	case DW_FORM_data1:
 	case DW_FORM_data2:
 	case DW_FORM_data4:
 	case DW_FORM_data8:
-	case DW_FORM_ref1:
-	case DW_FORM_ref2:
-	case DW_FORM_ref4:
 	case DW_FORM_udata:
 	case DW_FORM_sdata:
 	  fprintf_unfiltered (gdb_stderr, "constant: %ld", DW_UNSND (&die->attrs[i]));
@@ -8279,22 +8390,16 @@ dump_die_list (struct die_info *die)
 }
 
 static void
-store_in_ref_table (unsigned int offset, struct die_info *die)
+store_in_ref_table (unsigned int offset, struct die_info *die,
+		    struct dwarf2_cu *cu)
 {
   int h;
   struct die_info *old;
 
   h = (offset % REF_HASH_SIZE);
-  old = die_ref_table[h];
+  old = cu->die_ref_table[h];
   die->next_ref = old;
-  die_ref_table[h] = die;
-}
-
-
-static void
-dwarf2_empty_hash_tables (void)
-{
-  memset (die_ref_table, 0, sizeof (die_ref_table));
+  cu->die_ref_table[h] = die;
 }
 
 static unsigned int
@@ -8305,14 +8410,12 @@ dwarf2_get_ref_die_offset (struct attrib
   switch (attr->form)
     {
     case DW_FORM_ref_addr:
-      result = DW_ADDR (attr);
-      break;
     case DW_FORM_ref1:
     case DW_FORM_ref2:
     case DW_FORM_ref4:
     case DW_FORM_ref8:
     case DW_FORM_ref_udata:
-      result = cu->header.offset + DW_UNSND (attr);
+      result = DW_ADDR (attr);
       break;
     default:
       complaint (&symfile_complaints,
@@ -8345,21 +8448,41 @@ dwarf2_get_attr_constant_value (struct a
 }
 
 static struct die_info *
-follow_die_ref (unsigned int offset)
+follow_die_ref (struct die_info *src_die, struct attribute *attr,
+		struct dwarf2_cu *cu)
 {
   struct die_info *die;
+  unsigned int offset;
   int h;
+  struct die_info temp_die;
+  struct dwarf2_cu *target_cu;
+
+  offset = dwarf2_get_ref_die_offset (attr, cu);
+
+  if (DW_ADDR (attr) < cu->header.offset
+      || DW_ADDR (attr) >= cu->header.offset + cu->header.length)
+    {
+      struct dwarf2_per_cu_data *per_cu;
+      per_cu = dwarf2_find_containing_comp_unit (DW_ADDR (attr),
+						 cu->objfile);
+      target_cu = per_cu->cu;
+    }
+  else
+    target_cu = cu;
 
   h = (offset % REF_HASH_SIZE);
-  die = die_ref_table[h];
+  die = target_cu->die_ref_table[h];
   while (die)
     {
       if (die->offset == offset)
-	{
-	  return die;
-	}
+	return die;
       die = die->next_ref;
     }
+
+  error ("Dwarf Error: Cannot find DIE at 0x%lx referenced from DIE "
+	 "at 0x%lx [in module %s]",
+	 (long) src_die->offset, (long) offset, cu->objfile->name);
+
   return NULL;
 }
 
@@ -9089,7 +9212,7 @@ dwarf2_symbol_mark_computed (struct attr
 }
 
 /* Locate the compilation unit from CU's objfile which contains the
-   DIE at OFFSET.  Returns NULL on failure.  */
+   DIE at OFFSET.  Raises an error on failure.  */
 
 static struct dwarf2_per_cu_data *
 dwarf2_find_containing_comp_unit (unsigned long offset,
@@ -9098,11 +9221,6 @@ dwarf2_find_containing_comp_unit (unsign
   struct dwarf2_per_cu_data *this_cu;
   int low, high;
 
-  if (dwarf2_per_objfile->all_comp_units == NULL)
-    error ("Dwarf Error: offset 0x%lx points outside this "
-	   "compilation unit [in module %s]",
-	   offset, bfd_get_filename (objfile->obfd));
-
   low = 0;
   high = dwarf2_per_objfile->n_comp_units - 1;
   while (high > low)
@@ -9116,7 +9234,11 @@ dwarf2_find_containing_comp_unit (unsign
   gdb_assert (low == high);
   if (dwarf2_per_objfile->all_comp_units[low]->offset > offset)
     {
-      gdb_assert (low > 0);
+      if (low == 0)
+	error ("Dwarf Error: could not find partial DIE containing "
+	       "offset 0x%lx [in module %s]",
+	       (long) offset, bfd_get_filename (objfile->obfd));
+
       gdb_assert (dwarf2_per_objfile->all_comp_units[low-1]->offset <= offset);
       return dwarf2_per_objfile->all_comp_units[low-1];
     }
@@ -9131,6 +9253,9 @@ dwarf2_find_containing_comp_unit (unsign
     }
 }
 
+/* Locate the compilation unit from OBJFILE which is located at exactly
+   OFFSET.  Raises an error on failure.  */
+
 static struct dwarf2_per_cu_data *
 dwarf2_find_comp_unit (unsigned long offset, struct objfile *objfile)
 {
@@ -9155,6 +9280,8 @@ free_one_comp_unit (void *data)
   cu->per_cu = NULL;
 
   obstack_free (&cu->comp_unit_obstack, NULL);
+  if (cu->dies)
+    free_die_list (cu->dies);
 
   xfree (cu);
 }
@@ -9335,8 +9462,6 @@ set_die_type (struct die_info *die, stru
   **slot = ofs;
 }
 
-#if 0
-
 /* Find the type for DIE in TYPE_HASH, or return NULL if DIE does not
    have a saved type.  */
 
@@ -9372,17 +9497,56 @@ reset_die_and_siblings_types (struct die
     }
 }
 
-#endif
+/* Set the mark field in CU and in every other compilation unit in the
+   cache that we must keep because we are keeping CU.  */
+
+/* Add a dependence relationship from CU to REF_PER_CU.  */
+
+static void
+dwarf2_add_dependence (struct dwarf2_cu *cu,
+		       struct dwarf2_per_cu_data *ref_per_cu)
+{
+  void **slot;
+
+  if (cu->dependencies == NULL)
+    cu->dependencies
+      = htab_create_alloc_ex (5, htab_hash_pointer, htab_eq_pointer,
+			      NULL, &cu->comp_unit_obstack,
+			      hashtab_obstack_allocate,
+			      dummy_obstack_deallocate);
+
+  slot = htab_find_slot (cu->dependencies, ref_per_cu, INSERT);
+  if (*slot == NULL)
+    *slot = ref_per_cu;
+}
 
 /* Set the mark field in CU and in every other compilation unit in the
    cache that we must keep because we are keeping CU.  */
 
+static int
+dwarf2_mark_helper (void **slot, void *data)
+{
+  struct dwarf2_per_cu_data *per_cu;
+
+  per_cu = (struct dwarf2_per_cu_data *) *slot;
+  if (per_cu->cu->mark)
+    return 1;
+  per_cu->cu->mark = 1;
+
+  if (per_cu->cu->dependencies != NULL)
+    htab_traverse (per_cu->cu->dependencies, dwarf2_mark_helper, NULL);
+
+  return 1;
+}
+
 static void
 dwarf2_mark (struct dwarf2_cu *cu)
 {
   if (cu->mark)
     return;
   cu->mark = 1;
+  if (cu->dependencies != NULL)
+    htab_traverse (cu->dependencies, dwarf2_mark_helper, NULL);
 }
 
 static void

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

Daniel Jacobowitz <drow@false.org> writes:
> On Mon, Oct 04, 2004 at 04:14:24PM -0500, Jim Blandy wrote:
> > 
> > Daniel Jacobowitz <drow@false.org> writes:
> > > On Wed, Sep 29, 2004 at 12:49:35PM -0500, Jim Blandy wrote:
> > > > Since we never toss types anyway, would it make sense to move
> > > > type_hash to dwarf2_per_objfile?
> > > 
> > > I don't think so.  type_hash is used in two ways: individual items are
> > > set, when we know which CU we ought to have, and a whole CU is
> > > restored, when we know which CU we're restoring.  It's always more
> > > efficient to have a lot of small hash tables if you know precisely
> > > which one you'll need; fewer collisions.
> > 
> > Really?  libiberty/hashtab.c is a resizing hash table; I thought hash
> > table resizing was supposed to keep the collision rate roughly
> > constant (modulo hysteresis) regardless of the number of elements.  If
> > that's not so, doesn't that mean your hash function isn't doing its
> > job spreading the elements across the (adequately sized) table?
> 
> Poor job of thinking on my part, there.  The rest of the paragraph
> still makes sense to me, though.  For instance, in a resizing hash
> table, I suspect that there is more copying to have one large
> expandable hash table than several small ones.
> 
> I haven't done the math for that, of course.  Maybe I've got it
> backwards.  Do you think there would be any advantage to doing it the
> other way round?

The only advantage I had in mind was simplicity, and it didn't seem
like it'd be a performance hit.

The libiberty hash table expands by a given ratio each time, which
means that, overall, the number of rehashings per element is constant
no matter how large the table gets.  It's similar to the analysis that
shows that ye olde buffer doubling trick ends up being linear time.
(I'm thinking on my own here, not quoting anyone, so be critical...)

There could be a locality disadvantage to doing it all in one big hash
table.  When the time comes to restore a given CU's types, their table
entries will be sharing cache blocks with those of other, irrelevant
CU's.  That doesn't happen if we use for per-CU hash tables: table
entries will never share cache blocks with table entries for other
CU's (assuming the tail of one table doesn't share a block with the
head of another, blah blah blah...).

I'm concerned about the legacy of complexity we'll leave.  Wrinkles
should prove they can pay their own way.  :)

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

Daniel Jacobowitz <drow@false.org> writes:
> Here's a revision which always creates the CU tree.  Retested on
> i686-pc-linux-gnu; no changes.

Looks great.  So the only open issue now is the type hash.

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Daniel Jacobowitz]

On Tue, Oct 05, 2004 at 12:04:26AM -0500, Jim Blandy wrote:
> The only advantage I had in mind was simplicity, and it didn't seem
> like it'd be a performance hit.
> 
> The libiberty hash table expands by a given ratio each time, which
> means that, overall, the number of rehashings per element is constant
> no matter how large the table gets.  It's similar to the analysis that
> shows that ye olde buffer doubling trick ends up being linear time.
> (I'm thinking on my own here, not quoting anyone, so be critical...)
> 
> There could be a locality disadvantage to doing it all in one big hash
> table.  When the time comes to restore a given CU's types, their table
> entries will be sharing cache blocks with those of other, irrelevant
> CU's.  That doesn't happen if we use for per-CU hash tables: table
> entries will never share cache blocks with table entries for other
> CU's (assuming the tail of one table doesn't share a block with the
> head of another, blah blah blah...).
> 
> I'm concerned about the legacy of complexity we'll leave.  Wrinkles
> should prove they can pay their own way.  :)

Then there's only one thing to do... I'll time it.

Using a per-objfile type_hash saves a little memory in overhead, and
probably a little more in hash table size - I didn't instrument memory
use.  But it's definitely slower.  From 1% to 4.3% depending on the
test case.

I believe this happens because we can create the per-comp-unit hash
tables at the correct size - I use a heuristic based on the size of the
CU, although by this point I could use a more accurate one based on the
number of DIEs if I thought it would be worthwhile.  If we create a
per-objfile CU, then we don't get this benefit, so we do a lot of
copying.  There's also the locality benefit.

Also, it saves no code - unless you see something I'm missing, it was
basically s/cu->per_cu->type_hash/dwarf2_per_objfile->type_hash/.  So,
OK with the per-comp-unit hash?

-- 
Daniel Jacobowitz

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

Regarding the type hash:

I've been looking over the code again this morning, and I don't see
that it really makes much of a difference in clarity to make the type
hash per-objfile instead of per-CU.  So I think that last patch you
posted is ready to go in.

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

Daniel Jacobowitz <drow@false.org> writes:
> On Tue, Oct 05, 2004 at 12:04:26AM -0500, Jim Blandy wrote:
> > The only advantage I had in mind was simplicity, and it didn't seem
> > like it'd be a performance hit.
> > 
> > The libiberty hash table expands by a given ratio each time, which
> > means that, overall, the number of rehashings per element is constant
> > no matter how large the table gets.  It's similar to the analysis that
> > shows that ye olde buffer doubling trick ends up being linear time.
> > (I'm thinking on my own here, not quoting anyone, so be critical...)
> > 
> > There could be a locality disadvantage to doing it all in one big hash
> > table.  When the time comes to restore a given CU's types, their table
> > entries will be sharing cache blocks with those of other, irrelevant
> > CU's.  That doesn't happen if we use for per-CU hash tables: table
> > entries will never share cache blocks with table entries for other
> > CU's (assuming the tail of one table doesn't share a block with the
> > head of another, blah blah blah...).
> > 
> > I'm concerned about the legacy of complexity we'll leave.  Wrinkles
> > should prove they can pay their own way.  :)
> 
> Then there's only one thing to do... I'll time it.
> 
> Using a per-objfile type_hash saves a little memory in overhead, and
> probably a little more in hash table size - I didn't instrument memory
> use.  But it's definitely slower.  From 1% to 4.3% depending on the
> test case.
> 
> I believe this happens because we can create the per-comp-unit hash
> tables at the correct size - I use a heuristic based on the size of the
> CU, although by this point I could use a more accurate one based on the
> number of DIEs if I thought it would be worthwhile.  If we create a
> per-objfile CU, then we don't get this benefit, so we do a lot of
> copying.  There's also the locality benefit.
> 
> Also, it saves no code - unless you see something I'm missing, it was
> basically s/cu->per_cu->type_hash/dwarf2_per_objfile->type_hash/.  So,
> OK with the per-comp-unit hash?

Yep, that's fine.  Thanks for persuing it; I'm comfortable with this
now.

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Daniel Jacobowitz]

On Tue, Oct 05, 2004 at 11:11:35AM -0500, Jim Blandy wrote:
> 
> Regarding the type hash:
> 
> I've been looking over the code again this morning, and I don't see
> that it really makes much of a difference in clarity to make the type
> hash per-objfile instead of per-CU.  So I think that last patch you
> posted is ready to go in.

Checked in after final testing on i686-pc-linux-gnu.

To do:
  - NEWS entry
  - Testsuite
  - Hopefully some internals documentation
  - Investigate removing die->type in favor of the hash table

Thanks!

-- 
Daniel Jacobowitz

Re: [rfa/dwarf] Support for attributes pointing to a different CU

[Jim Blandy]

Daniel Jacobowitz <drow@false.org> writes:
> On Tue, Oct 05, 2004 at 11:11:35AM -0500, Jim Blandy wrote:
> > 
> > Regarding the type hash:
> > 
> > I've been looking over the code again this morning, and I don't see
> > that it really makes much of a difference in clarity to make the type
> > hash per-objfile instead of per-CU.  So I think that last patch you
> > posted is ready to go in.
> 
> Checked in after final testing on i686-pc-linux-gnu.

Fantastic!  Thanks very much!