Re: [RFC v5 01/18] gdb: introduce expand_symtabs_maybe_overlapping

["Jan Vraný" <Jan.Vrany@labware.com>]

On Tue, 2025-06-24 at 09:22 -0600, Tom Tromey wrote:
> > > > > > "Jan" == Jan Vrany <jan.vrany@labware.com> writes:
> 
> Jan> This commit introduces expand_symtabs_maybe_overlapping, a new
> "quick
> Jan> symbol function".  This will be used later by Python API to
> create new
> Jan> compunits to ensure to-be-created compunit does not overlap with
> Jan> existing one.
> 
> Jan> For DWARF2 base index functions, all compunits are expanded
> Jan> unconditionally since there seem to be no way to extract
> unexpanded
> Jan> compunit's conservative address range.
> 
> This is extremely expensive, so I don't think it's a good approach.
> 
> You can probably call cooked_index::lookup to see if an address is
> covered:
> 
>   /* Look up ADDR in the address map, and return either the
>      corresponding CU, or nullptr if the address could not be
>      found.  */
>   dwarf2_per_cu *lookup (unrelocated_addr addr) override;

Ah, I overlooked that. I have updated this patch (below) to
use m_addrmap in cooked_index_shard to find CUs that need
expanding.

Hopefully it would work, I'm not sure how to test this.

For dwarf2_base_index_functions it still expands all symtabs. Is 
there a way to get memory ranges of CUs for 
dwarf2_base_index_functions?

Thanks!

Jan


-- >8 --
From a8c2506bd37dab4a7bc63f6da4d7945da4714724 Mon Sep 17 00:00:00 2001
From: Jan Vrany <jan.vrany@labware.com>
Date: Mon, 23 Jun 2025 17:07:55 +0100
Subject: [PATCH 02/19] gdb: introduce expand_symtabs_maybe_overlapping

This commit introduces expand_symtabs_maybe_overlapping, a new "quick
symbol function".  This will be used later by Python API to create new
compunits to ensure to-be-created compunit does not overlap with
existing one.

For DWARF2 base index functions, all compunits are expanded
unconditionally since there seem to be no way to extract unexpanded
compunit's conservative address range.

For "cooked index" functions, expand_symtabs_maybe_overlapping () makes
use of address map in cooked_index to find overlapping CUs and expands
only those. This required to export dw2_instantiate_symtab ().
---
 gdb/dwarf2/cooked-index-shard.c | 34 ++++++++++++++++++++++++++++++++
 gdb/dwarf2/cooked-index-shard.h |  5 +++++
 gdb/dwarf2/cooked-index.c       | 17 ++++++++++++++++
 gdb/dwarf2/cooked-index.h       | 24 ++++++++++++++++++++++
 gdb/dwarf2/read.c               | 18 ++++++++++++-----
 gdb/dwarf2/read.h               | 12 +++++++++++
 gdb/objfiles.h                  |  3 +++
 gdb/psymtab.c                   | 21 ++++++++++++++++++++
 gdb/psymtab.h                   |  3 +++
 gdb/quick-symbol.h              |  5 +++++
 gdb/symfile-debug.c             | 16 +++++++++++++++
 gdbsupport/range.h              | 35 +++++++++++++++++++++++++++++++++
 12 files changed, 188 insertions(+), 5 deletions(-)
 create mode 100644 gdbsupport/range.h

diff --git a/gdb/dwarf2/cooked-index-shard.c b/gdb/dwarf2/cooked-index-
shard.c
index c717bbbf6c0..1ee6077dc20 100644
--- a/gdb/dwarf2/cooked-index-shard.c
+++ b/gdb/dwarf2/cooked-index-shard.c
@@ -329,3 +329,37 @@ cooked_index_shard::find (const std::string &name,
bool completing) const
 				     ? cooked_index_entry::COMPLETE
 				     : cooked_index_entry::MATCH) }));
 }
+
+/* See cooked-index-shard.h.  */
+
+std::vector<dwarf2_per_cu *>
+cooked_index_shard::lookup_overlapping (unrelocated_addr start,
+					unrelocated_addr end)
+  {
+    std::vector<dwarf2_per_cu *> result;
+
+    CORE_ADDR start_addr = (CORE_ADDR)start;
+    CORE_ADDR end_addr = (CORE_ADDR)end;
+    CORE_ADDR prev_addr = 0;
+    void *prev_obj = nullptr;
+
+    if (m_addrmap != nullptr)
+      m_addrmap->foreach  ([&] (CORE_ADDR curr_addr, void *curr_obj) -
> int
+      {
+	if (curr_addr >= end_addr)
+	  return 1;
+
+	if (curr_addr != 0
+	    && prev_obj != nullptr
+	    && ranges_overlap (start_addr, end_addr, prev_addr,
curr_addr-1))
+	  {
+	    result.push_back (static_cast<dwarf2_per_cu *>
(prev_obj));
+	  }
+	prev_addr = curr_addr;
+	prev_obj = curr_obj;
+
+	return 0;
+      });
+
+    return result;
+  }
diff --git a/gdb/dwarf2/cooked-index-shard.h b/gdb/dwarf2/cooked-index-
shard.h
index 925960b7ece..62244ac5d3e 100644
--- a/gdb/dwarf2/cooked-index-shard.h
+++ b/gdb/dwarf2/cooked-index-shard.h
@@ -26,6 +26,7 @@
 #include "addrmap.h"
 #include "gdbsupport/iterator-range.h"
 #include "gdbsupport/string-set.h"
+#include "gdbsupport/range.h"
 
 /* An index of interesting DIEs.  This is "cooked", in contrast to a
    mapped .debug_names or .gdb_index, which are "raw".  An entry in
@@ -92,6 +93,10 @@ class cooked_index_shard
     return (static_cast<dwarf2_per_cu *> (m_addrmap->find ((CORE_ADDR)
addr)));
   }
 
+  /* Look up all CUs that overlap with range [START, END).  */
+  std::vector<dwarf2_per_cu *> lookup_overlapping (unrelocated_addr
start,
+						   unrelocated_addr
end);
+
   /* Create a new cooked_index_entry and register it with this object.
      Entries are owned by this object.  The new item is returned.  */
   cooked_index_entry *create (sect_offset die_offset,
diff --git a/gdb/dwarf2/cooked-index.c b/gdb/dwarf2/cooked-index.c
index 7232f286a40..0770eb797f2 100644
--- a/gdb/dwarf2/cooked-index.c
+++ b/gdb/dwarf2/cooked-index.c
@@ -144,6 +144,23 @@ cooked_index::lookup (unrelocated_addr addr)
 
 /* See cooked-index.h.  */
 
+std::vector<dwarf2_per_cu *>
+cooked_index::lookup_overlapping (unrelocated_addr start,
+				  unrelocated_addr end)
+{
+  /* Ensure that the address maps are ready.  */
+  wait (cooked_state::MAIN_AVAILABLE, true);
+  std::vector<dwarf2_per_cu *> result;
+  for (const auto &shard : m_shards)
+    {
+      for (dwarf2_per_cu *cu : shard->lookup_overlapping (start, end))
+	result.push_back (cu);
+    }
+  return result;
+}
+
+/* See cooked-index.h.  */
+
 std::vector<const addrmap *>
 cooked_index::get_addrmaps ()
 {
diff --git a/gdb/dwarf2/cooked-index.h b/gdb/dwarf2/cooked-index.h
index 43b27232aec..605112f63fd 100644
--- a/gdb/dwarf2/cooked-index.h
+++ b/gdb/dwarf2/cooked-index.h
@@ -132,6 +132,10 @@ class cooked_index : public dwarf_scanner_base
      found.  */
   dwarf2_per_cu *lookup (unrelocated_addr addr) override;
 
+  /* Look up all CUs that overlap with range <START, END).  */
+  std::vector<dwarf2_per_cu *> lookup_overlapping (unrelocated_addr
start,
+						   unrelocated_addr
end);
+
   /* Return a new vector of all the addrmaps used by all the indexes
      held by this object.
 
@@ -237,6 +241,26 @@ struct cooked_index_functions : public
dwarf2_base_index_functions
     dwarf2_base_index_functions::expand_all_symtabs (objfile);
   }
 
+  /* Cooked index's version of expand_symtabs_maybe_overlapping.  See
its
+     definition in the definition of quick_symbol_functions in
symfile.h.  */
+
+  void expand_symtabs_maybe_overlapping (struct objfile *objfile,
+	CORE_ADDR start, CORE_ADDR end) override
+  {
+    cooked_index *index = wait (objfile, true);
+
+    unrelocated_addr unrel_start
+      = unrelocated_addr (start - objfile->text_section_offset ());
+    unrelocated_addr unrel_end
+      = unrelocated_addr (end - objfile->text_section_offset ());
+
+    for (dwarf2_per_cu *cu
+	  : index->lookup_overlapping (unrel_start, unrel_end))
+      {
+	dw2_instantiate_symtab (cu, get_dwarf2_per_objfile (objfile),
false);
+      }
+  }
+
   bool expand_symtabs_matching
     (struct objfile *objfile,
      expand_symtabs_file_matcher file_matcher,
diff --git a/gdb/dwarf2/read.c b/gdb/dwarf2/read.c
index 2f27b7cff4c..981b4ed7bcc 100644
--- a/gdb/dwarf2/read.c
+++ b/gdb/dwarf2/read.c
@@ -1660,12 +1660,9 @@ dw2_do_instantiate_symtab (dwarf2_per_cu
*per_cu,
   per_objfile->age_comp_units ();
 }
 
-/* Ensure that the symbols for PER_CU have been read in. 
DWARF2_PER_OBJFILE is
-   the per-objfile for which this symtab is instantiated.
-
-   Returns the resulting symbol table.  */
+/* See read.h.  */
 
-static struct compunit_symtab *
+compunit_symtab *
 dw2_instantiate_symtab (dwarf2_per_cu *per_cu, dwarf2_per_objfile
*per_objfile,
 			bool skip_partial)
 {
@@ -1926,6 +1923,17 @@ dwarf2_base_index_functions::expand_all_symtabs
(struct objfile *objfile)
     }
 }
 
+/* DWARF2 index's version of expand_symtabs_maybe_overlapping.  See
its definition
+   in the definition of quick_symbol_functions in symfile.h.  */
+
+void
+dwarf2_base_index_functions::expand_symtabs_maybe_overlapping
+  (struct objfile *objfile, CORE_ADDR start, CORE_ADDR end)
+{
+  /* Simply expand all symtabs. I do not know how to extract (if at
all
+     possible) conservative address range from dwarf2_per_cu_data.  */
+  expand_all_symtabs (objfile);
+}
 /* See read.h.  */
 
 bool
diff --git a/gdb/dwarf2/read.h b/gdb/dwarf2/read.h
index a5cfb3144f7..a5cbc1b0ae0 100644
--- a/gdb/dwarf2/read.h
+++ b/gdb/dwarf2/read.h
@@ -1192,6 +1192,9 @@ struct dwarf2_base_index_functions : public
quick_symbol_functions
 
   void expand_all_symtabs (struct objfile *objfile) override;
 
+  virtual void expand_symtabs_maybe_overlapping (struct objfile
*objfile,
+	CORE_ADDR start, CORE_ADDR end) override;
+
   struct compunit_symtab *find_pc_sect_compunit_symtab
     (struct objfile *objfile, bound_minimal_symbol msymbol,
      CORE_ADDR pc, struct obj_section *section, int warn_if_readin)
@@ -1327,4 +1330,13 @@ extern int dwarf2_ranges_read (unsigned offset,
unrelocated_addr *low_return,
 extern file_and_directory &find_file_and_directory (die_info *die,
 						    dwarf2_cu *cu);
 
+/* Ensure that the symbols for PER_CU have been read in. 
DWARF2_PER_OBJFILE is
+   the per-objfile for which this symtab is instantiated.
+
+   Returns the resulting symbol table.  */
+
+extern compunit_symtab *
+dw2_instantiate_symtab (dwarf2_per_cu *per_cu, dwarf2_per_objfile
*per_objfile,
+			bool skip_partial);
+
 #endif /* GDB_DWARF2_READ_H */
diff --git a/gdb/objfiles.h b/gdb/objfiles.h
index 4a3475862a9..0206b49e00c 100644
--- a/gdb/objfiles.h
+++ b/gdb/objfiles.h
@@ -597,6 +597,9 @@ struct objfile : intrusive_list_node<objfile>
      code, e.g., DW_TAG_type_unit for dwarf debug info.  */
   void expand_symtabs_with_fullname (const char *fullname);
 
+  /* See quick_symbol_functions.  */
+  void expand_symtabs_maybe_overlapping (CORE_ADDR start, CORE_ADDR
end);
+
   /* See quick_symbol_functions.  */
   bool expand_symtabs_matching
     (expand_symtabs_file_matcher file_matcher,
diff --git a/gdb/psymtab.c b/gdb/psymtab.c
index 986ef44f6c7..d40afd5877f 100644
--- a/gdb/psymtab.c
+++ b/gdb/psymtab.c
@@ -36,6 +36,7 @@
 #include <algorithm>
 #include <set>
 #include "gdbsupport/buildargv.h"
+#include "gdbsupport/range.h"
 
 static const struct partial_symbol *lookup_partial_symbol
      (struct objfile *, struct partial_symtab *, const
lookup_name_info &,
@@ -731,6 +732,26 @@ psymbol_functions::expand_all_symtabs (struct
objfile *objfile)
     psymtab_to_symtab (objfile, psymtab);
 }
 
+/* Psymtab version of expand_symtabs_maybe_overlapping.  See its
definition in
+   the definition of quick_symbol_functions in symfile.h.  */
+
+void
+psymbol_functions::expand_symtabs_maybe_overlapping
+  (struct objfile *objfile, CORE_ADDR start, CORE_ADDR end)
+{
+  for (partial_symtab *psymtab : partial_symbols (objfile))
+    {
+      if (psymtab->text_low_valid && psymtab->text_high_valid)
+	{
+	  CORE_ADDR text_low = psymtab->text_low (objfile);
+	  CORE_ADDR text_high = psymtab->text_high (objfile);
+
+	  if (ranges_overlap (start, end, text_low, text_high))
+	    psymtab_to_symtab (objfile, psymtab);
+	}
+    }
+}
+
 /* Psymtab version of map_symbol_filenames.  See its definition in
    the definition of quick_symbol_functions in symfile.h.  */
 
diff --git a/gdb/psymtab.h b/gdb/psymtab.h
index ad0b2ff4234..8a8ea738bbd 100644
--- a/gdb/psymtab.h
+++ b/gdb/psymtab.h
@@ -626,6 +626,9 @@ struct psymbol_functions : public
quick_symbol_functions
 
   void expand_all_symtabs (struct objfile *objfile) override;
 
+  virtual void expand_symtabs_maybe_overlapping (struct objfile
*objfile,
+	CORE_ADDR start, CORE_ADDR end) override;
+
   bool expand_symtabs_matching
     (struct objfile *objfile,
      expand_symtabs_file_matcher file_matcher,
diff --git a/gdb/quick-symbol.h b/gdb/quick-symbol.h
index 9db1f1860cb..899f2f22239 100644
--- a/gdb/quick-symbol.h
+++ b/gdb/quick-symbol.h
@@ -130,6 +130,11 @@ struct quick_symbol_functions
   /* Read all symbol tables associated with OBJFILE.  */
   virtual void expand_all_symtabs (struct objfile *objfile) = 0;
 
+  /* Read all symbol tables associated with OBJFILE which may overlap
+     with range [START, END).  */
+  virtual void expand_symtabs_maybe_overlapping (struct objfile
*objfile,
+	CORE_ADDR start, CORE_ADDR end) = 0;
+
   /* Expand all symbol tables in OBJFILE matching some criteria.
 
      If LANG_MATCHER returns false, expansion of the symbol table may
be
diff --git a/gdb/symfile-debug.c b/gdb/symfile-debug.c
index 9c5ce85d26e..900d1bf90e2 100644
--- a/gdb/symfile-debug.c
+++ b/gdb/symfile-debug.c
@@ -343,6 +343,22 @@ objfile::expand_all_symtabs ()
     iter->expand_all_symtabs (this);
 }
 
+/* See objfiles.h.  */
+
+void
+objfile::expand_symtabs_maybe_overlapping (CORE_ADDR start, CORE_ADDR
end)
+{
+  if (debug_symfile)
+    gdb_printf (gdb_stdlog,
+		"qf->expand_symtabs_maybe_overlapping (%s, %s, %s)\n",
+		objfile_debug_name (this),
+		hex_string (start),
+		hex_string (end));
+
+  for (const auto &iter : qf)
+    iter->expand_symtabs_maybe_overlapping (this, start, end);
+}
+
 void
 objfile::expand_symtabs_with_fullname (const char *fullname)
 {
diff --git a/gdbsupport/range.h b/gdbsupport/range.h
new file mode 100644
index 00000000000..b665943cfd3
--- /dev/null
+++ b/gdbsupport/range.h
@@ -0,0 +1,35 @@
+/* Copyright (C) 2025-2025 Free Software Foundation, Inc.
+
+   This file is part of GDB.
+
+   This program is free software; you can redistribute it and/or
modify
+   it under the terms of the GNU General Public License as published
by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see
<http://www.gnu.org/licenses/>.  */
+
+#ifndef GDBSUPPORT_RANGE_H
+#define GDBSUPPORT_RANGE_H
+
+/* Return true if given ranges [AL, AH) and [BL, BH) overlap.  Return
false
+   otherwise.  */
+
+template <typename T>
+bool ranges_overlap (T al, T ah, T bl, T bh)
+{
+  static_assert (std::is_integral<T>::value, "Integral type
required");
+
+  gdb_assert (al <= ah);
+  gdb_assert (bl <= bh);
+
+  return !(ah <= bl || bh <= al);
+}
+
+#endif /* GDBSUPPORT_RANGE_H */
-- 
2.47.2