From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id nMMGDtKLD2gmVwoAWB0awg (envelope-from ) for ; Mon, 28 Apr 2025 10:08:18 -0400 Authentication-Results: simark.ca; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Of212mJB; dkim-atps=neutral Received: by simark.ca (Postfix, from userid 112) id 28E5B1E10E; Mon, 28 Apr 2025 10:08:18 -0400 (EDT) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-25) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID,BAYES_00, DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_SBL_CSS autolearn=ham autolearn_force=no version=4.0.1 Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id E18251E0C0 for ; Mon, 28 Apr 2025 10:08:16 -0400 (EDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 65F223858031 for ; Mon, 28 Apr 2025 14:08:16 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 65F223858031 Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Of212mJB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTP id ABE373858C60 for ; Mon, 28 Apr 2025 14:07:40 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org ABE373858C60 Authentication-Results: sourceware.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org ABE373858C60 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1745849260; cv=none; b=iI2H1YHK8dEVhOEh62CcTsdMaTgGYxm4zbaw7hQq5J3BkH8E5lsDPweK5f2bp1U7SVVDJFkomvNMKPC3K6hb+o4KvZpmxlzHZ6ExxyfdyyTAQhkHvMqhGYKQHHs66FcvQT6kDJkJFB0/l0EcY5a7ToS6qO8amvx851blW+b5ZRM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1745849260; c=relaxed/simple; bh=jN/dEe8/PIwyeOUzqCjeBCX5D3DTKEkauWcBD9MymQ0=; h=DKIM-Signature:Message-ID:Date:MIME-Version:Subject:To:From; b=M8v0G56vdApPqadVMzLTWhVtTvGs/sVmqR3fJH4QR5MDLoBkD1ibqeE8J2CT3IQxkdDFkUQn0Y8FOJUwUvto+dWAFW3Dev191Cc3hEMtW2SsoPtNmUSV6Z1Ac7yFeraDrSNpCF8dYbJ0JpHytfvBu+SCff+dmzr7KHe3I2M2bao= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org ABE373858C60 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1745849260; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3yabOvTrmeiw7NzC+ckPMWy0mLyqN/L1Zsm/ZZLVm6s=; b=Of212mJB3knUzIxBaYZS5C4/wDyu8hltGn+7ynGhIfnX1rmKFUOhaNiYgZEt9JuyT58FX8 4trEQk4eVNqldRtLcpstBnhZovDo6FUsD7V+hPgkUBu+Q/Am4JdC3go9uAwembL3tFZxq3 GokLj8SAs6hoQwUNpeLuJwvBD9OMtoo= Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-84-TZxyn1hzMXyt7JOcoTU72w-1; Mon, 28 Apr 2025 10:07:39 -0400 X-MC-Unique: TZxyn1hzMXyt7JOcoTU72w-1 X-Mimecast-MFC-AGG-ID: TZxyn1hzMXyt7JOcoTU72w_1745849258 Received: by mail-pg1-f197.google.com with SMTP id 41be03b00d2f7-b0f807421c9so2676029a12.0 for ; Mon, 28 Apr 2025 07:07:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745849258; x=1746454058; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3yabOvTrmeiw7NzC+ckPMWy0mLyqN/L1Zsm/ZZLVm6s=; b=n0cQJy3+8Y+8tbvW+nzIEWyVqiQ0HCD1Tw3H/Raiixvlsj/N4/pF3AAYzcmVQMbRRm Bk3EKEC9Cvk6/MCjf9GF45N9eOvVBsh+B/kIHE2QwgB82MxOH1G9PFhxHdMvrWCHg9Xy jx3mqaQk5ih5Pr81oK0D4AU9iDL2bTGA6nUipJJYiFt+KTfneH+SBhlMSdnOYQ104vBd 0rXWXLMDUcCk9a1F83rTaz1kvyKC/3NS4WABcHRtNv8rutYFAP1UZh3ZR+63dFuu7TPg GXqTfbqg2jIb+cWSD0IUSOjxeN1hgUaUTUMo6Aj09D83mwKSTUPlqcOjIecaVJWvhWGl ADfg== X-Forwarded-Encrypted: i=1; AJvYcCU+7sKcCQZgZYB5WQe7rB89Mr1Qs07oy/k7dQakuDtPAPbuwpQUTgDC1EOtriSsPuRtQBhzWM/tASAPag==@sourceware.org X-Gm-Message-State: AOJu0YzHngy5XoBozZJgb8ySKe15LeVvJiKYlx3jEgyCi+DSonYoB47X ij5/dm9t8JF32OtOrHzPRRQkw5n6tODAw6hh86AatAxyKxprZFifSxaRpFzwFLheI+Q7JKfUFfR 4ln1Qtvi7+HxwSLYoHpzheAjjPcdKNREmBmRNufsbkbPYyi6YSahXrZpI87M= X-Gm-Gg: ASbGnctkDLJb/Sf+XlRkEBRAtDbTpMLYluVz0gkVuGPQh4gcrItrjdtZ44XcaIY0Lmy Dtdz+LMz1WD0EU1qnsIZjdGF1mFe3NuKHFLKqYoI4Mr193pBGc6qMuTdQIuLxi5DTbYyrTTrByZ IVJCO8Ru2T2Tvnth1Y0TFlRbdGVfplf+0OJ6FzPY55xSUBCrKImQPa6uYdt6AUqe2aewJ0Vhq7m u5HhA6ByjcfRUnNZE5SzyGzneMezL38MSDVyZ0oIKSMZa/plralkiHIqLWetz9CWSoXC7N8Tgs1 I0ADY/eFpF9x/WVWEa+SnkA= X-Received: by 2002:a05:6a20:3d0c:b0:1f5:80a3:b006 with SMTP id adf61e73a8af0-2045b741217mr17730142637.21.1745849257545; Mon, 28 Apr 2025 07:07:37 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEw6NfnZM233QpezXtY0DYVtaJb5+P764tjgG8bAe9CrDecK0BtbErL0PmOqe1zh4T7faYYuQ== X-Received: by 2002:a05:6a20:3d0c:b0:1f5:80a3:b006 with SMTP id adf61e73a8af0-2045b741217mr17730101637.21.1745849257115; Mon, 28 Apr 2025 07:07:37 -0700 (PDT) Received: from ?IPV6:2804:14d:8084:9a69::1000? ([2804:14d:8084:9a69::1000]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-73e25912faesm8165999b3a.17.2025.04.28.07.07.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 28 Apr 2025 07:07:36 -0700 (PDT) Message-ID: Date: Mon, 28 Apr 2025 11:07:34 -0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 00/28] Search symbols via quick API To: Tom Tromey , gdb-patches@sourceware.org References: <20250402-search-in-psyms-v2-0-ea91704487cb@tromey.com> From: Guinevere Larsen In-Reply-To: <20250402-search-in-psyms-v2-0-ea91704487cb@tromey.com> X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: sRdlndWENKgaL6G6eVKw6aAdkZsSWPwJOG8vc3u8FqY_1745849258 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gdb-patches-bounces~public-inbox=simark.ca@sourceware.org Hi Tromey, I am having an issue with the quick_fns interface, and while it isn't affected by this series directly (as I originally thought it could be), I would appreciate if you could share your expertise on this area. The TL;DR of the issue is: the quick API seems to be caching something and searching objfile B for a symbol can return a symbol with details from objfile A (that was loaded first). I tried looking around for where the cache could be hidden, but couldn't find it. Do you know if where the cache is, if it really exists? Some more context: Now that we have a way to get all solibs (and thus objfiles) that belong to one namespace, so I'm trying to implement search for a symbol inside a specific namespace. To do this, I iterate through objfiles in the selected namespace and use lookup_symbol_in_objfile on all of them. The first issue I ran into was the gdb_bfd_cache making the second objfile's symbols not be read, which I already managed to fix. I know we have correct reading of symbols and addresses if I follow the dwarf reading, so if symbol lookup was correct, everything should be working. Yet, the symbols found through the quick_fns all seem to point to the same inferior address (I'm unsure if they are different symbols, or the same symbol returned through dynamic allocation so pointing to different addresses), meaning that we still effectively only find the first one and return it. I can verify it by either taking the address directly, or setting the variable in one namespace and reading it in another. If you'd like to play around with my changes, I pushed them to the users/guinevere/symbol_in_linker_namespace branch, and you can use the test gdb.base/dlmopen-ns-ids to search for the variable gdb_dlmopen_glob in different namespaces. (gdb.base/dlmopen.exp may not work because I haven't looked into LOC_UNRESOLVED variables yet, only LOC_STATIC). -- Cheers, Guinevere Larsen She/Her/Hers On 4/2/25 8:44 PM, Tom Tromey wrote: > This series changes how symbols are looked up in gdb. > > Currently, symbol lookup is done in two phases. In one phase, gdb > searches all existing symtabs for a symbol. In another phase, gdb > will call expand_symtabs_matching to expand some symtabs. > > Different spots in gdb may order these phases differently -- so some > places will expand symtabs and then search the compunits, but other > places will first search expanded symtabs and then use the > expand_symtabs_matching callback to add further results. > > This approach has a few drawbacks. > > * The double search means that some discrepancies between the indexer > and the full reader go unnoticed. This may arguably be a strength > of the approach, though frequently a carefully crafted test case can > show this as a bug. Essentially, though, some searches only work by > accident. > > * Searching all expanded symtabs means that, as a debug session drags > on, searches will examine many irrelevant symtabs. > > * In some places, the two phases use different code to perform the > actual search, meaning that the results can depend on previous CU > expansion decisions. > > * Similarly, if just a single symbol is needed, then both approaches > are bad: expand-then-search will be unnecessarily inefficient, while > search-then-expand approach means that the result depends on which > CUs happen to have already been expanded. > > This series changes all of this. Now, all symbol lookups are done via > the "quick" API, with the idea being that the final search of the > expanded symtab is done via the expand_symtabs_matching callback. > > This fixes all the issues pointed out above: > > * Only relevant symtabs are searched, because the index only considers > matching symbols. > > * Some discrepancies between the indexer and the full reader show up > as symbol lookup failures. Others (if the indexer thinks a symbol > exists but the full reader does not) will just be inefficient -- but > we could add a verifier for this. > > * Lookups are no longer dependent on symtab expansion state, because > again the indexer is pre-filtering the matches. > > Re-reading the series -- it's been in development for quite a while > and I've already landed ~20 preliminary patches -- shows that there > are still a few cleanups in here that could perhaps have gone in > separately. > > Anyway, the essential change is to make the implementations of > expand_symtabs_matching also call the callback when a symtab has > already been expanded. After this, most of the work is cleaning up > individual callers. > > This change lead to some surprising places. For example, I had to > rewrite the .gdb_index reader, because the work done for > "templates.exp" simply never worked in this mode -- and the test > obscured the problems, a classic case of lookups working by accident. > > I regression tested each patch in this series. Furthermore I > regression tested the series as a whole using the cc-with-debug-names > and cc-with-gdb-index target boards. All of this was done on x86-64 > Fedora 40. > > I think maybe I've separately submitted "Ada import functions not in > index" patch. Anyway it's part of this series now. > > Signed-off-by: Tom Tromey > --- > Changes in v2: > - Rebased for all the other recent DWARF changes > - Link to v1: https://inbox.sourceware.org/gdb-patches/20250311-search-in-psyms-v1-0-d73d9be20983@tromey.com > > --- > Tom Tromey (28): > Add another minor hack to cooked_index_entry::full_name > Change ada_decode to preserve upper-case in some situations > Emit some type declarations in .gdb_index > Ada import functions not in index > Fix index's handling of DW_TAG_imported_declaration > Put all CTF symbols in global scope > Restore "ingestion" of .debug_str when writing .debug_names > Entries from anon-struct.exp not in cooked index > Remove dwarf2_per_cu_data::mark > Have expand_symtabs_matching work for already-expanded CUs > Rewrite the .gdb_index reader > Convert default_collect_symbol_completion_matches_break_on > Convert gdbpy_lookup_static_symbols > Convert ada_add_global_exceptions > Convert ada_language_defn::collect_symbol_completion_matches > Convert ada-lang.c:map_matching_symbols > Remove expand_symtabs_matching > Simplify basic_lookup_transparent_type > Remove objfile::expand_symtabs_for_function > Convert linespec.c:iterate_over_all_matching_symtabs > Simplify block_lookup_symbol_primary > Pass lookup_name_info to block_lookup_symbol_primary > Simplify block_lookup_symbol > Add best_symbol_tracker > Convert lookup_symbol_via_quick_fns > Convert lookup_symbol_in_objfile > Make dw_expand_symtabs_matching_file_matcher static > Remove enter_symbol_lookup > > gdb/ada-lang.c | 221 ++--- > gdb/ada-lang.h | 15 +- > gdb/block.c | 57 +- > gdb/block.h | 27 +- > gdb/cp-support.c | 28 +- > gdb/ctfread.c | 6 +- > gdb/dwarf2/abbrev.c | 7 +- > gdb/dwarf2/abbrev.h | 8 + > gdb/dwarf2/cooked-index-entry.c | 7 + > gdb/dwarf2/cooked-index-shard.c | 13 +- > gdb/dwarf2/cooked-index-shard.h | 7 + > gdb/dwarf2/cooked-index-worker.h | 17 +- > gdb/dwarf2/cooked-indexer.c | 37 +- > gdb/dwarf2/index-write.c | 103 ++- > gdb/dwarf2/read-gdb-index.c | 1268 ++++++----------------------- > gdb/dwarf2/read-gdb-index.h | 14 + > gdb/dwarf2/read.c | 176 ++-- > gdb/dwarf2/read.h | 48 +- > gdb/dwarf2/tag.h | 12 + > gdb/linespec.c | 17 +- > gdb/objfiles.h | 4 - > gdb/psymtab.c | 3 - > gdb/python/py-symbol.c | 29 +- > gdb/symfile-debug.c | 22 - > gdb/symfile.c | 25 - > gdb/symfile.h | 9 - > gdb/symtab.c | 223 ++--- > gdb/symtab.h | 2 +- > gdb/testsuite/gdb.ada/import.exp | 28 +- > gdb/testsuite/gdb.ctf/cross-tu-cyclic.exp | 4 +- > 30 files changed, 861 insertions(+), 1576 deletions(-) > --- > base-commit: 60ac9c60fe5b6dc5a59a30a971c3fad020fecf45 > change-id: 20250311-search-in-psyms-3eb1049177e0 > > Best regards,