From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id ztO+FVefXWhCqiAAWB0awg (envelope-from ) for ; Thu, 26 Jun 2025 15:28:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=simark.ca; s=mail; t=1750966103; bh=qht6fr0IyXonAhmBaoK5XZ+vtRZLTJyB2JhwQaPQDkQ=; h=Date:Subject:To:Cc:References:From:In-Reply-To:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From; b=fPV4w4thTu4eoaukWkMGx+aZGjVHyrDwSsIMLGYxMfHHepT2Jk0W+ocnV8UEVhRzQ I0het2BFYkoi9u0BVoIbYiSX8dzy0e9/O6TiXXIGrp86sVYNj4RJakbq9ijUxC9fZ5 vcLblNOCeKYCTb9Iu7uUBqcnp8u3BIZEdqb3bzM4= Received: by simark.ca (Postfix, from userid 112) id 4CD6A1E11E; Thu, 26 Jun 2025 15:28:23 -0400 (EDT) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-25) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-9.1 required=5.0 tests=ARC_SIGNED,ARC_VALID,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_VALIDITY_CERTIFIED,RCVD_IN_VALIDITY_RPBL, RCVD_IN_VALIDITY_SAFE autolearn=unavailable autolearn_force=no version=4.0.1 Authentication-Results: simark.ca; dkim=pass (1024-bit key; unprotected) header.d=simark.ca header.i=@simark.ca header.a=rsa-sha256 header.s=mail header.b=Bbte+1xl; dkim=pass (1024-bit key) header.d=simark.ca header.i=@simark.ca header.a=rsa-sha256 header.s=mail header.b=cWBY2Mvg; dkim-atps=neutral Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id BE6901E0C2 for ; Thu, 26 Jun 2025 15:28:22 -0400 (EDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5D9223854A85 for ; Thu, 26 Jun 2025 19:28:22 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5D9223854A85 Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=simark.ca header.i=@simark.ca header.a=rsa-sha256 header.s=mail header.b=Bbte+1xl; dkim=pass (1024-bit key) header.d=simark.ca header.i=@simark.ca header.a=rsa-sha256 header.s=mail header.b=cWBY2Mvg Received: from simark.ca (simark.ca [158.69.221.121]) by sourceware.org (Postfix) with ESMTPS id CD83E385C6DE for ; Thu, 26 Jun 2025 19:27:52 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org CD83E385C6DE Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=simark.ca Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=simark.ca ARC-Filter: OpenARC Filter v1.0.0 sourceware.org CD83E385C6DE Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=158.69.221.121 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1750966072; cv=none; b=WOXjsFeCtKJ8UobyWa5IMCCyH+TUBosiFTJjr43YGwrWxnGhKo0nCv+ArIZr2EuCZ3QdIm4UCkyr5i8zGgw4BmvmEONyg09Fs/VcZiDhk+d5z+aU46/wMvXOHLi5Y5lRttZiQBdfOL9uhiWylNvhxiOFPNGtxTB/Gf/pi0NrGrw= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1750966072; c=relaxed/simple; bh=qht6fr0IyXonAhmBaoK5XZ+vtRZLTJyB2JhwQaPQDkQ=; h=DKIM-Signature:DKIM-Signature:Message-ID:Date:MIME-Version: Subject:To:From; b=w0XvdnOo+qLsI1ZKgNhznVpzuYiLwCEDbYU/ZpLNbPuKutDjF3++fBNNsMe5kQC80j6uJcMjqS9uhG/4pOf0SFQaQoSaXEM+B8xFkfF+oU3D/To7MWhm2wyE7vP8EJwh6tQ+v7toeqfc1F41ZIqoQdiFymPVu9RrGl1t1Ix4Ixg= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org CD83E385C6DE DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=simark.ca; s=mail; t=1750966072; bh=qht6fr0IyXonAhmBaoK5XZ+vtRZLTJyB2JhwQaPQDkQ=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=Bbte+1xlWL9JwpY+5TC4Ak3/BRFSmXFXThBdRvajYec8V5OuYjw2Sdfg3+oxmZ4tU nmBQRCCgC7jMq2ggxysT7cKqS+oz5d9M4TdQEu14reoN5+oz55xlEGPYQ1I1pvWltV gM32k3+reGLP8j4ry2xQWSlzJDePvbWvk9aqmOwM= Received: by simark.ca (Postfix, from userid 112) id 153131E11F; Thu, 26 Jun 2025 15:27:52 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=simark.ca; s=mail; t=1750966070; bh=qht6fr0IyXonAhmBaoK5XZ+vtRZLTJyB2JhwQaPQDkQ=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=cWBY2MvgdGojUzLstw9YSCKpTk0ptBnI3b6AQI6D1lSbtRIz5G26UmP9OzrT1zJCq +Vkz1r62nvsf9+jpVvE9O3rgr33z9k1m4MbzcU0SnrBpl8L5RQ6n2hd9grVFxZRxcE ZIeocN1tpz4dJy3uw4T2POoDMws8+uRmktjaVFbI= Received: from [10.0.0.11] (modemcable238.237-201-24.mc.videotron.ca [24.201.237.238]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPSA id 011AA1E0C2; Thu, 26 Jun 2025 15:27:49 -0400 (EDT) Message-ID: <3a7fcb84-773c-4c08-ac59-23bd6e5a508e@simark.ca> Date: Thu, 26 Jun 2025 15:27:49 -0400 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 3/6] gdbsupport: use dynamic partitioning in gdb::parallel_for_each To: Simon Marchi , Tom Tromey , Simon Marchi Cc: gdb-patches@sourceware.org References: <20250505201548.184917-1-simon.marchi@efficios.com> <20250505201548.184917-3-simon.marchi@efficios.com> <87tt4jjy13.fsf@tromey.com> <46f028fd-14f8-49a5-afc1-d6661d6f84ac@polymtl.ca> <2047e4f3-49d3-4dde-991a-8e22bd80c6ef@polymtl.ca> Content-Language: en-US From: Simon Marchi In-Reply-To: <2047e4f3-49d3-4dde-991a-8e22bd80c6ef@polymtl.ca> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gdb-patches-bounces~public-inbox=simark.ca@sourceware.org On 2025-06-13 16:12, Simon Marchi wrote: > On 6/13/25 3:56 PM, Simon Marchi wrote: >> On 6/13/25 3:22 PM, Simon Marchi wrote: >>> Another idea I had (but didn't try) was to make lambdas receive a >>> "magic" range object, like athis: >>> >>> [&] (gdb::dynamic_range range) >>> { >>> for (dwarf2_per_cu *cu : range) >>> process_unit (cu); >>> } >>> >>> "range" would get work items from the work queue in batches, but yield >>> one at a time. It would reach its end when the work queue is empty. >>> It will be a bit of work to implement, but it would have the advantage >>> that any per-worker state could be right there in the lambda, like we >>> have today. >> >> Ah, I remembered why I didn't do this. It wouldn't play well with how >> minimal_symbol_reader::install() currently uses gdb::parallel_for_each. >> Currently, it receives one contiguous range [start,end). After having >> computed the demangled names and hashes for the whole range, each worker >> locks a mutex and installs the names in a shared hash table. >> >> In my patch, it's similar, but each worker installs the names in the >> share hash table at the end of each small range it receives. >> >> With my "magic" iterator idea, it wouldn't be clear when and how to >> install the names in the shared hash table. > > One way to make this work would be to make the iterator yield batches, > so you would have two levels of iteration: > > [&] (gdb::dynamic_range range) > { > for (gdb::batch batch : range) > { > for (minimal_symbol *msym : batch) > { > // compute demangled name and hash > } > > // lock mutex > > for (minimal_symbol *msym : batch) > { > // install msym in the shared hash table > } > } > } > > Let me know if you see an easier solution to this. I have thought about this problem some more, and I discussed this with my team. I still think that the functor approach is the only one that relatively cleanly meets the various requirements: - be able to run a per-worker initialize step - be able to keep some per-worker data for the duration of the for-each - be able to run a per-worker finalize step Simon