From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id LqBDHgUV22KMyxgAWB0awg (envelope-from ) for ; Fri, 22 Jul 2022 17:22:13 -0400 Received: by simark.ca (Postfix, from userid 112) id 6AD411E5EA; Fri, 22 Jul 2022 17:22:13 -0400 (EDT) Authentication-Results: simark.ca; dkim=fail reason="signature verification failed" (768-bit key; unprotected) header.d=tromey.com header.i=@tromey.com header.a=rsa-sha256 header.s=default header.b=BOyBzOan; dkim-atps=neutral X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RDNS_DYNAMIC,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id B13D51E13B for ; Fri, 22 Jul 2022 17:22:12 -0400 (EDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3935F3857C49 for ; Fri, 22 Jul 2022 21:22:12 +0000 (GMT) Received: from gproxy3-pub.mail.unifiedlayer.com (gproxy3-pub.mail.unifiedlayer.com [69.89.30.42]) by sourceware.org (Postfix) with ESMTPS id 99BAA3858405 for ; Fri, 22 Jul 2022 21:21:57 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 99BAA3858405 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=tromey.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=tromey.com Received: from cmgw11.mail.unifiedlayer.com (unknown [10.0.90.126]) by progateway5.mail.pro1.eigbox.com (Postfix) with ESMTP id EEA3D1003F750 for ; Fri, 22 Jul 2022 21:21:45 +0000 (UTC) Received: from box5379.bluehost.com ([162.241.216.53]) by cmsmtp with ESMTP id F05hodwQYq24iF05hofB7U; Fri, 22 Jul 2022 21:21:45 +0000 X-Authority-Reason: nr=8 X-Authority-Analysis: v=2.4 cv=DaLSFthW c=1 sm=1 tr=0 ts=62db14e9 a=ApxJNpeYhEAb1aAlGBBbmA==:117 a=ApxJNpeYhEAb1aAlGBBbmA==:17 a=dLZJa+xiwSxG16/P+YVxDGlgEgI=:19 a=RgO8CyIxsXoA:10:nop_rcvd_month_year a=Qbun_eYptAEA:10:endurance_base64_authed_username_1 a=zstS-IiYAAAA:8 a=Vjrnj6MlIloWe8n4GjgA:9 a=4G6NA9xxw8l3yy4pmD5M:22 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=tromey.com; s=default; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:References :Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=siD2o6h70OLS1TDsoL/Y0zUhQKQd7U5l2tLLKJe88z4=; b=BOyBzOany0dpE78o1olJ9aIQ2l Ec8XyHRBnAVz4ykYhUwsHbZCbzQs6NLT0403ozoS/6Bu94/uH+isIGoHmpKVJGl23fBqtpKI7xa/m jvs3Gjc30owPsP3CNy/LPQrrm; Received: from 71-211-185-228.hlrn.qwest.net ([71.211.185.228]:35556 helo=prentzel) by box5379.bluehost.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oF05h-003Gip-9o; Fri, 22 Jul 2022 15:21:45 -0600 From: Tom Tromey To: Tom Tromey Subject: Re: [PATCH][gdbsupport] Use task size in parallel_for_each References: <20220718194219.GA16823@delia.home> <4fc23fcd-c15d-7622-8b51-cc48cd3cba16@palves.net> <75931310-5dcd-059d-9221-6c94dbcd231f@suse.de> <87leslj786.fsf@tromey.com> X-Attribution: Tom Date: Fri, 22 Jul 2022 15:21:44 -0600 In-Reply-To: <87leslj786.fsf@tromey.com> (Tom Tromey's message of "Fri, 22 Jul 2022 13:08:25 -0600") Message-ID: <87zgh0sv13.fsf@tromey.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - box5379.bluehost.com X-AntiAbuse: Original Domain - sourceware.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - tromey.com X-BWhitelist: no X-Source-IP: 71.211.185.228 X-Source-L: No X-Exim-ID: 1oF05h-003Gip-9o X-Source: X-Source-Args: X-Source-Dir: X-Source-Sender: 71-211-185-228.hlrn.qwest.net (prentzel) [71.211.185.228]:35556 X-Source-Auth: tom+tromey.com X-Email-Count: 1 X-Source-Cap: ZWx5bnJvYmk7ZWx5bnJvYmk7Ym94NTM3OS5ibHVlaG9zdC5jb20= X-Local-Domain: yes X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Pedro Alves , Tom de Vries via Gdb-patches Errors-To: gdb-patches-bounces+public-inbox=simark.ca@sourceware.org Sender: "Gdb-patches" Tom> Then, rather than using parallel_for, the DWARF reader could send N jobs Tom> to the thread pool, and each job would simply take the next available CU Tom> by incrementing an atomic counter. When the counter reached the number Tom> of CUs, a job would stop. Here's a patch. I didn't test it much, though according to "maint time 1", it is ~10% faster on gdb itself. I pushed it as "t/work-stealing" on my github as well, in case you want to try it out. Tom commit 0d530b3a072fd7acd9fc89c755ab4ded2719a17d Author: Tom Tromey Date: Fri Jul 22 15:16:22 2022 -0600 patch diff --git a/gdb/dwarf2/read.c b/gdb/dwarf2/read.c index 42230607fe0..7768d02b4f3 100644 --- a/gdb/dwarf2/read.c +++ b/gdb/dwarf2/read.c @@ -7074,28 +7074,44 @@ dwarf2_build_psymtabs_hard (dwarf2_per_objfile *per_objfile) prompt, which looks weird. */ using result_type = std::pair, std::vector>; - std::vector results - = gdb::parallel_for_each (1, per_bfd->all_comp_units.begin (), - per_bfd->all_comp_units.end (), - [=] (iter_type iter, iter_type end) + const size_t n_units = per_bfd->all_comp_units.size (); + std::vector results (n_units); + + const size_t n_threads + = std::min (gdb::thread_pool::g_thread_pool->thread_count (), n_units); + gdb::future futures[n_threads]; + + std::atomic next_cu = n_threads; + for (size_t i = 0; i < n_threads; ++i) { - std::vector errors; - cooked_index_storage thread_storage; - for (; iter != end; ++iter) + futures[i] = gdb::thread_pool::g_thread_pool->post_task ([&, i] () { - dwarf2_per_cu_data *per_cu = iter->get (); - try - { - process_psymtab_comp_unit (per_cu, per_objfile, - &thread_storage); - } - catch (gdb_exception &except) + std::vector &errors = results[i].second; + cooked_index_storage thread_storage; + + size_t this_cu = i; + while (this_cu < n_units) { - errors.push_back (std::move (except)); + dwarf2_per_cu_data *per_cu = per_bfd->get_cu (this_cu); + try + { + process_psymtab_comp_unit (per_cu, per_objfile, + &thread_storage); + } + catch (gdb_exception &except) + { + errors.push_back (std::move (except)); + } + + this_cu = next_cu++; } - } - return result_type (thread_storage.release (), std::move (errors)); - }); + + results[i].first = thread_storage.release (); + }); + } + + for (size_t i = 0; i < n_threads; ++i) + futures[i].get (); /* Only show a given exception a single time. */ std::unordered_set seen_exceptions;