From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id wD7FIk3A+WV00g0AWB0awg (envelope-from ) for ; Tue, 19 Mar 2024 12:41:49 -0400 Authentication-Results: simark.ca; dkim=pass (2048-bit key; secure) header.d=freebsd.org header.i=@freebsd.org header.a=rsa-sha256 header.s=dkim header.b=MXwgx93s; dkim-atps=neutral Received: by simark.ca (Postfix, from userid 112) id 8B3C21E0BB; Tue, 19 Mar 2024 12:41:49 -0400 (EDT) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id 7340F1E0AC for ; Tue, 19 Mar 2024 12:41:47 -0400 (EDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 1B0F33858C98 for ; Tue, 19 Mar 2024 16:41:47 +0000 (GMT) Received: from mx2.freebsd.org (mx2.freebsd.org [96.47.72.81]) by sourceware.org (Postfix) with ESMTPS id 8D7233858425 for ; Tue, 19 Mar 2024 16:41:24 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 8D7233858425 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=FreeBSD.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=FreeBSD.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 8D7233858425 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=96.47.72.81 ARC-Seal: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1710866487; cv=pass; b=xOeW4TEE5dEBjNCNHGYAfSkPKDp4G6ya94BDhKaqYOIY/zYNFKZ2ZIUWV+qSZxLd4GAQvLaSUUoqBjixm4s3dODdu37J91mIooaGuegnwSh8Ff/0rRkdatAFk36E658CzaudmMyZFiKHTFFRgH5WpXywqrqM9+oxGsi1/3m+G9k= ARC-Message-Signature: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1710866487; c=relaxed/simple; bh=K6JCnsAaurvOafnoqbQ7FxEK326nWq+BoaYk1rwO8K8=; h=DKIM-Signature:Message-ID:Date:MIME-Version:Subject:To:From; b=ZICk3SZg4aRlRkzX0EqQP0NPPMogJNnV/HCYNySN+9XfdOnaaqR1kd5bmcLZReyJylsPNB2UREtf8NnSg56XOe40tfTKEvZ0FsHDq+CNkHGXCa9x7pZjqIFqOH14lZstat+TSrqtVnq9f6rOtSyN36D0HmFS3Iihs6AtWcrXGJA= ARC-Authentication-Results: i=2; server2.sourceware.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits)) (Client CN "mx1.freebsd.org", Issuer "R3" (verified OK)) by mx2.freebsd.org (Postfix) with ESMTPS id 4TzctS3wRBz434N; Tue, 19 Mar 2024 16:41:20 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TzctR5HYwz4W5C; Tue, 19 Mar 2024 16:41:19 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710866479; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+rY/q+5kUf4iu0zIDmUnqHuuMiya63gdhjCVFtXBYPI=; b=MXwgx93sggi6npQWTB5dM4IoyAFSH1gd6k4OaEzrJ8Y0NeoSvowMrGnwrouOfHVb34JXAV 3Xsb6sIpBuagNpWOSQBUdBilkJPvan6wWiNjTxM8dCHn7klssz61FluJFNPp56Wgf1jgVv YFT9bnTB0gmHSmv+/ArctADCb234j9SAflr9A8pD+SRHbWybWelSXuNfdqyyqlQ8yAMm9/ OU7xiyXD3lYJ0EG5OubQeKxDSwLKY9mGu5k7W/ZsLCRwhJi/YpJ/O7HZvQyux/CePkpk1g oBGn+kewHVQL/GxBSPb7QM7KHgwP9qtcutHspw2Hy9D0AKG6BzfvIDKqdWjCyQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1710866479; a=rsa-sha256; cv=none; b=GtApdo4bVjzt4u4jdxzJuFoMpQYTfLwbWd0oSyXWbC+NS1LRA4E2B3tZiX2LlU709Jk85f Sm8cbitYP16US8H/fkff9OEaHOXMwQYVauI0WNujj+/F924ui+qss/JXqA+FOcobYhYcoI iM2nC2/LK//t0Mg0t1hLA8AdbkdYwKNNDmlvqji1G134pzUzjz0ya4H4fcLgJcGuHPADrJ IMIxGFROqA4R7F7+T5wavdhgUn8I9YnEyqbrsAgqhvD8aNeVZygJJ7P57VEaBdtg+L4vSz cuyWOfJGOGoH4AzepOc2x+xfNXUL7I3u3MbIaYrlddfhYLXPtz4ozvSqPRcIzQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710866479; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+rY/q+5kUf4iu0zIDmUnqHuuMiya63gdhjCVFtXBYPI=; b=tkcupDul9eYE59n2pbN8N/O7HwBpZI04ZDzO3gbn9snkuTO/BfHtUQmHohQ8ZJU8iLICS9 NmWCPvJO/aocADTbHcSG32VVMMGcdWriWQ1dqe/pVIC4jVkRDLqGl4lwdBk1KZYCOzNFkS KtUSXqEmdEKv6tjWmrvVpsjqg2eP9T1ioJH85urvDTN7qLoC5eS8tZKzNKLsqEBfELNpjQ +1Iodo3KzIYqmD1K/vAyQLqDmDOWDeXkAVfAk5iDTTWnQXowr9bdVYm2ohw0LXnX5ypJ0b Y+C6p3Jh7oJRTyv5U2TNtf5Z+6d+HhXyfbZd5w728EaxovXzuEB+ExU77P0rkw== Received: from [IPV6:2601:644:937f:4c50:8511:5903:3902:6538] (unknown [IPv6:2601:644:937f:4c50:8511:5903:3902:6538]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4TzctR2R4Tz15SN; Tue, 19 Mar 2024 16:41:19 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: <843cb5af-a424-4432-9787-63a86053469a@FreeBSD.org> Date: Tue, 19 Mar 2024 09:41:17 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3] Teach GDB to generate sparse core files (PR corefiles/31494) Content-Language: en-US To: Pedro Alves , gdb-patches@sourceware.org Cc: Lancelot Six References: <20240315182705.4064062-1-pedro@palves.net> <1cb2e4f4-f14d-4434-9eb2-b33fdf4bf0bb@palves.net> <269ff31a-9aeb-4293-a4d9-df0f16f12e88@palves.net> From: John Baldwin In-Reply-To: <269ff31a-9aeb-4293-a4d9-df0f16f12e88@palves.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gdb-patches-bounces+public-inbox=simark.ca@sourceware.org On 3/18/24 10:43 AM, Pedro Alves wrote: > On 2024-03-18 13:29, Pedro Alves wrote: >> On 2024-03-15 18:27, Pedro Alves wrote: > >> + /* If we already know we have an all-zero block at the next >> + offset, we can skip calling get_all_zero_block_size for >> + it again. */ >> + if (next_all_zero_block.offset != 0) >> + data_offset += next_all_zero_block.offset; > > Err, all the effort to pass down the size, only to typo and not use it... Sigh. > That last line should be: > > data_offset += next_all_zero_block.size; > > Here's the corrected patch... > > From adb681ce583fa640c4fb6883a827f3ab6b28b1c0 Mon Sep 17 00:00:00 2001 > From: Pedro Alves > Date: Mon, 18 Mar 2024 13:16:10 +0000 > Subject: [PATCH v3] Teach GDB to generate sparse core files (PR > corefiles/31494) > > This commit teaches GDB's gcore command to generate sparse core files > (if supported by the filesystem). > > To create a sparse file, all you have to do is skip writing zeros to > the file, instead lseek'ing-ahead over them. > > The sparse logic is applied when writing the memory sections, as > that's where the bulk of the data and the zeros are. > > The commit also tweaks gdb.base/bigcore.exp to make it exercise > gdb-generated cores in addition to kernel-generated cores. We > couldn't do that before, because GDB's gcore on that test's program > would generate a multi-GB non-sparse core (16GB on my system). > > After this commit, gdb.base/bigcore.exp generates, when testing with > GDB's gcore, a much smaller core file, roughly in line with what the > kernel produces: > > real sizes: > > $ du --hu testsuite/outputs/gdb.base/bigcore/bigcore.corefile.* > 2.2M testsuite/outputs/gdb.base/bigcore/bigcore.corefile.gdb > 2.0M testsuite/outputs/gdb.base/bigcore/bigcore.corefile.kernel > > apparent sizes: > > $ du --hu --apparent-size testsuite/outputs/gdb.base/bigcore/bigcore.corefile.* > 16G testsuite/outputs/gdb.base/bigcore/bigcore.corefile.gdb > 16G testsuite/outputs/gdb.base/bigcore/bigcore.corefile.kernel > > Time to generate the core also goes down significantly. On my machine, I get: > > when writing to an SSD, from 21.0s, down to 8.0s > when writing to an HDD, from 31.0s, down to 8.5s > > The changes to gdb.base/bigcore.exp are smaller than they look at > first sight. It's basically mostly refactoring -- moving most of the > code to a new procedure which takes as argument who should dump the > core, and then calling the procedure twice. I purposedly did not > modernize any of the refactored code in this patch. > > Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31494 > Reviewed-By: Lancelot Six > Reviewed-By: Eli Zaretskii > Change-Id: I2554a6a4a72d8c199ce31f176e0ead0c0c76cff1 Looks ok to me in general. Reviewed-By: John Baldwin > + > + size_t data_offset = 0; > + while (data_offset < size) > + { > + size_t all_zero_block_size > + = get_all_zero_block_size (data + data_offset, size - data_offset); > + if (all_zero_block_size != 0) > + data_offset += all_zero_block_size; > + else I would maybe use a continue here to avoid having to indent the large else block, but either way is fine. > + { > + /* We have some non-zero data to write to file. Find the > + next all-zero block within the data, and only write up to > + it. */ > + > + offset_and_size next_all_zero_block > + = find_next_all_zero_block (data, > + data_offset + SPARSE_BLOCK_SIZE, > + size); > + size_t next_data_offset = (next_all_zero_block.offset == 0 > + ? size > + : next_all_zero_block.offset); > + > + if (!bfd_set_section_contents (obfd, osec, data + data_offset, > + sec_offset + data_offset, > + next_data_offset - data_offset)) > + return false; > + > + data_offset = next_data_offset; > + > + /* If we already know we have an all-zero block at the next > + offset, we can skip calling get_all_zero_block_size for > + it again. */ > + if (next_all_zero_block.offset != 0) > + data_offset += next_all_zero_block.size; > + } > + } > + > + return true; > +} > + -- John Baldwin