From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id CuBWEOhlHWFUbQAAWB0awg (envelope-from ) for ; Wed, 18 Aug 2021 15:56:24 -0400 Received: by simark.ca (Postfix, from userid 112) id 329931EDFB; Wed, 18 Aug 2021 15:56:24 -0400 (EDT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,MAILING_LIST_MULTI,RDNS_DYNAMIC,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id 606071E4A3 for ; Wed, 18 Aug 2021 15:56:23 -0400 (EDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A4EFA3985C54 for ; Wed, 18 Aug 2021 19:56:22 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A4EFA3985C54 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1629316582; bh=TpOc3+vyyAW5K8srOs/28L4cl4MiDltrMl7qdjENGcI=; h=Subject:To:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=VfPNrDItXI6PQ9pgyYj5dRZ99ZnaJnPQlRELU1Rqb18dMHd89rBep5TT6Gq8L3GXH dR3DqOFlaOmqWNxhSeEJjL25S89IEw8vY0t1TjAvtn31LV2Qw4Ft/nx7cwFm4LdqU8 iYS9BgRjMl7iLFKYhExz5aYBFMwGJYW/Uv70St7s= Received: from eggs.gnu.org (eggs.gnu.org [IPv6:2001:470:142:3::10]) by sourceware.org (Postfix) with ESMTPS id 525053985C54 for ; Wed, 18 Aug 2021 19:55:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 525053985C54 Received: from fencepost.gnu.org ([2001:470:142:3::e]:36120) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mGRfK-0006cn-12 for gdb@sourceware.org; Wed, 18 Aug 2021 15:55:58 -0400 Received: from [160.231.0.90] (port=29911 helo=pdslaptop) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mGRfJ-0000pF-SR for gdb@sourceware.org; Wed, 18 Aug 2021 15:55:57 -0400 Message-ID: <3ba28620390947de2a84cabbd1958852aad97d5e.camel@gnu.org> Subject: Tools to debug multiple cores/processes at the same time? To: gdb@sourceware.org Date: Wed, 18 Aug 2021 15:55:57 -0400 Organization: GNU's Not UNIX! Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.36.5-0ubuntu1 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-BeenThere: gdb@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Paul Smith via Gdb Reply-To: psmith@gnu.org Errors-To: gdb-bounces+public-inbox=simark.ca@sourceware.org Sender: "Gdb" In my environment I'm working on a distributed system where the same process is running on a number of different systems. The way these processes work is that they should all have (approximately) the same set of in-memory content at the same time. Our test environment is set up so that if something "bad" happens, we SIGABRT all the processes on all the systems, so I often end up with 4, 5, 6, or more core files, all from the same process, with approximately the same in-memory content (obviously not identical addresses but the same structure) taken at about the same time. The host systems can be assumed, at least initially, to be identical so no issue with different system libraries etc. What would be really useful is for me to be able to open all these cores at the same time, controlled with a single UI (CLI prompt is fine), and be able to both (a) run commands that manipulate a single core, and also (b) run commands which manipulate all core files at once. That would include, for example, setting convenience variables locally for each core, but then running the same command that might use those variables on every core and getting back the results. I'm not really sure how this might work best: maybe a separate terminal per core file and one "controlling" terminal that I would enter commands into, or even commands run one after the other in the same terminal with some kind of header. Don't know. I can imagine multiple ways it might work. What I'm really wondering is, has anyone heard of something like this, or is there any support (even partial support) for something like this in GDB today? I get that it seems like a somewhat specialized request. Debugging multiple live processes at the same time would also be great, of course.