From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id 4CyJBSEwTGihjAwAWB0awg (envelope-from ) for ; Fri, 13 Jun 2025 10:05:21 -0400 Authentication-Results: simark.ca; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=NPJzL7SG; dkim-atps=neutral Received: by simark.ca (Postfix, from userid 112) id 108481E102; Fri, 13 Jun 2025 10:05:21 -0400 (EDT) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-25) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-9.1 required=5.0 tests=ARC_SIGNED,ARC_VALID,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_VALIDITY_CERTIFIED,RCVD_IN_VALIDITY_RPBL, RCVD_IN_VALIDITY_SAFE autolearn=unavailable autolearn_force=no version=4.0.1 Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id 055331E089 for ; Fri, 13 Jun 2025 10:05:20 -0400 (EDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 58253392EC67 for ; Fri, 13 Jun 2025 14:05:19 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 58253392EC67 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1749823519; bh=RXlcDq7KgZP90OPftsyvj+4bD16CKFdvQm09/oJU7Pk=; h=To:Cc:Subject:In-Reply-To:References:Date:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=NPJzL7SGDFuPEl0Mccbt+I/NEbEjfQP2IlT8AuNZrG1iJ4/YWbHFI1b819R/Nwr/P kwh9xXSWLZSfVVS1CIcvZKfNf8PjPUL5vDfoR1eELa2QK7C1tSNodHc8xGozxkLKCz nxIUXtY34czA4F29lGZMsApY7kM8DmDLiZj6oH7I= Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTP id 0535938DC126 for ; Fri, 13 Jun 2025 14:04:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0535938DC126 ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 0535938DC126 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1749823457; cv=none; b=NdRjyRrxl9EgjIi97CweQLEUaqyjjIvdZBYlqinOvS65zbQTsKZmJe5sj7Nh80Xa5IiMyTeXu/SD0/lu5UUWDQYNZXmIMTrqGnBS5MDmCb7DTtFaUou6JkARJsuU8f36a/3At9N//TxMG7ygM+bsZAe8I7fXZtqHe2zmhJnsm3k= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1749823457; c=relaxed/simple; bh=d7swqBJChdpJMtcOLLRpMmC79mozvLWerO4DFG06Qts=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=IFRf4qjjh3izG/+5QXLczjRKEFu9bC1rFWHma8Gm7DxR38D/oTu2u235bdPtFH8kfshCsXBp2klCdNoMgqpfK1mdxNp5AabyrX6h53oSXZxkin5rmq0m/I49KwrWIX1I38FrOlSZtJcdfluCIDJGdgDDbGAXT8EWVthjjn8H9Jk= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 0535938DC126 Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-445-FG9Y3E7VPYazQ3FMlQHKdg-1; Fri, 13 Jun 2025 10:04:14 -0400 X-MC-Unique: FG9Y3E7VPYazQ3FMlQHKdg-1 X-Mimecast-MFC-AGG-ID: FG9Y3E7VPYazQ3FMlQHKdg_1749823453 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-450eaae2934so18721085e9.2 for ; Fri, 13 Jun 2025 07:04:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749823453; x=1750428253; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=suLxdvvsJDyGuiourCO5l7I7ptDUD8RjVOa5uMh1ZgY=; b=GNTqEvXABHW/IySMm6voK5dTYI0Ic/tWk1+gghRLvxK2ANtTJtCsvT/Vk5ugUz3uAx 3jGN3/hg4Qp5L10iUyPDkw1781STF4IyHIdpQFkSEftcqrI9uD/HGjFBw6trqqsxtKG2 PI8ufvr2oBBFOpVZjfNuHNJCwQopCVT/S2JHqSOhj2fANOq+o3diPS4kLjnCCJsqscKe 75Ucs0yU0+W8iUUVNFzhtl83jpZfGZV0nRUYSQBNxqU6AlNxkC87+5hzQesYHUaugvCY zuYaje/QZfJiE1Bgo/eecssiUQfrrK255veGW4HeQ9aIA7OYnV9gndkaJELbYY15Ywwu Yzjg== X-Gm-Message-State: AOJu0YyPFrkI1Sw8wjZnsJuu6klS/jslYfwTMRpNRpeg1M6JQhYTLjXK uYrihtnBsIhKJATpmg+ImbMCz0XR4D3Fs/hldxf09jqd62YSVe6iv/jGj4/tY2xu+FGs874pwYW rA3b7gHjO12t8fNvYfoZ7xQKWrMRB9KODQvYrkdUFQ7DLUBTgevAF X-Gm-Gg: ASbGncs5IcBjLHj7//4Ctr2TeU8Mq/3jOEvKbUN/Zal4MugrF8qz0/+jx0QMU7e5Uak ST7lcRp8hpdcfFuN0Wnh1hheGMVY4rSOEuAw8ZSUGucEmFaBL27XzO1cj+5TMEUA0l/qpKqi5Fd pb3axGv/AIApYRBwStZKgl3Q+yRiwaaw4iVeI90/N53tfImzs+QEs7hcyrBM8lpoIC1htANnxie UHn1rseZofPYd2IOt09/mJzSuoMFquuWgg/S0ivM9TlXVZeIo1kAmt+QvsktGCIlK4bzkhikeSm PfjAzKpV5YNJA4rdDeNDuJK+i1XZZbFOv1sQitlYmNtSn4c= X-Received: by 2002:a05:600c:4450:b0:453:2066:4a26 with SMTP id 5b1f17b1804b1-45334b26f31mr36773265e9.16.1749823452864; Fri, 13 Jun 2025 07:04:12 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFeOyRKHEEPDt/ebPvBMKOyuxWrdihp3mXikPqtK3G2j4cf1UU3LdKGc5M1uiNHQl8TVkJrAQ== X-Received: by 2002:a05:600c:4450:b0:453:2066:4a26 with SMTP id 5b1f17b1804b1-45334b26f31mr36769915e9.16.1749823450356; Fri, 13 Jun 2025 07:04:10 -0700 (PDT) Received: from localhost (75.226.159.143.dyn.plus.net. [143.159.226.75]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4532e13cfabsm53024235e9.19.2025.06.13.07.04.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Jun 2025 07:04:09 -0700 (PDT) To: Tom Tromey , Matthieu Longo Cc: gdb@sourceware.org, Tamar Christina , Andre Simoes Dias Vieira , Tom Tromey , Simon Marchi , Luis Machado Subject: Re: [RFC] Allowing GDB to use a more recent version of Python at runtime than it was compiled with In-Reply-To: <87ecw7b9f8.fsf@tromey.com> References: <314abf0a-007c-457d-bcc3-c28384b9f098@arm.com> <87ecw7b9f8.fsf@tromey.com> Date: Fri, 13 Jun 2025 15:04:08 +0100 Message-ID: <87h60jn3fr.fsf@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: L3BDNUcHwwRkDAXGqHAjyV1A9MRbrWK2wZkVX9mNkcw_1749823453 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: gdb@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gdb mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Andrew Burgess via Gdb Reply-To: Andrew Burgess Errors-To: gdb-bounces~public-inbox=simark.ca@sourceware.org Sender: "Gdb" Tom Tromey writes: >>>>>> "Matthieu" =3D=3D Matthieu Longo writes: > > Matthieu> ## End goal > Matthieu> Allowing GDB to use a more recent version of Python at runtime = than it > Matthieu> was compiled with. > > FWIW I've looked at this several times over the years. See: > > https://sourceware.org/bugzilla/show_bug.cgi?id=3D23830 > > Matthieu> I got errors about Py_buffer and PyBuffer_Release (part of the = Stable > Matthieu> ABI (including all members) since version 3.11, so changed > Matthieu> Py_LIMITED_API to 0x030b0000): > Matthieu> https://docs.python.org/3/c-api/buffer.html#c.Py_buffer > > This is new since the last time I looked. > > Note that gdb still supports a very old Python by default -- like 3.4 I > think. This is bad but it is hard to change. > > Matthieu> There might be more issues as the build process stopped and did= n't go > Matthieu> over all the files in gdb/python. > > Yeah, there's the readline stuff mentioned in the bug. Maybe this could > be disabled. One idea might be to have a configure flag requesting the > stable API and then just drop this module in this case. The specific issue mentioned in bug 23830: ../../binutils-gdb/gdb/python/py-gdb-readline.c:58:29: error: =E2=80=98_P= yOS_ReadlineTState=E2=80=99 was not declared in this scope Was fixed by this commit: commit 8170efad364aa8e062cba3b722b81aca9eda8cf5 Author: Andrew Burgess Date: Wed Dec 6 16:57:11 2023 +0100 =20 gdb/python: avoid use of _PyOS_ReadlineTState You're welcome :) I tried compiling py-gdb-readline with Py_LIMITED_API defined. With the old (3.7) version of Python I have installed on my machine I'm seeing problems with the following things: Py_buffer type -- added to the stable API in 3.11 PyBuffer_Release function -- likewise added in 3.11 PyMem_RawMalloc function -- added to stable API in 3.13 PyRun_SimpleString function -- This ones harder. So, just for the readline module, the real issue is the PyRun_SimpleString function. At a quick glance, I would guess we need to rewrite this using primitives that are part of the stable API. > > This idea might 'interface' with the 3.4 requirement above. Like, we > can have a fallback mode in the source for people building against older > versions of Python. > > Matthieu> Making the usage of PyTypeObject opaque would consist in transf= orming > Matthieu> the declaration of those PyTypeObjects to a static PyType_Slot = and > Matthieu> PyType_Spec equivalent, and then creating a PyObject* by callin= g > Matthieu> PyType_FromSpec() instead of PyModule_AddObject(), which is, by= the > Matthieu> way, "soft deprecated" since Python 3.13. > > Yes, converting to PyType_FromSpec would be fine. Agreed, and also this is likely the lowest risk change as it's going to be pretty formulaic, and I would expect any issues to show up pretty quickl= y. > > Matthieu> Following the logic of PEP-384, linking against the Python limi= ted C > Matthieu> API means linking against libpython3.so, not libpython3.x.so > > Matthieu> With in the current state of Python packaging, this creates iss= ues at > Matthieu> different stages: > > I don't know what to do about this one. But I guess one idea would be > that gdb could adopt the changes that make sense, and then you could > handle the build-time madness for your own builds. Then this could be > improved in gdb as the upstream Python/distro situation develops. Based on what I wrote above, with some things we use not being part of the stable API until 3.13, and given our desire to support older versions of Python. I think even the Py_LIMITED_API define would want (at least initially) to be optional. So we can adjust the code as much as possible (without raising the current minimum version) to only use stable API calls, that's fine to do for everyone. Then we can have a configure switch to say "enforce stable API", which would define Py_LIMITED_API, and (eventually) would change which Python library we link against 3.x.so vs 3.so. > > Matthieu> - testing. I have had a look at the test coverage of the Python= GDB > Matthieu> API, and it seems ok at a first glance. Please let me know if= you > Matthieu> can see any difficulty with the current coverage. > > How did you test coverage? > > I used to routinely to --coverage builds and test these, but they > haven't been reliable for me for a couple of years now :-( > > Matthieu> - any performance impact caused by this transition that I shoul= d be > Matthieu> aware of. And how did you measure it ? How critical are perfo= rmances > Matthieu> regarding the Python limited C API ? > > I would not worry about this at all. Agreed. Thanks, Andrew