[PATCH v2 0/2] Add experimental option --enable-py-limited-api

[PATCH v2 0/2] Add experimental option --enable-py-limited-api

[Matthieu Longo]

This patch series introduces an experimental build option, '--enable-py-limited-api', which allows GDB to be compiled against the Python 3 stable C API.

As explained in [1] and [2], using the stable API would make it possible to build GDB against one version of Python 3, while still allowing it to run with other versions available on the user's system.

The migration to the stable API can be done incrementally. These patches do not attempt to port the entire codebase at once. Instead, they provide a development mode where contributors can experiment with the limited API and progressively fix incompatibilities. By default, this option is disabled (--enable-py-limited-api=no).

The series contains two patches:
- Update the Python conftest to avoid use of unstable API calls, ensuring compatibility with the limited API.
- Introduce the --enable-py-limited-api option in GDB's configure.

Tested with Python 3.4 and 3.13.

Diff against revision 1:
- addressed comments of Tom Tromey regarding code formatting, and removal of config.h

Regards,
Matthieu

[1] PR gdb/23830
    https://sourceware.org/bugzilla/show_bug.cgi?id=23830
[2] [RFC] Allowing GDB to use a more recent version of Python at runtime than it was compiled with
    https://inbox.sourceware.org/gdb/314abf0a-007c-457d-bcc3-c28384b9f098@arm.com/


Matthieu Longo (2):
  gdb: make Python conftest compatible with Python limited C API
  gdb: add experimental option --enable-py-limited-api

 gdb/config.in    |  3 +++
 gdb/configure    | 47 ++++++++++++++++++++++++++++++++++++++++++-----
 gdb/configure.ac | 29 ++++++++++++++++++++++++++---
 3 files changed, 71 insertions(+), 8 deletions(-)

-- 
2.51.0

[PATCH v2 1/2] gdb: make Python conftest compatible with Python limited C API

[Matthieu Longo]

The current test to check the support of '--dynamic-list' linker flag
uses PyRun_SimpleString (), which is part of the unstable API. As it is
now, the test will systematically fail due to the undefined symbol
rather than testing the import of ctypes.
This patch replaces PyRun_SimpleString () by an equivalent code relying
on the limited C API, and compatible with Python 3.4.

Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23830
---
 gdb/configure    | 13 ++++++++++---
 gdb/configure.ac | 13 ++++++++++---
 2 files changed, 20 insertions(+), 6 deletions(-)

diff --git a/gdb/configure b/gdb/configure
index db63481b1c6..b7a2079d206 100755
--- a/gdb/configure
+++ b/gdb/configure
@@ -31034,11 +31034,18 @@ else
 int
 main ()
 {
-int err;
+
+          const char *code = "import ctypes\n";
           Py_Initialize ();
-          err = PyRun_SimpleString ("import ctypes\n");
+          PyObject *main_module = PyImport_AddModule ("__main__");
+          PyObject *global_dict = PyModule_GetDict (main_module);
+          PyObject *local_dict = PyDict_New ();
+          PyObject *py_code = Py_CompileString (code, "test", Py_single_input);
+          if (py_code == NULL)
+            return 1;
+          PyObject *res = PyEval_EvalCode (py_code, global_dict, local_dict);
           Py_Finalize ();
-          return err == 0 ? 0 : 1;
+          return res ? 0 : 1;
   ;
   return 0;
 }
diff --git a/gdb/configure.ac b/gdb/configure.ac
index 52924106bca..a88b6ebffe5 100644
--- a/gdb/configure.ac
+++ b/gdb/configure.ac
@@ -1761,11 +1761,18 @@ if test "${gdb_native}" = yes; then
      AC_RUN_IFELSE(
        [AC_LANG_PROGRAM(
          [#include "Python.h"],
-         [int err;
+         [
+          const char *code = "import ctypes\n";
           Py_Initialize ();
-          err = PyRun_SimpleString ("import ctypes\n");
+          PyObject *main_module = PyImport_AddModule ("__main__");
+          PyObject *global_dict = PyModule_GetDict (main_module);
+          PyObject *local_dict = PyDict_New ();
+          PyObject *py_code = Py_CompileString (code, "test", Py_single_input);
+          if (py_code == NULL)
+            return 1;
+          PyObject *res = PyEval_EvalCode (py_code, global_dict, local_dict);
           Py_Finalize ();
-          return err == 0 ? 0 : 1;])],
+          return res ? 0 : 1;])],
        [dynamic_list=true], [], [true])
      LIBS="$old_LIBS"
      CFLAGS="$old_CFLAGS"
-- 
2.51.0

[PATCH v2 2/2] gdb: add experimental option --enable-py-limited-api

[Matthieu Longo]

Today, GDB links against the Python library using the unstable API. This
approach causes portability issues of the generated GDB artifact. Indeed
the built artifact is tighly coupled with the specific version of Python
that it was compiled with. Using a slighly minor version of Python can
cause unpredictable crashes at runtime due to ABI instability between
the Python versions, even minor ones.

The solution would consist in restricting the usage of Python functions
to the limited C API controlled via Py_LIMITED_API that must be defined
before the inclusion of <Python.h>.

This patch does not aim at porting the whole GDB codebase to the Python
limited C API, but rather enabling a development mode where developers
can experiment with the Python limited C API, and fix issues.
This development mode is accessible with the configure option
--enable-py-limited-api which is set by default to 'no'.

Note: the version of the Python limited API is currently set to 3.11
because of PyBuffer_FillInfo and PyBuffer_Release. This choice is not
frozen, and could be reviewed later on depending on newly discovered
issues during the migration.

Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23830
---
 gdb/config.in    |  3 +++
 gdb/configure    | 34 ++++++++++++++++++++++++++++++++--
 gdb/configure.ac | 16 ++++++++++++++++
 3 files changed, 51 insertions(+), 2 deletions(-)

diff --git a/gdb/config.in b/gdb/config.in
index efc3100cb9e..2a2dcb809dd 100644
--- a/gdb/config.in
+++ b/gdb/config.in
@@ -710,6 +710,9 @@
 /* Define if the python directory should be relocated when GDB is moved. */
 #undef PYTHON_PATH_RELOCATABLE
 
+/* Define if GDB should be built against the Python limited C API. */
+#undef Py_LIMITED_API
+
 /* Relocated directory for source files. */
 #undef RELOC_SRCDIR
 
diff --git a/gdb/configure b/gdb/configure
index b7a2079d206..04bb60f9f1f 100755
--- a/gdb/configure
+++ b/gdb/configure
@@ -957,6 +957,7 @@ with_libexpat_prefix
 with_libexpat_type
 with_python
 with_python_libdir
+enable_py_limited_api
 with_guile
 enable_gdb_compile
 enable_source_highlight
@@ -1662,6 +1663,8 @@ Optional Features:
   --enable-gdbtk          enable gdbtk graphical user interface (GUI)
   --enable-profiling      enable profiling of GDB
   --enable-codesign=CERT  sign gdb with 'codesign -s CERT'
+  --enable-py-limited-api enable build against the Python limited C API,
+                          default 'no'
   --enable-gdb-compile    enable support for the compile subsystem, default
                           'yes'
   --enable-source-highlight
@@ -11888,7 +11891,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 11891 "configure"
+#line 11894 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -11994,7 +11997,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 11997 "configure"
+#line 12000 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -28726,6 +28729,33 @@ else
 fi
 
 
+# Check whether to build GDB against Python limited C API.
+# Check whether --enable-py-limited-api was given.
+if test "${enable_py_limited_api+set}" = set; then :
+  enableval=$enable_py_limited_api;
+	   case $enableval in
+	     yes | no)
+	       ;;
+	     *)
+	       as_fn_error $? "bad value $enableval for --enable-py-limited-api" "$LINENO" 5
+	       ;;
+	   esac
+
+else
+  enable_py_limited_api=no
+fi
+
+
+if test "$enable_py_limited_api" == yes; then
+  # The minimal Python limited API version is currently set to 3.11 for the
+  # support of PyBuffer_FillInfo and PyBuffer_Release.
+  # The choice of the minimal version for the Python limited API won't be frozen
+  # until the end of the migration.
+
+$as_echo "#define Py_LIMITED_API 0x030b0000" >>confdefs.h
+
+fi
+
 # -------------------- #
 # Check for libguile.  #
 # -------------------- #
diff --git a/gdb/configure.ac b/gdb/configure.ac
index a88b6ebffe5..5e5a3a01d06 100644
--- a/gdb/configure.ac
+++ b/gdb/configure.ac
@@ -1070,6 +1070,22 @@ AC_SUBST(PYTHON_CPPFLAGS)
 AC_SUBST(PYTHON_LIBS)
 AM_CONDITIONAL(HAVE_PYTHON, test "${have_libpython}" != no)
 
+# Check whether to build GDB against Python limited C API.
+AC_ARG_ENABLE([py-limited-api],
+	      [AS_HELP_STRING([--enable-py-limited-api],
+			      [enable build against the Python limited C API, default 'no'])],
+	      [GDB_CHECK_YES_NO_VAL([$enableval], [--enable-py-limited-api])],
+	      [enable_py_limited_api=no])
+
+if test "$enable_py_limited_api" == yes; then
+  # The minimal Python limited API version is currently set to 3.11 for the
+  # support of PyBuffer_FillInfo and PyBuffer_Release.
+  # The choice of the minimal version for the Python limited API won't be frozen
+  # until the end of the migration.
+  AC_DEFINE(Py_LIMITED_API, 0x030b0000,
+	    [Define if GDB should be built against the Python limited C API.])
+fi
+
 # -------------------- #
 # Check for libguile.  #
 # -------------------- #
-- 
2.51.0

Re: [PATCH v2 0/2] Add experimental option --enable-py-limited-api

[Eli Zaretskii]

> From: Matthieu Longo <matthieu.longo@arm.com>
> CC: Tom Tromey <tom@tromey.com>, Andrew Burgess <aburgess@redhat.com>,
>  Matthieu Longo <matthieu.longo@arm.com>
> Date: Tue, 14 Oct 2025 17:44:56 +0100
> 
> This patch series introduces an experimental build option, '--enable-py-limited-api', which allows GDB to be compiled against the Python 3 stable C API.
> 
> As explained in [1] and [2], using the stable API would make it possible to build GDB against one version of Python 3, while still allowing it to run with other versions available on the user's system.
> 
> The migration to the stable API can be done incrementally. These patches do not attempt to port the entire codebase at once. Instead, they provide a development mode where contributors can experiment with the limited API and progressively fix incompatibilities. By default, this option is disabled (--enable-py-limited-api=no).
> 
> The series contains two patches:
> - Update the Python conftest to avoid use of unstable API calls, ensuring compatibility with the limited API.
> - Introduce the --enable-py-limited-api option in GDB's configure.
> 
> Tested with Python 3.4 and 3.13.
> 
> Diff against revision 1:
> - addressed comments of Tom Tromey regarding code formatting, and removal of config.h

Thanks.  Should this be in NEWS?

Also, I'm guessing that this change will not allow to run the MinGW
build of GDB with versions of Python other than the one against which
GDB was compiled, because AFAIK linking against shared libraries on
Windows records that library in the executable, and GDB will therefore
refuse to run when that DLL cannot be found.  So, for example, if GDB
was linked against python34.dll, it will insist on finding it when
Windows loads the executable, and if not found, will not run.

If we want this to work on Windows, we need to load the Python DLL at
runtime, using the likes of dlopen, and then manually import the APIs
we need using dladdr.  Which AFAIU, this patch doesn't do.

Apologies if I'm missing something, and thus make no sense.

Re: [PATCH v2 0/2] Add experimental option --enable-py-limited-api

[Matthieu Longo]

On 2025-10-14 18:22, Eli Zaretskii wrote:
>> From: Matthieu Longo <matthieu.longo@arm.com>
>> CC: Tom Tromey <tom@tromey.com>, Andrew Burgess <aburgess@redhat.com>,
>>   Matthieu Longo <matthieu.longo@arm.com>
>> Date: Tue, 14 Oct 2025 17:44:56 +0100
>>
>> This patch series introduces an experimental build option, '--enable-py-limited-api', which allows GDB to be compiled against the Python 3 stable C API.
>>
>> As explained in [1] and [2], using the stable API would make it possible to build GDB against one version of Python 3, while still allowing it to run with other versions available on the user's system.
>>
>> The migration to the stable API can be done incrementally. These patches do not attempt to port the entire codebase at once. Instead, they provide a development mode where contributors can experiment with the limited API and progressively fix incompatibilities. By default, this option is disabled (--enable-py-limited-api=no).
>>
>> The series contains two patches:
>> - Update the Python conftest to avoid use of unstable API calls, ensuring compatibility with the limited API.
>> - Introduce the --enable-py-limited-api option in GDB's configure.
>>
>> Tested with Python 3.4 and 3.13.
>>
>> Diff against revision 1:
>> - addressed comments of Tom Tromey regarding code formatting, and removal of config.h
> 
> Thanks.  Should this be in NEWS?
> 

For now, this patch only provides the first brick of a migration of GDB 
to the Python limited API, i.e. an experimentation flag to enable 
developers who want to contribute to the migration to rebuild GDB easily 
with the limited API enabled.
There remains much to be done. Consequently, I don't think that there is 
much to write in NEWS today.

> Also, I'm guessing that this change will not allow to run the MinGW
> build of GDB with versions of Python other than the one against which
> GDB was compiled, because AFAIK linking against shared libraries on
> Windows records that library in the executable, and GDB will therefore
> refuse to run when that DLL cannot be found.  So, for example, if GDB
> was linked against python34.dll, it will insist on finding it when
> Windows loads the executable, and if not found, will not run.
> 
> If we want this to work on Windows, we need to load the Python DLL at
> runtime, using the likes of dlopen, and then manually import the APIs
> we need using dladdr.  Which AFAIU, this patch doesn't do.
> 

This issue is discussed in [2], section "Shared library naming issue in 
CPython" (see cover letter). The proposed approach would rely on 
python3.dll, but it can be reviewed later on if it becomes a blocker.

> Apologies if I'm missing something, and thus make no sense.

No worry.

Matthieu

Re: [PATCH v2 0/2] Add experimental option --enable-py-limited-api

[Tom Tromey]

>>>>> "Matthieu" == Matthieu Longo <matthieu.longo@arm.com> writes:

Matthieu> This patch series introduces an experimental build option,
Matthieu> '--enable-py-limited-api', which allows GDB to be compiled
Matthieu> against the Python 3 stable C API.

Thanks, these look good to me.

Approved-By: Tom Tromey <tom@tromey.com>

Tom

[PATCH] Drop bashism from configure script

[Kévin Le Gouguec]

Results of evaluating 'test "no" == yes' in non-Bash shells range from
unfortunate (dash: "test: no: unexpected operator") to comically wrong
(AdaCore's gsh: returns 0).
---

FTR: formatted with -- ':!gdb/configure' to honor wiki recommendation
to drop generated files from list submission; local commit also
contains the regenerated gdb/configure.

 gdb/configure.ac | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gdb/configure.ac b/gdb/configure.ac
index 5e5a3a01d06..9c258eedc75 100644
--- a/gdb/configure.ac
+++ b/gdb/configure.ac
@@ -1077,7 +1077,7 @@ AC_ARG_ENABLE([py-limited-api],
 	      [GDB_CHECK_YES_NO_VAL([$enableval], [--enable-py-limited-api])],
 	      [enable_py_limited_api=no])
 
-if test "$enable_py_limited_api" == yes; then
+if test "$enable_py_limited_api" = yes; then
   # The minimal Python limited API version is currently set to 3.11 for the
   # support of PyBuffer_FillInfo and PyBuffer_Release.
   # The choice of the minimal version for the Python limited API won't be frozen
-- 
2.43.0

Re: [PATCH] Drop bashism from configure script

[Tom Tromey]

>>>>> "Kévin" == Kévin Le Gouguec <legouguec@adacore.com> writes:

Kévin> Results of evaluating 'test "no" == yes' in non-Bash shells range from
Kévin> unfortunate (dash: "test: no: unexpected operator") to comically wrong
Kévin> (AdaCore's gsh: returns 0).

This is ok.  I'd even say it's obvious.  IIRC this particular problem
has come up before, it's an easy mistake to make.

Approved-By: Tom Tromey <tom@tromey.com>

thanks,
Tom

Re: [PATCH] Drop bashism from configure script

[Kévin Le Gouguec]

Tom Tromey <tom@tromey.com> writes:

>>>>>> "Kévin" == Kévin Le Gouguec <legouguec@adacore.com> writes:
>
> Kévin> Results of evaluating 'test "no" == yes' in non-Bash shells range from
> Kévin> unfortunate (dash: "test: no: unexpected operator") to comically wrong
> Kévin> (AdaCore's gsh: returns 0).
>
> This is ok.  I'd even say it's obvious.  IIRC this particular problem
> has come up before, it's an easy mistake to make.

ACK, next time then I might go a head & push directly, then post here
FTR.

> Approved-By: Tom Tromey <tom@tromey.com>
>
> thanks,

Thank you!

> Tom