Re: RFC: prototype of C extensions using the Python limited API

[Matthieu Longo via Gdb <gdb@sourceware.org>]

On 10/06/2026 17:10, Andrew Burgess wrote:
> Matthieu Longo <matthieu.longo@arm.com> writes:
> 
>> On 01/06/2026 11:10, Matthieu Longo via Gdb wrote:
>>> On 28/05/2026 17:31, Matthieu Longo via Gdb wrote:
>>>> On 28/05/2026 17:24, Matthieu Longo via Gdb wrote:
>>>>> Hi Tom,
>>>>>
>>>>> As I previously told you (at least, I believe so but I am not 100% sure), I finished migrating all
>>>>> the Python C extensions some time ago. I had discovered some issues with a case of inheritance
>>>>> causing unexpected crashes.
>>>>> To facilitate the debugging, I decided to implement a prototype outside of GDB, and managed to make
>>>>> it work. I finally got the prototype reviewed by Victor Stinner, a Python core developer, to double-
>>>>> check whether my design was working as expected.
>>>>>
>>>>> Please find attached my last iteration. I also provided an archive which should be more convenient
>>>>> for you if you want to compile and test it.
>>>>>
>>>>> Could you please review it so that we validate this new approach before I migrate all my previous
>>>>> patches to it ?
>>>>>
>>>>> PS:
>>>>> - a README is provided with all the needed instruction inside.
>>>>> - this patch contains code from outside GDB's repository, and should not be merged inside GDB's
>>>>> repository.
>>>>>
>>>>> Regards,
>>>>> Matthieu
>>>>
>>>> With the archive, hopefully not inlined.
>>>
>>> Hi Tom,
>>>
>>> Any chance for you to have a look at this in the next two weeks ?
>>>
>>> Matthieu
>>
>> Ping.
>> Adding Andrew too in the loop.
> 
> Would it be possible to give a brief explanation for what this is all
> about?  When reviewing something I usually start by reading the
> description and then check the code matches what was written, but your
> text doesn't really help me ... I suspect there's some back story that
> I'm missing, but it would be neat if that could all be summarised, or at
> least links to it added, here.
> 
> Thanks,
> Andrew
> 

I apologize for the lack of clarity.

This is a prototype proposing an approach to migrate the Python C extensions to the limited API.
I already started the work a long time ago, but had some memory issues (leaks and crashes due to 
double calls to destructors in the inheritance cases). I used this prototype to refine the 
interface, and discuss the issues with Victor Stinner, a CPython core maintainer working on the C API.

This versions of the prototype is now be free from leaks and crashes. You can test this on your side 
by running the example containing different cases (inside pydbg.cpp).
The README provides the instruction to build and run the example.

Before I apply this approach to all the 40+ files in GDB, I would like to discuss it with GDB 
maintainers.

The prototype is organized as follows:
- include/
   - py-ref.hpp: copied from GDB. Addition of a clear() method in gdbpy_ref_policy.
   - py-obj-dict.hpp: I decided to abandon gdbpy_dict_wrapper because the inheritance was causing 
issues, and opted for a new class, gdbpy_dict_helpers, with logic related to the dict management 
gathered in static methods.
   - py-obj-type.hpp:
     + new convenient wrappers for a PyTypeObject: gdbpy_object_type_ref, gdbpy_object_type_bref and 
gdbpy_py_obj_type_bref.
     + gdbpy_heap_type: generic logic for dynamically-allocated types (tp_traverse, tp_clear, 
tp_dealloc)
     + auto_gc_track: this class allows to enforce that PyObject_GC_Track() is always called when 
allocating a object manually. See example in src/ObjWithAlloc.cpp
   - gdb_ref_ptr.hpp: copied from GDB. Addition of a clear() method.
   - gdb-others.hpp, python-traits.hpp: some boiler plates copied from GDB. You can ignore the content.
   - python-internal.hpp: the new gdbpy_type_initialize() used to instantiate and register in the 
module the Python types.

- src/
   - py-obj-type.cpp, gdb-others.cpp: boiler plates already in GDB. You can ignore the content.
   - python-internal.cpp: code for gdbpy_type_initialize
   - my_py_module.hpp, my_py_module.cpp: declaration of the module and classes A, B and ObjWithAlloc.
   - A.cpp, B.cpp: examples with inheritance.
   - ObjWithAlloc.cpp: example with a cycle and manually-allocated object via PyObject_GC_New
- CMakeLists.txt: cmake file to build the project.
- pydbg.cpp: example containing the different use cases.
- README


I don't expect a thorough review of all the code, but a validation of the approach.

Matthieu