Finding GDB's Python Libraries

Andrew Burgess
2020-07-06T17:33:49Z

I have been thinking about how we tell GDB where to find the Python
libraries, and I'm currently going around in circles trying to figure
out what the right thing to do is.

Some background reading, some original work to tell GDB about where to
find Python libraries:

https://sourceware.org/legacy-ml/gdb-patches/2010-05/msg00434.html

This allowed GDB to figure out where the Python it was configured with
was located, and from that Python can figure out where its libraries are.

Then there was this:

https://sourceware.org/pipermail/gdb-patches/2020-February/165389.html

which built on the above, but was more flexible in some cross
compiling situations, not requiring Python to actually be installed
into the GDB install tree (just the libraries can be installed).

I recently ran into a situation where I needed to cross-build GDB for
a MinGW target, including Python support.

By writing a support script as described here:

https://sourceware.org/gdb/wiki/CrossCompilingWithPythonSupport

I was able to successfully build a MinGW GDB on Linux. I configured
this GDB something like this:

/project/gdb/src/configure --target=..... --host=i686-w64-mingw32 \
--prefix=/project/gdb/install
--with-python=/project/gdb/x-compile-script.sh
--with-python-libdir=/project/gdb/install/lib/

After building and installing GDB I then copied the Python libraries
and the Python DLL from an unpacked i686-w64-mingw32 Python package
into the GDB install directory. With this done I was able to zip the
GDB install directory and distribute it as needed, it all worked fine,
except....

... I ran into a case where a user had PYTHONHOME set in their
environment, and the Python libraries that this path pointed to were
not compatible with the MinGW Python version that GDB was linking
against.

The motivation for originally setting PYTHONHOME in the environment
have been lost to the mists of time, but this started me thinking.

If we have configured GDB to use a specific set of Python libraries,
even going so far (in this case) to ship the libraries along with GDB,
should we allow PYTHONHOME to override that library selection?

My initial thought was sure, that makes sense, that's what PYTHONHOME
is for. But, what if the user really needed PYTHONHOME to be set in
order to correctly use their installed (nothing to do with GDB) Python
interpreter? In that case they now have a situation, through no fault
of their own, where GDB doesn't "just work".

A quick digression on --with-python and --with-python-libdir:
Currently, even when the --with-python-libdir flag is not passed at
configure time, if configure chooses to use Python, then a setting for
this flag is created within the configure script. As far as GDB is
concerned it is impossible to tell if the user specified
--with-python-libdir, or if configure filled this in for us. I think
this might be important shortly.

Currently within GDB we call Py_SetProgramName. This call tells the
Python library where the Python executable is, and from this path the
Python library figures out where the libraries are. Funnily the value
we pass to Py_SetProgramName is actually built within GDB based on the
location where the libraries are. So,

--with-python-libdir=/usr/lib

Causes us to call:

Py_SetProgramName ("/usr/bin/python")

From which the Python library figures out that the libraries are
located at:

/usr/lib

It doesn't actually matter to the Python library if there isn't a
python at /usr/bin/python, so long as the libraries are where it
expects them all is good. This is why we can do:

--with-python=/usr/bin/python
--with-python-libdir=/project/gdb/install/lib

Even if there is never a /project/gdb/install/bin/python, our the
Python library will still find the libraries correctly.

Setting the program name in this way still allows for PYTHONHOME to
override the location of the libraries. However, we could switch to
using Py_SetPythonHome, which also tells the Python library where to
finds its library files, but doesn't honour PYTHONHOME.

In some cases using Py_SetPythonHome seems like a better choice, but
in other cases not listening to PYTHONHOME feels like a mistake, and
Py_SetProgramName feels like a better choice. The problem I'm
struggling with is what would be a good set of rules for choosing
between these two options?

I initially thought that maybe if the user has specified a
--with-python-libdir that is under the install prefix of GDB then we
should not honour PYTHONPATH (so use Py_SetPythonHome), however, that
doesn't feel strong enough.

What if the user does:

--with-python=/opt/my-special-python/bin/python
--with-python-libdir=/opt/my-special-python/lib

Should they not expect GDB to pick this up in preference to any other
Python?

So then I wondered if we could detect the case where configure has
found the "default" Python on the machine, so, if the user configures
like:

--with-python=yes

Then they will get whatever python configure can find, maybe in this
case, and this case only we should listen to PYTHONHOME, but if the
user has specifically specified a particular version of python in any
location, then we should force GDB to use that python above all
others?

If we did go down this route (or make any use of Py_SetPythonHome)
then I would suggest we have GDB look for a new environment variable
GDB_PYTHONHOME, which would be just like PYTHONHOME, but only for GDB,
and would override the value from --with-python-libdir.

I'd be interested to hear if anyone has any thoughts on this issue.

Thanks,
Andrew