Is it possible to statically link python into gdb?

Is it possible to statically link python into gdb?

[Terry Guo]

Hi there,

I am trying to build a gdb with python support which doesn't depend on
system python at all. I can manage to build python into static library
like libpython2.7.a, then I can see that this static library is linked
into gdb. With command ldd, there is no libpython dependence in final
gdb. So far all good. But when run this gdb on another machine, I ran
into below errors and gdb can't be initiated:

Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]

I am working on Ubuntu x86 machines with gdb from master branch and
Python 2.7. Could some one please help me? Thanks in advance.

BR,
Terry

Re: Is it possible to statically link python into gdb?

[Tim Sander]

Hi Terry
> I am trying to build a gdb with python support which doesn't depend on
> system python at all. I can manage to build python into static library
> like libpython2.7.a, then I can see that this static library is linked
> into gdb. With command ldd, there is no libpython dependence in final
> gdb. So far all good. But when run this gdb on another machine, I ran
> into below errors and gdb can't be initiated:
> 
> Could not find platform independent libraries <prefix>
> Could not find platform dependent libraries <exec_prefix>
> Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
> 
> I am working on Ubuntu x86 machines with gdb from master branch and
> Python 2.7. Could some one please help me? Thanks in advance.
I don't know much about python but might it be that you need the *.py files 
of PYTHONHOME/lib nevertheless to get a running python interpreter?

Best regards
Tim

Re: Is it possible to statically link python into gdb?

[Paul_Koning]

On Jun 23, 2014, at 9:49 AM, Tim Sander <tim@krieglstein.org> wrote:

> Hi Terry
>> I am trying to build a gdb with python support which doesn't depend on
>> system python at all. I can manage to build python into static library
>> like libpython2.7.a, then I can see that this static library is linked
>> into gdb. With command ldd, there is no libpython dependence in final
>> gdb. So far all good. But when run this gdb on another machine, I ran
>> into below errors and gdb can't be initiated:
>> 
>> Could not find platform independent libraries <prefix>
>> Could not find platform dependent libraries <exec_prefix>
>> Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
>> 
>> I am working on Ubuntu x86 machines with gdb from master branch and
>> Python 2.7. Could some one please help me? Thanks in advance.
> I don't know much about python but might it be that you need the *.py files 
> of PYTHONHOME/lib nevertheless to get a running python interpreter?

I assume you’d need at least a couple, for the standard Python startup machinery to work.  Some of that can be suppressed when Python is involved by a command; I assume this can also be done when Python is embedded.  

Note also that a lot of Python modules are not just .py files but also .so files or equivalent.

	paul

Re: Is it possible to statically link python into gdb?

[Terry Guo]

On Mon, Jun 23, 2014 at 11:48 PM,  <Paul_Koning@dell.com> wrote:
>
> On Jun 23, 2014, at 9:49 AM, Tim Sander <tim@krieglstein.org> wrote:
>
>> Hi Terry
>>> I am trying to build a gdb with python support which doesn't depend on
>>> system python at all. I can manage to build python into static library
>>> like libpython2.7.a, then I can see that this static library is linked
>>> into gdb. With command ldd, there is no libpython dependence in final
>>> gdb. So far all good. But when run this gdb on another machine, I ran
>>> into below errors and gdb can't be initiated:
>>>
>>> Could not find platform independent libraries <prefix>
>>> Could not find platform dependent libraries <exec_prefix>
>>> Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
>>>
>>> I am working on Ubuntu x86 machines with gdb from master branch and
>>> Python 2.7. Could some one please help me? Thanks in advance.
>> I don't know much about python but might it be that you need the *.py files
>> of PYTHONHOME/lib nevertheless to get a running python interpreter?
>
> I assume you’d need at least a couple, for the standard Python startup machinery to work.  Some of that can be suppressed when Python is involved by a command; I assume this can also be done when Python is embedded.
>
> Note also that a lot of Python modules are not just .py files but also .so files or equivalent.
>
>         paul

Hi folks,

Very appreciate your help. Indeed the python that is statically linked
into gdb needs other modules to run correctly. I now worked around
this issue by following steps:

1). build and install python 2.7 in dynamic way (produced the
libpython2.7.so instead of libpython2.7.a). This python is installed
to my own path instead of system path.
2). unset the variable PYTHONHOME and then export
PYTHONHOME=MY-OWN-PATH:/usr, then configure, build and install gdb.
3). now copy the gdb to other machine, everything works fine. The only
requirement is user will need to install libpython2.7 in their
machine.

I admit that I have to make a compromise here. I am delivering gdb to
my customer and my expectation is that they can use my gdb with python
support no matter there is python in their system and no matter the
version of their installed python. After a lot of attempts, I think I
can't reach my expectation. I guess I have to live with my above
workaround.

BR,
Terry

OT: Re: Is it possible to statically link python into gdb?

[Tim Sander]

Hi Terry
> Very appreciate your help. Indeed the python that is statically linked
> into gdb needs other modules to run correctly. I now worked around
> this issue by following steps:
> 
> 1). build and install python 2.7 in dynamic way (produced the
> libpython2.7.so instead of libpython2.7.a). This python is installed
> to my own path instead of system path.
> 2). unset the variable PYTHONHOME and then export
> PYTHONHOME=MY-OWN-PATH:/usr, then configure, build and install gdb.
> 3). now copy the gdb to other machine, everything works fine. The only
> requirement is user will need to install libpython2.7 in their
> machine.
> 
> I admit that I have to make a compromise here. I am delivering gdb to
> my customer and my expectation is that they can use my gdb with python
> support no matter there is python in their system and no matter the
> version of their installed python. After a lot of attempts, I think I
> can't reach my expectation. I guess I have to live with my above
> workaround.
Its good you found a workable solution.

I think i have seen your name on the 
https://launchpad.net/gcc-arm-embedded? Is your work on this related? It would 
be really nice to have a python enabled gdb in this toolchain.

I am working on the baremetal plugin for qtcreator. With 3.1, qtcreator has a 
dependecy on a python enabled gdb. It would be really great if there would be 
a ready to use toolchain for this to use for embedded development with 
qtcreator.

Best regards
Tim

Re: OT: Re: Is it possible to statically link python into gdb?

[Terry Guo]

On Wed, Jun 25, 2014 at 4:32 PM, Tim Sander <tim@krieglstein.org> wrote:
> Hi Terry
>> Very appreciate your help. Indeed the python that is statically linked
>> into gdb needs other modules to run correctly. I now worked around
>> this issue by following steps:
>>
>> 1). build and install python 2.7 in dynamic way (produced the
>> libpython2.7.so instead of libpython2.7.a). This python is installed
>> to my own path instead of system path.
>> 2). unset the variable PYTHONHOME and then export
>> PYTHONHOME=MY-OWN-PATH:/usr, then configure, build and install gdb.
>> 3). now copy the gdb to other machine, everything works fine. The only
>> requirement is user will need to install libpython2.7 in their
>> machine.
>>
>> I admit that I have to make a compromise here. I am delivering gdb to
>> my customer and my expectation is that they can use my gdb with python
>> support no matter there is python in their system and no matter the
>> version of their installed python. After a lot of attempts, I think I
>> can't reach my expectation. I guess I have to live with my above
>> workaround.
> Its good you found a workable solution.
>
> I think i have seen your name on the
> https://launchpad.net/gcc-arm-embedded? Is your work on this related? It would
> be really nice to have a python enabled gdb in this toolchain.
>
> I am working on the baremetal plugin for qtcreator. With 3.1, qtcreator has a
> dependecy on a python enabled gdb. It would be really great if there would be
> a ready to use toolchain for this to use for embedded development with
> qtcreator.
>
> Best regards
> Tim

You are right. It's me. Hopefully we will enable this feature in
upcoming 4.9 based release.

BR,
Terry

Re: Is it possible to statically link python into gdb?

[Joel Brobecker]

> I admit that I have to make a compromise here. I am delivering gdb to
> my customer and my expectation is that they can use my gdb with python
> support no matter there is python in their system and no matter the
> version of their installed python. After a lot of attempts, I think I
> can't reach my expectation. I guess I have to live with my above
> workaround.

What we do is statically link Python into GDB, and then distribute
both GDB + Python. As long as the path to Python is inside the
configured prefix, the path to the Python libraries should be
"relocatable", allowing users to install the binary package anywhere
and GDB will always find them.

One thing we also do that's not done in the FSF version of GDB
is the following:

@@ -1680,6 +1680,10 @@ message == an error message without a stack will be printed."),
 #else
   Py_SetProgramName (progname);
 #endif
+
+  /* We override any value that the PYTHONHOME might have, as we want
+     to make sure that we use the Python library that comes with GDB.  */
+  Py_SetPythonHome (ldirname (python_libdir));
 #endif
 
   Py_Initialize ();

Some users have had their own version of Python which may be incompatible
with the version we ship in GDB, so we make sure that within GDB, we use
the Python that we built with.

-- 
Joel

Re: Is it possible to statically link python into gdb?

[Hector Oron]

Hello

2014-06-25 16:36 GMT+02:00 Joel Brobecker <brobecker@adacore.com>:
>> I admit that I have to make a compromise here. I am delivering gdb to
>> my customer and my expectation is that they can use my gdb with python
>> support no matter there is python in their system and no matter the
>> version of their installed python. After a lot of attempts, I think I
>> can't reach my expectation. I guess I have to live with my above
>> workaround.
>
> What we do is statically link Python into GDB, and then distribute
> both GDB + Python. As long as the path to Python is inside the
> configured prefix, the path to the Python libraries should be
> "relocatable", allowing users to install the binary package anywhere
> and GDB will always find them.

Wouldn't it be better if python was loaded as external plugin, that
way user/admin can pick up which python version they want to use along
GDB.

In my use case, when distributing GDB some users want it to still be
using python2, some others want to move forward and start moving, i.e.
pretty printers, etc.. to use python3 instead. How could we get both
kind of users happy? Nowadays I am thinking on providing 2 different
builds per each python version, but maybe this should be better loaded
via plugin framework, which I am unsure if that currently exists on
FSF GDB.

Regards,
-- 
 Héctor Orón  -.. . -... .. .- -.   -.. . ...- . .-.. --- .--. . .-.

Re: Is it possible to statically link python into gdb?

[Joel Brobecker]

> Wouldn't it be better if python was loaded as external plugin, that
> way user/admin can pick up which python version they want to use along
> GDB.

The answer to that question depends on what your constraints are.

-- 
Joel

Re: Is it possible to statically link python into gdb?

[Hector Oron]

Hello,

2014-06-25 17:17 GMT+02:00 Joel Brobecker <brobecker@adacore.com>:
>> Wouldn't it be better if python was loaded as external plugin, that
>> way user/admin can pick up which python version they want to use along
>> GDB.
>
> The answer to that question depends on what your constraints are.

I tried to explain my constraints in second paragraph, as I got users
wanting to use python2 and other python3 with gdb.
Do you know if FSF GDB provides the infrastructure or interfaces to be
able to load python as plugin?

Regards
-- 
 Héctor Orón  -.. . -... .. .- -.   -.. . ...- . .-.. --- .--. . .-.

Re: Is it possible to statically link python into gdb?

[Tom Tromey]

>>>>> "Hector" == Hector Oron <hector.oron@gmail.com> writes:

Hector> Do you know if FSF GDB provides the infrastructure or interfaces to be
Hector> able to load python as plugin?

It doesn't.
It could perhaps be done but there are several difficult issues.

Tom