Add LIBDIR/gdb/python-<version> to Python search path

Add LIBDIR/gdb/python-<version> to Python search path

[Joel Brobecker]

Hello,

I wrote the following two patches in order to allow GDB's python
interpreter to import the modules provided by the python library
from a location that's relative to GDB itself.

The intent is to allow us to package up a GDB binary with python support.
Since we do not know whether the target system on which GDB is going to
be installed has the correct version of python (or even if it has, whether
it has been built correctly), we want to also package up the python
library as well.  That way, regardless of what the target system provides,
we know the debugger will be using a well-defined version of the python
library.

The location I chose for that is GDB_LIBDIR/python-<version>, where
GDB_LIBDIR is LIBDIR/gdb by default.  So, for instance, with a prefix
set to /my/prefix/dir, a default GDB_LIDIR, and Python version 2.5,
we'd be using: /my/prefix/dir/lib/gdb/python-2.5.

So the first patch introduces --with-gdb-libdir as a new command-line
configure switch.  This allows us to change GDB_LIBDIR to something
different.

The second patch makes use of this new variable by adding the python
lib dir at the start of the python search path.  We can possibly
make it more configurable for the user by allowing him to override
the path to the python library (Eg: --with-gdb-libpythondir), but
I don't think that this is going to bring much.

Any thoughts on the approach? Obviously, this also needs documentation,
but I'd rather have the go-ahead before I start documenting the new
switch...

Thanks,
-- 
Joel

[PATCH 2/2] Add gdb_libdir/python-<version> in front of Python search path.

[Joel Brobecker]

This patch puts gdb_libdir/python-<version> at the beginning of
the python search path.  This allows GDB to use its own copy/version
of the Python library.

2010-05-20  Joel Brobecker  <brobecker@adacore.com>

        * python.c (_initialize_python): Add variable libdir in gdb module.
        Insert gdb_libdir/python-<version> at head of python search path.

Tested on x86_64-linux. No regression.

---
 gdb/python/python.c |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/gdb/python/python.c b/gdb/python/python.c
index 3110328..fa76838 100644
--- a/gdb/python/python.c
+++ b/gdb/python/python.c
@@ -654,6 +654,7 @@ Enables or disables printing of Python stack traces."),
   PyModule_AddStringConstant (gdb_module, "VERSION", (char*) version);
   PyModule_AddStringConstant (gdb_module, "HOST_CONFIG", (char*) host_name);
   PyModule_AddStringConstant (gdb_module, "TARGET_CONFIG", (char*) target_name);
+  PyModule_AddStringConstant (gdb_module, "libdir", (char*) gdb_libdir);
 
   gdbpy_initialize_auto_load ();
   gdbpy_initialize_values ();
@@ -702,6 +703,11 @@ class GdbOutputFile:\n\
 \n\
 sys.stderr = GdbOutputFile()\n\
 sys.stdout = GdbOutputFile()\n\
+\n\
+# Put gdb_libdir/python-<version> in the python search path.  This allows\n\
+# us to provide GDB bundled with its own python runtime (the .py/pyc files).\n\
+sys.path.insert(0, gdb.libdir + \"/python-%d.%d\" % (sys.version_info[0],\n\
+                                                     sys.version_info[1]))\n\
 ");
 
   /* Release the GIL while gdb runs.  */
-- 
1.7.1

Re: [PATCH 2/2] Add gdb_libdir/python-<version> in front of Python search path.

[Jan Kratochvil]

On Thu, 20 May 2010 20:40:03 +0200, Joel Brobecker wrote:
> +\n\
> +# Put gdb_libdir/python-<version> in the python search path.  This allows\n\
> +# us to provide GDB bundled with its own python runtime (the .py/pyc files).\n\
> +sys.path.insert(0, gdb.libdir + \"/python-%d.%d\" % (sys.version_info[0],\n\
> +                                                     sys.version_info[1]))\n\

While I do not mind wrt FSF GDB I find this kind of patch as the schoolbook
example of a distro patch nobody expects to be accepted upstream.

I could push the same way patch for using gcc44 for specific testcases
requiring newer gcc, workaround of Red Hat specific kernel incompatibility or
workaround of upstream ccache bug which you voted for not including in FSF GDB
	http://sourceware.org/ml/gdb-patches/2009-03/msg00026.html


Regards,
Jan

Re: [PATCH 2/2] Add gdb_libdir/python-<version> in front of Python search path.

[Joel Brobecker]

On Thu, May 20, 2010 at 11:28:05PM +0200, Jan Kratochvil wrote:
> On Thu, 20 May 2010 20:40:03 +0200, Joel Brobecker wrote:
> > +\n\
> > +# Put gdb_libdir/python-<version> in the python search path.  This allows\n\
> > +# us to provide GDB bundled with its own python runtime (the .py/pyc files).\n\
> > +sys.path.insert(0, gdb.libdir + \"/python-%d.%d\" % (sys.version_info[0],\n\
> > +                                                     sys.version_info[1]))\n\
> 
> While I do not mind wrt FSF GDB I find this kind of patch as the schoolbook
> example of a distro patch nobody expects to be accepted upstream.

I have to disagree about that ("textbook example" is a little harsh, no?)
- I wouldn't have proposed this idea if I didn't think that it wouldn't
be useful to others. I guess we have to take one step back and decide
whether we want to help users who build binaries like AdaCore does, or
not.  I don't think that this is the first time that someone introduces
a patch that helps in that direction, and I don't think we're doing
a disservice to GDB to adding it either. It's not like it's a complicated
and intrusive change...

I'm going to hold off on that for a day or two, see what everyone
thinks. If it has to be kept inside AdaCore, then so be it.  But it's
clear, just with Dan's answer, that someone else would benefit from
the code. And code sharing means more time to work on other stuff.

> I could push the same way patch for using gcc44 for specific testcases
> requiring newer gcc, workaround of Red Hat specific kernel incompatibility or
> workaround of upstream ccache bug which you voted for not including in FSF GDB
> 	http://sourceware.org/ml/gdb-patches/2009-03/msg00026.html

We can revisit the decision. I don't remember the details, but I feel that
the situation was not the same then.

-- 
Joel

Re: [PATCH 2/2] Add gdb_libdir/python-<version> in front of Python search path.

[Jan Kratochvil]

On Thu, 20 May 2010 23:46:47 +0200, Joel Brobecker wrote:
> I have to disagree about that ("textbook example" is a little harsh, no?)

Adjusting installation paths is the most common distro change, isn't it?
I do not find it as anything wrong, just as the most typical packaging task.


> If it has to be kept inside AdaCore, then so be it.  But it's clear, just
> with Dan's answer, that someone else would benefit from the code.

AdaCore and CodeSourcery seem to have the same distribution model.  Tens
GNU/Linux distributions I met before do not use any relocatable packages.
I do not find this patch as a distro-neutral one.  But np with it.


Regards,
Jan

Re: [PATCH 2/2] Add gdb_libdir/python-<version> in front of Python search path.

[Joel Brobecker]

> > I have to disagree about that ("textbook example" is a little harsh, no?)
> 
> Adjusting installation paths is the most common distro change, isn't it?
> I do not find it as anything wrong, just as the most typical packaging task.
[...]
> AdaCore and CodeSourcery seem to have the same distribution model.  Tens
> GNU/Linux distributions I met before do not use any relocatable packages.
> I do not find this patch as a distro-neutral one.  But np with it.

OK, I think I see where you are coming from.  But the reality of things
is that we cannot all have the same model, because our operating models
are different.  So some changes will have no benefit to certain types
of distributions, but I think that's OK, as long as some people benefit
without hurting the rest, I think it's a win.

-- 
Joel

Re: [PATCH 2/2] Add gdb_libdir/python-<version> in front of Python search path.

[Doug Evans]

[-- Attachment #1: Type: text/plain, Size: 4047 bytes --]

On Thu, May 20, 2010 at 11:40 AM, Joel Brobecker <brobecker@adacore.com> wrote:
> This patch puts gdb_libdir/python-<version> at the beginning of
> the python search path.  This allows GDB to use its own copy/version
> of the Python library.
>
> 2010-05-20  Joel Brobecker  <brobecker@adacore.com>
>
>        * python.c (_initialize_python): Add variable libdir in gdb module.
>        Insert gdb_libdir/python-<version> at head of python search path.
>
> Tested on x86_64-linux. No regression.
>
> ---
>  gdb/python/python.c |    6 ++++++
>  1 files changed, 6 insertions(+), 0 deletions(-)
>
> diff --git a/gdb/python/python.c b/gdb/python/python.c
> index 3110328..fa76838 100644
> --- a/gdb/python/python.c
> +++ b/gdb/python/python.c
> @@ -654,6 +654,7 @@ Enables or disables printing of Python stack traces."),
>   PyModule_AddStringConstant (gdb_module, "VERSION", (char*) version);
>   PyModule_AddStringConstant (gdb_module, "HOST_CONFIG", (char*) host_name);
>   PyModule_AddStringConstant (gdb_module, "TARGET_CONFIG", (char*) target_name);
> +  PyModule_AddStringConstant (gdb_module, "libdir", (char*) gdb_libdir);
>
>   gdbpy_initialize_auto_load ();
>   gdbpy_initialize_values ();
> @@ -702,6 +703,11 @@ class GdbOutputFile:\n\
>  \n\
>  sys.stderr = GdbOutputFile()\n\
>  sys.stdout = GdbOutputFile()\n\
> +\n\
> +# Put gdb_libdir/python-<version> in the python search path.  This allows\n\
> +# us to provide GDB bundled with its own python runtime (the .py/pyc files).\n\
> +sys.path.insert(0, gdb.libdir + \"/python-%d.%d\" % (sys.version_info[0],\n\
> +                                                     sys.version_info[1]))\n\
>  ");
>
>   /* Release the GIL while gdb runs.  */
> --
> 1.7.1
>
>

Hi.
I think(!) this is a better approach.  Could be wrong of course. :-)
I think adding one directory to sys.path is insufficient, and from
what I've read it seems like Py_SetProgramName is the preferred
solution to this problem.

This patch doesn't currently handle your case, where the relationship
between the location of python and gdb at build time is different than
at runtime.  I think the solution to that is another config parameter
to specify where to find python at run time.  Currently, when given a
path, --with-python specifies where to find python at build time, and,
for now, also at runtime.
I can create another patch to add this new option
(--with-python-libdir or some such) if you think it's worth pursuing.
Regardless of whether --with-python-libdir is useful to add to the FSF
tree, I think the attached patch is worthy.

Note that Python's initialization assumes a particular layout that
your current plan violates (I think!).
[ref Modules/getpath.c in the python sources]
/foo/bin/python
/foo/lib/pythonX.Y/...

The attached patch will work with your --with-gdb-libdir, but python
would need to be installed as ${gdb_libdir}/lib/pythonX.Y [or
${gdb_libdir}/foo/lib/pythonX.Y].  The important part is to not
separate "lib" from "pythonX.Y".  [ref: variable lib_python in
Modules/getpath.c]

Tested with:
python configure --prefix=/mumble/lib/gdb
gdb configure --prefix=/mumble --with-python=/mumble/lib/gdb
and then renaming /mumble to /mumble2
and then trying "python import itertools" after starting gdb.

2010-05-21  Doug Evans  <dje@google.com>

        Allow python to find its files if moved from original location.
        * acinclude.m4 (GDB_AC_DEFINE_RELOCATABLE): New function.
        (GDB_AC_WITH_DIR): Call it.
        * configure.ac: If a path is provided to --with-python,
        define WITH_PYTHON_PATH.
        * config.in: Regenerate.
        * configure: Regenerate.
        * defs.h (python_libdir): Declare.
        * main.c (python_libdir): Define.
        (captured_main): Initialize python_libdir.
        * python/python.c (_initialize_python): If configured with
        --with-python=/path, call Py_SetProgramName to make sure python
        can find its libraries and modules.

[-- Attachment #2: gdb-100521-with-python-path-1.patch.txt --]
[-- Type: text/plain, Size: 6579 bytes --]

2010-05-21  Doug Evans  <dje@google.com>

	Allow python to find its files if moved from original location.
	* acinclude.m4 (GDB_AC_DEFINE_RELOCATABLE): New function.
	(GDB_AC_WITH_DIR): Call it.
	* configure.ac: If a path is provided to --with-python,
	define WITH_PYTHON_PATH.
	* config.in: Regenerate.
	* configure: Regenerate.
	* defs.h (python_libdir): Declare.
	* main.c (python_libdir): Define.
	(captured_main): Initialize python_libdir.
	* python/python.c (_initialize_python): If configured with
	--with-python=/path, call Py_SetProgramName to make sure python
	can find its libraries and modules.

Index: acinclude.m4
===================================================================
RCS file: /cvs/src/src/gdb/acinclude.m4,v
retrieving revision 1.35
diff -u -p -r1.35 acinclude.m4
--- acinclude.m4	8 Jan 2010 07:16:43 -0000	1.35
+++ acinclude.m4	21 May 2010 08:04:12 -0000
@@ -403,22 +403,12 @@ AC_DEFUN([CY_AC_TK_PRIVATE_HEADERS], [
   fi
 ])
 
-dnl GDB_AC_WITH_DIR([VARIABLE], [ARG-NAME], [HELP], [DEFAULT])
-dnl Add a new --with option that defines a directory.
-dnl The result is stored in VARIABLE.  AC_DEFINE_DIR is called
-dnl on this variable, as is AC_SUBST.
-dnl ARG-NAME is the base name of the argument (without "--with").
-dnl HELP is the help text to use.
-dnl If the user's choice is relative to the prefix, then the
+dnl GDB_AC_DEFINE_RELOCATABLE([VARIABLE], [ARG-NAME], [SHELL-VARIABLE])
+dnl For use in processing directory values for --with-foo.
+dnl If the path in SHELL_VARIABLE is relative to the prefix, then the
 dnl result is relocatable, then this will define the C macro
 dnl VARIABLE_RELOCATABLE to 1; otherwise it is defined as 0.
-dnl DEFAULT is the default value, which is used if the user
-dnl does not specify the argument.
-AC_DEFUN([GDB_AC_WITH_DIR], [
-  AC_ARG_WITH([$2], AS_HELP_STRING([--with-][$2][=PATH], [$3]), [
-    [$1]=$withval], [[$1]=[$4]])
-  AC_DEFINE_DIR([$1], [$1], [$3])
-  AC_SUBST([$1])
+AC_DEFUN([GDB_AC_DEFINE_RELOCATABLE], [
   if test "x$exec_prefix" = xNONE || test "x$exec_prefix" = 'x${prefix}'; then
      if test "x$prefix" = xNONE; then
      	test_prefix=/usr/local
@@ -429,11 +419,30 @@ AC_DEFUN([GDB_AC_WITH_DIR], [
      test_prefix=$exec_prefix
   fi
   value=0
-  case ${ac_define_dir} in
+  case [$3] in
      "${test_prefix}"|"${test_prefix}/"*|\
 	'${exec_prefix}'|'${exec_prefix}/'*)
      value=1
      ;;
   esac
   AC_DEFINE_UNQUOTED([$1]_RELOCATABLE, $value, [Define if the $2 directory should be relocated when GDB is moved.])
+])
+
+dnl GDB_AC_WITH_DIR([VARIABLE], [ARG-NAME], [HELP], [DEFAULT])
+dnl Add a new --with option that defines a directory.
+dnl The result is stored in VARIABLE.  AC_DEFINE_DIR is called
+dnl on this variable, as is AC_SUBST.
+dnl ARG-NAME is the base name of the argument (without "--with").
+dnl HELP is the help text to use.
+dnl If the user's choice is relative to the prefix, then the
+dnl result is relocatable, then this will define the C macro
+dnl VARIABLE_RELOCATABLE to 1; otherwise it is defined as 0.
+dnl DEFAULT is the default value, which is used if the user
+dnl does not specify the argument.
+AC_DEFUN([GDB_AC_WITH_DIR], [
+  AC_ARG_WITH([$2], AS_HELP_STRING([--with-][$2][=PATH], [$3]), [
+    [$1]=$withval], [[$1]=[$4]])
+  AC_DEFINE_DIR([$1], [$1], [$3])
+  AC_SUBST([$1])
+  GDB_AC_DEFINE_RELOCATABLE([$1], [$2], ${ac_define_dir})
   ])
Index: configure.ac
===================================================================
RCS file: /cvs/src/src/gdb/configure.ac,v
retrieving revision 1.117
diff -u -p -r1.117 configure.ac
--- configure.ac	23 Apr 2010 18:07:26 -0000	1.117
+++ configure.ac	21 May 2010 08:11:50 -0000
@@ -669,6 +669,13 @@ else
     esac
     CPPFLAGS=$save_CPPFLAGS
     LIBS=$save_LIBS
+  else
+    case "${with_python}" in
+    /*)
+      AC_DEFINE_UNQUOTED(WITH_PYTHON_PATH, "${with_python}", [Define if --with-python contains a path.])
+      GDB_AC_DEFINE_RELOCATABLE(PYTHON_PATH, python-path, ${with_python})
+      ;;
+    esac
   fi
 fi
 
Index: defs.h
===================================================================
RCS file: /cvs/src/src/gdb/defs.h,v
retrieving revision 1.273
diff -u -p -r1.273 defs.h
--- defs.h	2 May 2010 23:52:14 -0000	1.273
+++ defs.h	21 May 2010 08:04:12 -0000
@@ -157,6 +157,10 @@ extern char *gdb_sysroot;
 /* GDB datadir, used to store data files.  */
 extern char *gdb_datadir;
 
+/* If non-NULL, the possibly relocated path to bin/python specified with
+   --with-python-path.  */
+extern char *python_libdir;
+
 /* Search path for separate debug files.  */
 extern char *debug_file_directory;
 
Index: main.c
===================================================================
RCS file: /cvs/src/src/gdb/main.c,v
retrieving revision 1.84
diff -u -p -r1.84 main.c
--- main.c	16 May 2010 00:18:02 -0000	1.84
+++ main.c	21 May 2010 08:04:12 -0000
@@ -68,6 +68,10 @@ char *gdb_sysroot = 0;
 /* GDB datadir, used to store data files.  */
 char *gdb_datadir = 0;
 
+/* If gdb was configured with --with-python=/path,
+   the possibly relocated path to python's lib directory.  */
+char *python_libdir = 0;
+
 struct ui_file *gdb_stdout;
 struct ui_file *gdb_stderr;
 struct ui_file *gdb_stdlog;
@@ -351,6 +358,14 @@ captured_main (void *data)
   gdb_datadir = relocate_directory (argv[0], GDB_DATADIR,
 				    GDB_DATADIR_RELOCATABLE);
 
+#ifdef WITH_PYTHON_PATH
+  /* For later use in helping Python find itself.  */
+  python_libdir = relocate_directory (argv[0],
+				      concat (WITH_PYTHON_PATH,
+					      SLASH_STRING, "lib", NULL),
+				      PYTHON_PATH_RELOCATABLE);
+#endif
+
 #ifdef RELOC_SRCDIR
   add_substitute_path_rule (RELOC_SRCDIR,
 			    make_relative_prefix (argv[0], BINDIR,
Index: python/python.c
===================================================================
RCS file: /cvs/src/src/gdb/python/python.c,v
retrieving revision 1.38
diff -u -p -r1.38 python.c
--- python/python.c	19 May 2010 23:32:24 -0000	1.38
+++ python/python.c	21 May 2010 08:04:12 -0000
@@ -645,6 +645,17 @@ Enables or disables printing of Python s
 			   &show_python_list);
 
 #ifdef HAVE_PYTHON
+#ifdef WITH_PYTHON_PATH
+  /* Work around problem where python gets confused about where it is,
+     and then can't find its libraries, etc.
+     NOTE: Python assumes the following layout:
+     /foo/bin/python
+     /foo/lib/pythonX.Y/...
+     This must be done before calling Py_Initialize.  */
+  Py_SetProgramName (concat (ldirname (python_libdir), SLASH_STRING, "bin",
+			     SLASH_STRING, "python", NULL));
+#endif
+
   Py_Initialize ();
   PyEval_InitThreads ();
 

Re: [PATCH 2/2] Add gdb_libdir/python-<version> in front of Python search path.

[Joel Brobecker]

> I think(!) this is a better approach.  Could be wrong of course. :-)
> I think adding one directory to sys.path is insufficient, and from
> what I've read it seems like Py_SetProgramName is the preferred
> solution to this problem.

ISTM for your research (thank you!) that not only is this a better
solution, but most likely the only really viable solution. We're going
to have to go with ${gdb_libdir=<prefix>/lib/gdb}/python/lib/python2.6.

> This patch doesn't currently handle your case, where the relationship
> between the location of python and gdb at build time is different than
> at runtime.  I think the solution to that is another config parameter
> to specify where to find python at run time.  Currently, when given a
> path, --with-python specifies where to find python at build time, and,
> for now, also at runtime.
> I can create another patch to add this new option
> (--with-python-libdir or some such) if you think it's worth pursuing.

I think we can work around it, by simply making sure that python is
already installed at desired location relative to the prefix used
for configuration.  In any case, if I need the extra option, I can
take care of that part.

> Regardless of whether --with-python-libdir is useful to add to the FSF
> tree, I think the attached patch is worthy.

I agree.

> 2010-05-21  Doug Evans  <dje@google.com>
> 
>         Allow python to find its files if moved from original location.
>         * acinclude.m4 (GDB_AC_DEFINE_RELOCATABLE): New function.
>         (GDB_AC_WITH_DIR): Call it.
>         * configure.ac: If a path is provided to --with-python,
>         define WITH_PYTHON_PATH.
>         * config.in: Regenerate.
>         * configure: Regenerate.
>         * defs.h (python_libdir): Declare.
>         * main.c (python_libdir): Define.
>         (captured_main): Initialize python_libdir.
>         * python/python.c (_initialize_python): If configured with
>         --with-python=/path, call Py_SetProgramName to make sure python
>         can find its libraries and modules.

I read the patch as carefully as I could, and nothing jumped out.

-- 
Joel

Re: [PATCH 2/2] Add gdb_libdir/python-<version> in front of Python search path.

[Tom Tromey]

>>>>> "Doug" == Doug Evans <dje@google.com> writes:

Doug> I think(!) this is a better approach.  Could be wrong of course. :-)
Doug> I think adding one directory to sys.path is insufficient, and from
Doug> what I've read it seems like Py_SetProgramName is the preferred
Doug> solution to this problem.

I read through this thread.  I also prefer this approach, mostly because
it doesn't change sys.path for the distro case.

Doug> This patch doesn't currently handle your case, where the relationship
Doug> between the location of python and gdb at build time is different than
Doug> at runtime.  I think the solution to that is another config parameter
Doug> to specify where to find python at run time.  Currently, when given a
Doug> path, --with-python specifies where to find python at build time, and,
Doug> for now, also at runtime.

According to the docs you can also override some Py_ functions to allow
more complete control:

The embedding application can steer the search by calling
`Py_SetProgramName(FILE)'  _before_ calling `Py_Initialize()'.  Note
that `PYTHONHOME' still overrides this and `PYTHONPATH' is still
inserted in front of the standard path.  An application that requires
total control has to provide its own implementation of `Py_GetPath()' ,
`Py_GetPrefix()' , `Py_GetExecPrefix()' , and `Py_GetProgramFullPath()'  (all
defined in `Modules/getpath.c').

Tom

Re: [PATCH 2/2] Add gdb_libdir/python-<version> in front of Python search path.

[Doug Evans]

On Fri, May 21, 2010 at 9:08 AM, Tom Tromey <tromey@redhat.com> wrote:
>>>>>> "Doug" == Doug Evans <dje@google.com> writes:
>
> Doug> I think(!) this is a better approach.  Could be wrong of course. :-)
> Doug> I think adding one directory to sys.path is insufficient, and from
> Doug> what I've read it seems like Py_SetProgramName is the preferred
> Doug> solution to this problem.
>
> I read through this thread.  I also prefer this approach, mostly because
> it doesn't change sys.path for the distro case.
>
> Doug> This patch doesn't currently handle your case, where the relationship
> Doug> between the location of python and gdb at build time is different than
> Doug> at runtime.  I think the solution to that is another config parameter
> Doug> to specify where to find python at run time.  Currently, when given a
> Doug> path, --with-python specifies where to find python at build time, and,
> Doug> for now, also at runtime.
>
> According to the docs you can also override some Py_ functions to allow
> more complete control:
>
> The embedding application can steer the search by calling
> `Py_SetProgramName(FILE)'  _before_ calling `Py_Initialize()'.  Note
> that `PYTHONHOME' still overrides this and `PYTHONPATH' is still
> inserted in front of the standard path.  An application that requires
> total control has to provide its own implementation of `Py_GetPath()' ,
> `Py_GetPrefix()' , `Py_GetExecPrefix()' , and `Py_GetProgramFullPath()'  (all
> defined in `Modules/getpath.c').

I tweaked my patch a bit to only #define WITH_PYTHON_PATH if $build = $host.
I'll let it soak another day and then check in.

[another config parameter to handle where-to-find-build-python vs
where-to-find-runtime-python may yet be a reasonable thing to have,
but I'll leave that for later.  --with-build-python might be a better
choice though]

[PATCH 1/2] Add support for --with-gdb-libdir (default: <prefix>/lib/gdb)

[Joel Brobecker]

This patch is adding a gdb_libdir path where we can store/install
gdb-specific library/object-code files... The default is lib/gdb.

2010-05-20  Joel Brobecker  <brobecker@adacore.com>

        * configure.ac: Add --with-gdb-libdir command-line option.
        * configure, config.in: Regenerate.
        * defs.h (gdb_libdir): Add declaration.
        * main.c (gdb_libdir): New global.
        (captured_main): Compute gdb_libdir.

Tested on x86_64-linux. No regression.

---
 gdb/config.in    |    7 +++++++
 gdb/configure    |   51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 gdb/configure.ac |    6 ++++++
 gdb/defs.h       |    3 +++
 gdb/main.c       |    6 ++++++
 5 files changed, 73 insertions(+), 0 deletions(-)

diff --git a/gdb/config.in b/gdb/config.in
index 5414b08..da098da 100644
--- a/gdb/config.in
+++ b/gdb/config.in
@@ -65,6 +65,13 @@
 /* Host long double floatformat */
 #undef GDB_HOST_LONG_DOUBLE_FORMAT
 
+/* look for global separate lib files in this path [LIBDIR/gdb] */
+#undef GDB_LIBDIR
+
+/* Define if the gdb-libdir directory should be relocated when GDB is moved.
+   */
+#undef GDB_LIBDIR_RELOCATABLE
+
 /* nativefile */
 #undef GDB_NM_FILE
 
diff --git a/gdb/configure b/gdb/configure
index 301394f..4432af1 100755
--- a/gdb/configure
+++ b/gdb/configure
@@ -676,6 +676,7 @@ REPORT_BUGS_TO
 PKGVERSION
 TARGET_OBS
 subdirs
+GDB_LIBDIR
 GDB_DATADIR
 DEBUGDIR
 am__fastdepCC_FALSE
@@ -884,6 +885,7 @@ enable_largefile
 enable_dependency_tracking
 with_separate_debug_dir
 with_gdb_datadir
+with_gdb_libdir
 with_relocated_sources
 enable_targets
 enable_64_bit_bfd
@@ -1584,6 +1586,8 @@ Optional Packages:
                           [LIBDIR/debug]
   --with-gdb-datadir=PATH look for global separate data files in this path
                           [DATADIR/gdb]
+  --with-gdb-libdir=PATH  look for global separate lib files in this path
+                          [LIBDIR/gdb]
   --with-relocated-sources=PATH
                           automatically relocate this path for source files
   --with-libunwind        use libunwind frame unwinding support
@@ -6847,6 +6851,53 @@ _ACEOF
 
 
 
+# GDB's libdir relocation
+
+
+
+# Check whether --with-gdb-libdir was given.
+if test "${with_gdb_libdir+set}" = set; then :
+  withval=$with_gdb_libdir;
+    GDB_LIBDIR=$withval
+else
+  GDB_LIBDIR=${libdir}/gdb
+fi
+
+
+  test "x$prefix" = xNONE && prefix="$ac_default_prefix"
+  test "x$exec_prefix" = xNONE && exec_prefix='${prefix}'
+  ac_define_dir=`eval echo $GDB_LIBDIR`
+  ac_define_dir=`eval echo $ac_define_dir`
+
+cat >>confdefs.h <<_ACEOF
+#define GDB_LIBDIR "$ac_define_dir"
+_ACEOF
+
+
+
+  if test "x$exec_prefix" = xNONE || test "x$exec_prefix" = 'x${prefix}'; then
+     if test "x$prefix" = xNONE; then
+     	test_prefix=/usr/local
+     else
+	test_prefix=$prefix
+     fi
+  else
+     test_prefix=$exec_prefix
+  fi
+  value=0
+  case ${ac_define_dir} in
+     "${test_prefix}"|"${test_prefix}/"*|\
+	'${exec_prefix}'|'${exec_prefix}/'*)
+     value=1
+     ;;
+  esac
+
+cat >>confdefs.h <<_ACEOF
+#define GDB_LIBDIR_RELOCATABLE $value
+_ACEOF
+
+
+
 
 # Check whether --with-relocated-sources was given.
 if test "${with_relocated_sources+set}" = set; then :
diff --git a/gdb/configure.ac b/gdb/configure.ac
index 4704a53..78eedde 100644
--- a/gdb/configure.ac
+++ b/gdb/configure.ac
@@ -101,6 +101,12 @@ GDB_AC_WITH_DIR(GDB_DATADIR, gdb-datadir,
     [look for global separate data files in this path @<:@DATADIR/gdb@:>@],
     [${datadir}/gdb])
 
+# GDB's libdir relocation
+
+GDB_AC_WITH_DIR(GDB_LIBDIR, gdb-libdir,
+    [look for global separate lib files in this path @<:@LIBDIR/gdb@:>@],
+    [${libdir}/gdb])
+
 AC_ARG_WITH(relocated-sources,
 AS_HELP_STRING([--with-relocated-sources=PATH], [automatically relocate this path for source files]),
 [reloc_srcdir="${withval}"
diff --git a/gdb/defs.h b/gdb/defs.h
index b18e03f..61fe7a3 100644
--- a/gdb/defs.h
+++ b/gdb/defs.h
@@ -157,6 +157,9 @@ extern char *gdb_sysroot;
 /* GDB datadir, used to store data files.  */
 extern char *gdb_datadir;
 
+/* GDB libdir, used to store lib files.  */
+extern char *gdb_libdir;
+
 /* Search path for separate debug files.  */
 extern char *debug_file_directory;
 
diff --git a/gdb/main.c b/gdb/main.c
index 254e74d..fab9012 100644
--- a/gdb/main.c
+++ b/gdb/main.c
@@ -68,6 +68,9 @@ char *gdb_sysroot = 0;
 /* GDB datadir, used to store data files.  */
 char *gdb_datadir = 0;
 
+/* GDB libdir, used to store lib files.  */
+char *gdb_libdir = 0;
+
 struct ui_file *gdb_stdout;
 struct ui_file *gdb_stderr;
 struct ui_file *gdb_stdlog;
@@ -351,6 +354,9 @@ captured_main (void *data)
   gdb_datadir = relocate_directory (argv[0], GDB_DATADIR,
 				    GDB_DATADIR_RELOCATABLE);
 
+  gdb_libdir = relocate_directory (argv[0], GDB_LIBDIR,
+				   GDB_LIBDIR_RELOCATABLE);
+
 #ifdef RELOC_SRCDIR
   add_substitute_path_rule (RELOC_SRCDIR,
 			    make_relative_prefix (argv[0], BINDIR,
-- 
1.7.1

Re: [PATCH 1/2] Add support for --with-gdb-libdir (default: <prefix>/lib/gdb)

[Joel Brobecker]

> This patch is adding a gdb_libdir path where we can store/install
> gdb-specific library/object-code files... The default is lib/gdb.
> 
> 2010-05-20  Joel Brobecker  <brobecker@adacore.com>
> 
>         * configure.ac: Add --with-gdb-libdir command-line option.
>         * configure, config.in: Regenerate.
>         * defs.h (gdb_libdir): Add declaration.
>         * main.c (gdb_libdir): New global.
>         (captured_main): Compute gdb_libdir.

This patch is no longer useful as of today, since it was meant to
be used to store the python library.  But Doug came up with something
better since then, now committed:

     http://www.sourceware.org/ml/gdb-patches/2010-05/msg00476.html
     (Allow python to find its files if moved from original location)

There might be a day when we have something else that we'd like to
store in the lib directory. But in the meantime, this is to confirm
that this patch is withdrawn.

-- 
Joel

Re: [PATCH 1/2] Add support for --with-gdb-libdir (default: <prefix>/lib/gdb)

[Doug Evans]

On Thu, May 27, 2010 at 8:00 AM, Joel Brobecker <brobecker@adacore.com> wrote:
>> This patch is adding a gdb_libdir path where we can store/install
>> gdb-specific library/object-code files... The default is lib/gdb.
>>
>> 2010-05-20  Joel Brobecker  <brobecker@adacore.com>
>>
>>         * configure.ac: Add --with-gdb-libdir command-line option.
>>         * configure, config.in: Regenerate.
>>         * defs.h (gdb_libdir): Add declaration.
>>         * main.c (gdb_libdir): New global.
>>         (captured_main): Compute gdb_libdir.
>
> This patch is no longer useful as of today, since it was meant to
> be used to store the python library.  But Doug came up with something
> better since then, now committed:
>
>     http://www.sourceware.org/ml/gdb-patches/2010-05/msg00476.html
>     (Allow python to find its files if moved from original location)
>
> There might be a day when we have something else that we'd like to
> store in the lib directory. But in the meantime, this is to confirm
> that this patch is withdrawn.

For reference sake,
I was thinking that since we had --with-gdb-datadir and
--with-gdb-libdir, do we need -with-gdb-pythondir?  :-)

Re: [PATCH 1/2] Add support for --with-gdb-libdir (default: <prefix>/lib/gdb)

[Joel Brobecker]

> For reference sake,
> I was thinking that since we had --with-gdb-datadir and
> --with-gdb-libdir, do we need -with-gdb-pythondir?  :-)

:-). I was actually suggesting we drop with --with-gdb-libdir,
so I think we still need --with-gdb-pythondir...

-- 
Joel

Re: Add LIBDIR/gdb/python-<version> to Python search path

[Daniel Jacobowitz]

On Thu, May 20, 2010 at 11:40:01AM -0700, Joel Brobecker wrote:
> The intent is to allow us to package up a GDB binary with python support.
> Since we do not know whether the target system on which GDB is going to
> be installed has the correct version of python (or even if it has, whether
> it has been built correctly), we want to also package up the python
> library as well.  That way, regardless of what the target system provides,
> we know the debugger will be using a well-defined version of the python
> library.
> 
> The location I chose for that is GDB_LIBDIR/python-<version>, where
> GDB_LIBDIR is LIBDIR/gdb by default.  So, for instance, with a prefix
> set to /my/prefix/dir, a default GDB_LIDIR, and Python version 2.5,
> we'd be using: /my/prefix/dir/lib/gdb/python-2.5.

I have a not-as-nice local patch to achieve a similar goal, so I'm in
favor.  But what do you see as the role of GDB_LIBDIR, other than
this?

GDB isn't the only program in our distribution that needs an included
Python, so we're not going to put it in lib/gdb.  But it'd be nice to
have other GDB-private data outside of top-level lib/.

-- 
Daniel Jacobowitz
CodeSourcery

Re: Add LIBDIR/gdb/python-<version> to Python search path

[Joel Brobecker]

> I have a not-as-nice local patch to achieve a similar goal, so I'm in
> favor.  But what do you see as the role of GDB_LIBDIR, other than
> this?

I admit that I don't see any other uses for now.  I picked libdir
because that seemed to be the appropriate location after reading
the GNU Coding Standards section on the various directories. I should
also give credit to Tom for suggesting it first (on IRC).

> GDB isn't the only program in our distribution that needs an included
> Python, so we're not going to put it in lib/gdb.  But it'd be nice to
> have other GDB-private data outside of top-level lib/.

That would argue in favor if adding an extra switch that allows you
to override the python libdir.  I can take care of that as part of
this patch set, for sure, if this helps in your case.

-- 
Joel

Re: Add LIBDIR/gdb/python-<version> to Python search path

[Daniel Jacobowitz]

On Thu, May 20, 2010 at 12:03:47PM -0700, Joel Brobecker wrote:
> > GDB isn't the only program in our distribution that needs an included
> > Python, so we're not going to put it in lib/gdb.  But it'd be nice to
> > have other GDB-private data outside of top-level lib/.
> 
> That would argue in favor if adding an extra switch that allows you
> to override the python libdir.  I can take care of that as part of
> this patch set, for sure, if this helps in your case.

If you're offering, I'm certainly happy.  The key, of course, is that
it's got to be relocatable just like this one.

I'd forgotten; we actually have something a little messier :-(  It
looks like this:

bin/arm-none-eabi-gdb
lib/libpython2.6.so.1.0
lib/python26.zip
lib/python2.6/_socket.so [ et cetera, ~ 50 ]

The zip file reduces the total number of files in the installation
significantly.  So I end up needing to put the zip file and the
directory both onto sys.path.  This may be more generality than the
FSF tree really wants; that's entirely fine by me.

-- 
Daniel Jacobowitz
CodeSourcery