Re: [PATCH 2/2] Add gdb_libdir/python-<version> in front of Python search path.

[Doug Evans <dje@google.com>]

[-- Attachment #1: Type: text/plain, Size: 4047 bytes --]

On Thu, May 20, 2010 at 11:40 AM, Joel Brobecker <brobecker@adacore.com> wrote:
> This patch puts gdb_libdir/python-<version> at the beginning of
> the python search path.  This allows GDB to use its own copy/version
> of the Python library.
>
> 2010-05-20  Joel Brobecker  <brobecker@adacore.com>
>
>        * python.c (_initialize_python): Add variable libdir in gdb module.
>        Insert gdb_libdir/python-<version> at head of python search path.
>
> Tested on x86_64-linux. No regression.
>
> ---
>  gdb/python/python.c |    6 ++++++
>  1 files changed, 6 insertions(+), 0 deletions(-)
>
> diff --git a/gdb/python/python.c b/gdb/python/python.c
> index 3110328..fa76838 100644
> --- a/gdb/python/python.c
> +++ b/gdb/python/python.c
> @@ -654,6 +654,7 @@ Enables or disables printing of Python stack traces."),
>   PyModule_AddStringConstant (gdb_module, "VERSION", (char*) version);
>   PyModule_AddStringConstant (gdb_module, "HOST_CONFIG", (char*) host_name);
>   PyModule_AddStringConstant (gdb_module, "TARGET_CONFIG", (char*) target_name);
> +  PyModule_AddStringConstant (gdb_module, "libdir", (char*) gdb_libdir);
>
>   gdbpy_initialize_auto_load ();
>   gdbpy_initialize_values ();
> @@ -702,6 +703,11 @@ class GdbOutputFile:\n\
>  \n\
>  sys.stderr = GdbOutputFile()\n\
>  sys.stdout = GdbOutputFile()\n\
> +\n\
> +# Put gdb_libdir/python-<version> in the python search path.  This allows\n\
> +# us to provide GDB bundled with its own python runtime (the .py/pyc files).\n\
> +sys.path.insert(0, gdb.libdir + \"/python-%d.%d\" % (sys.version_info[0],\n\
> +                                                     sys.version_info[1]))\n\
>  ");
>
>   /* Release the GIL while gdb runs.  */
> --
> 1.7.1
>
>

Hi.
I think(!) this is a better approach.  Could be wrong of course. :-)
I think adding one directory to sys.path is insufficient, and from
what I've read it seems like Py_SetProgramName is the preferred
solution to this problem.

This patch doesn't currently handle your case, where the relationship
between the location of python and gdb at build time is different than
at runtime.  I think the solution to that is another config parameter
to specify where to find python at run time.  Currently, when given a
path, --with-python specifies where to find python at build time, and,
for now, also at runtime.
I can create another patch to add this new option
(--with-python-libdir or some such) if you think it's worth pursuing.
Regardless of whether --with-python-libdir is useful to add to the FSF
tree, I think the attached patch is worthy.

Note that Python's initialization assumes a particular layout that
your current plan violates (I think!).
[ref Modules/getpath.c in the python sources]
/foo/bin/python
/foo/lib/pythonX.Y/...

The attached patch will work with your --with-gdb-libdir, but python
would need to be installed as ${gdb_libdir}/lib/pythonX.Y [or
${gdb_libdir}/foo/lib/pythonX.Y].  The important part is to not
separate "lib" from "pythonX.Y".  [ref: variable lib_python in
Modules/getpath.c]

Tested with:
python configure --prefix=/mumble/lib/gdb
gdb configure --prefix=/mumble --with-python=/mumble/lib/gdb
and then renaming /mumble to /mumble2
and then trying "python import itertools" after starting gdb.

2010-05-21  Doug Evans  <dje@google.com>

        Allow python to find its files if moved from original location.
        * acinclude.m4 (GDB_AC_DEFINE_RELOCATABLE): New function.
        (GDB_AC_WITH_DIR): Call it.
        * configure.ac: If a path is provided to --with-python,
        define WITH_PYTHON_PATH.
        * config.in: Regenerate.
        * configure: Regenerate.
        * defs.h (python_libdir): Declare.
        * main.c (python_libdir): Define.
        (captured_main): Initialize python_libdir.
        * python/python.c (_initialize_python): If configured with
        --with-python=/path, call Py_SetProgramName to make sure python
        can find its libraries and modules.

[-- Attachment #2: gdb-100521-with-python-path-1.patch.txt --]
[-- Type: text/plain, Size: 6579 bytes --]

2010-05-21  Doug Evans  <dje@google.com>

	Allow python to find its files if moved from original location.
	* acinclude.m4 (GDB_AC_DEFINE_RELOCATABLE): New function.
	(GDB_AC_WITH_DIR): Call it.
	* configure.ac: If a path is provided to --with-python,
	define WITH_PYTHON_PATH.
	* config.in: Regenerate.
	* configure: Regenerate.
	* defs.h (python_libdir): Declare.
	* main.c (python_libdir): Define.
	(captured_main): Initialize python_libdir.
	* python/python.c (_initialize_python): If configured with
	--with-python=/path, call Py_SetProgramName to make sure python
	can find its libraries and modules.

Index: acinclude.m4
===================================================================
RCS file: /cvs/src/src/gdb/acinclude.m4,v
retrieving revision 1.35
diff -u -p -r1.35 acinclude.m4
--- acinclude.m4	8 Jan 2010 07:16:43 -0000	1.35
+++ acinclude.m4	21 May 2010 08:04:12 -0000
@@ -403,22 +403,12 @@ AC_DEFUN([CY_AC_TK_PRIVATE_HEADERS], [
   fi
 ])
 
-dnl GDB_AC_WITH_DIR([VARIABLE], [ARG-NAME], [HELP], [DEFAULT])
-dnl Add a new --with option that defines a directory.
-dnl The result is stored in VARIABLE.  AC_DEFINE_DIR is called
-dnl on this variable, as is AC_SUBST.
-dnl ARG-NAME is the base name of the argument (without "--with").
-dnl HELP is the help text to use.
-dnl If the user's choice is relative to the prefix, then the
+dnl GDB_AC_DEFINE_RELOCATABLE([VARIABLE], [ARG-NAME], [SHELL-VARIABLE])
+dnl For use in processing directory values for --with-foo.
+dnl If the path in SHELL_VARIABLE is relative to the prefix, then the
 dnl result is relocatable, then this will define the C macro
 dnl VARIABLE_RELOCATABLE to 1; otherwise it is defined as 0.
-dnl DEFAULT is the default value, which is used if the user
-dnl does not specify the argument.
-AC_DEFUN([GDB_AC_WITH_DIR], [
-  AC_ARG_WITH([$2], AS_HELP_STRING([--with-][$2][=PATH], [$3]), [
-    [$1]=$withval], [[$1]=[$4]])
-  AC_DEFINE_DIR([$1], [$1], [$3])
-  AC_SUBST([$1])
+AC_DEFUN([GDB_AC_DEFINE_RELOCATABLE], [
   if test "x$exec_prefix" = xNONE || test "x$exec_prefix" = 'x${prefix}'; then
      if test "x$prefix" = xNONE; then
      	test_prefix=/usr/local
@@ -429,11 +419,30 @@ AC_DEFUN([GDB_AC_WITH_DIR], [
      test_prefix=$exec_prefix
   fi
   value=0
-  case ${ac_define_dir} in
+  case [$3] in
      "${test_prefix}"|"${test_prefix}/"*|\
 	'${exec_prefix}'|'${exec_prefix}/'*)
      value=1
      ;;
   esac
   AC_DEFINE_UNQUOTED([$1]_RELOCATABLE, $value, [Define if the $2 directory should be relocated when GDB is moved.])
+])
+
+dnl GDB_AC_WITH_DIR([VARIABLE], [ARG-NAME], [HELP], [DEFAULT])
+dnl Add a new --with option that defines a directory.
+dnl The result is stored in VARIABLE.  AC_DEFINE_DIR is called
+dnl on this variable, as is AC_SUBST.
+dnl ARG-NAME is the base name of the argument (without "--with").
+dnl HELP is the help text to use.
+dnl If the user's choice is relative to the prefix, then the
+dnl result is relocatable, then this will define the C macro
+dnl VARIABLE_RELOCATABLE to 1; otherwise it is defined as 0.
+dnl DEFAULT is the default value, which is used if the user
+dnl does not specify the argument.
+AC_DEFUN([GDB_AC_WITH_DIR], [
+  AC_ARG_WITH([$2], AS_HELP_STRING([--with-][$2][=PATH], [$3]), [
+    [$1]=$withval], [[$1]=[$4]])
+  AC_DEFINE_DIR([$1], [$1], [$3])
+  AC_SUBST([$1])
+  GDB_AC_DEFINE_RELOCATABLE([$1], [$2], ${ac_define_dir})
   ])
Index: configure.ac
===================================================================
RCS file: /cvs/src/src/gdb/configure.ac,v
retrieving revision 1.117
diff -u -p -r1.117 configure.ac
--- configure.ac	23 Apr 2010 18:07:26 -0000	1.117
+++ configure.ac	21 May 2010 08:11:50 -0000
@@ -669,6 +669,13 @@ else
     esac
     CPPFLAGS=$save_CPPFLAGS
     LIBS=$save_LIBS
+  else
+    case "${with_python}" in
+    /*)
+      AC_DEFINE_UNQUOTED(WITH_PYTHON_PATH, "${with_python}", [Define if --with-python contains a path.])
+      GDB_AC_DEFINE_RELOCATABLE(PYTHON_PATH, python-path, ${with_python})
+      ;;
+    esac
   fi
 fi
 
Index: defs.h
===================================================================
RCS file: /cvs/src/src/gdb/defs.h,v
retrieving revision 1.273
diff -u -p -r1.273 defs.h
--- defs.h	2 May 2010 23:52:14 -0000	1.273
+++ defs.h	21 May 2010 08:04:12 -0000
@@ -157,6 +157,10 @@ extern char *gdb_sysroot;
 /* GDB datadir, used to store data files.  */
 extern char *gdb_datadir;
 
+/* If non-NULL, the possibly relocated path to bin/python specified with
+   --with-python-path.  */
+extern char *python_libdir;
+
 /* Search path for separate debug files.  */
 extern char *debug_file_directory;
 
Index: main.c
===================================================================
RCS file: /cvs/src/src/gdb/main.c,v
retrieving revision 1.84
diff -u -p -r1.84 main.c
--- main.c	16 May 2010 00:18:02 -0000	1.84
+++ main.c	21 May 2010 08:04:12 -0000
@@ -68,6 +68,10 @@ char *gdb_sysroot = 0;
 /* GDB datadir, used to store data files.  */
 char *gdb_datadir = 0;
 
+/* If gdb was configured with --with-python=/path,
+   the possibly relocated path to python's lib directory.  */
+char *python_libdir = 0;
+
 struct ui_file *gdb_stdout;
 struct ui_file *gdb_stderr;
 struct ui_file *gdb_stdlog;
@@ -351,6 +358,14 @@ captured_main (void *data)
   gdb_datadir = relocate_directory (argv[0], GDB_DATADIR,
 				    GDB_DATADIR_RELOCATABLE);
 
+#ifdef WITH_PYTHON_PATH
+  /* For later use in helping Python find itself.  */
+  python_libdir = relocate_directory (argv[0],
+				      concat (WITH_PYTHON_PATH,
+					      SLASH_STRING, "lib", NULL),
+				      PYTHON_PATH_RELOCATABLE);
+#endif
+
 #ifdef RELOC_SRCDIR
   add_substitute_path_rule (RELOC_SRCDIR,
 			    make_relative_prefix (argv[0], BINDIR,
Index: python/python.c
===================================================================
RCS file: /cvs/src/src/gdb/python/python.c,v
retrieving revision 1.38
diff -u -p -r1.38 python.c
--- python/python.c	19 May 2010 23:32:24 -0000	1.38
+++ python/python.c	21 May 2010 08:04:12 -0000
@@ -645,6 +645,17 @@ Enables or disables printing of Python s
 			   &show_python_list);
 
 #ifdef HAVE_PYTHON
+#ifdef WITH_PYTHON_PATH
+  /* Work around problem where python gets confused about where it is,
+     and then can't find its libraries, etc.
+     NOTE: Python assumes the following layout:
+     /foo/bin/python
+     /foo/lib/pythonX.Y/...
+     This must be done before calling Py_Initialize.  */
+  Py_SetProgramName (concat (ldirname (python_libdir), SLASH_STRING, "bin",
+			     SLASH_STRING, "python", NULL));
+#endif
+
   Py_Initialize ();
   PyEval_InitThreads ();