Re: [PATCH] Make "backtrace" doesn't print python stack if init python dir get fail

[Hui Zhu <hui_zhu@mentor.com>]

On 12/04/13 04:26, Tom Tromey wrote:
>>>>>> "Hui" == Hui Zhu <hui_zhu@mentor.com> writes:
>
> Hui> I got a issue is ensure_python_env check gdb_python_initialized and
> Hui> throw error if it is 0.  So gdb_python_initialized to 0 will make
> Hui> some commands throw error when python dir has something error.  I
> Hui> removed this check in the patch.
>
> I think that's a useful consistency check, so better left in place.
>
> Two other approaches are possible here instead.  One, change
> finish_python_initialization to do the needed bit of locking by handy,
> not using ensure_python_env.  Or, two, don't release the GIL until
> somewhere in finish_python_initialization, and then it doesn't need
> to call ensure_python_env at all.
>
> Tom
>

Hi Tom,

I think the first way is better than second way because ensure_python_env has relationship with current_arch and current_language.
So I make a patch according the first way.

It extends gdb_python_initialized to a flags.  And add GDB_PYTHON_INITIALIZED_BASE and GDB_PYTHON_INITIALIZED_DIR as its flags.
In _initialize_python, set GDB_PYTHON_INITIALIZED_BASE to gdb_python_initialized.
In gdb_python_initialized, plus GDB_PYTHON_INITIALIZED_DIR to gdb_python_initialized.
Then in the function that need python lib support (apply_frame_filter, start_type_printers), check GDB_PYTHON_INITIALIZED_DIR.

It handle the issue and pass the regression test in x86_64 linux.

Please help me review it.

Thanks,
Hui

2013-12-04  Hui Zhu  <hui@codesourcery.com>

	* python/py-framefilter.c (apply_frame_filter): Change the check
	of "gdb_python_initialized".
	* python/python-internal.h (GDB_PYTHON_INITIALIZED_BASE,
	GDB_PYTHON_INITIALIZED_DIR): New.
	(gdb_python_initialized): Add comments.
	* python/python.c (gdb_python_initialized): Remove comments.
	(ensure_python_env): Change the check of "gdb_python_initialized".
	(start_type_printers): Ditto.
	(_initialize_python): Change the setup of "gdb_python_initialized".
	(finish_python_initialization): Add setup of
	"gdb_python_initialized".

--- a/gdb/python/py-framefilter.c
+++ b/gdb/python/py-framefilter.c
@@ -1474,7 +1474,7 @@ apply_frame_filter (struct frame_info *f
    PyObject *item;
    htab_t levels_printed;
  
-  if (!gdb_python_initialized)
+  if ((gdb_python_initialized & GDB_PYTHON_INITIALIZED_DIR) == 0)
      return PY_BT_NO_FILTERS;
  
    cleanups = ensure_python_env (gdbarch, current_language);
--- a/gdb/python/python-internal.h
+++ b/gdb/python/python-internal.h
@@ -204,6 +204,14 @@ struct program_space;
  struct bpstats;
  struct inferior;
  
+/* This bit will be true if Python libary has been successfully
+   initialized.  */
+#define GDB_PYTHON_INITIALIZED_BASE	0x1
+/* This bit will be true if Python directory has been successfully
+   initialized.  */
+#define GDB_PYTHON_INITIALIZED_DIR	0x2
+
+/* The status of Python initialized.  */
  extern int gdb_python_initialized;
  
  extern PyObject *gdb_module;
--- a/gdb/python/python.c
+++ b/gdb/python/python.c
@@ -73,9 +73,6 @@ static const char *gdbpy_should_print_st
  #include "interps.h"
  #include "event-top.h"
  
-/* True if Python has been successfully initialized, false
-   otherwise.  */
-
  int gdb_python_initialized;
  
  static PyMethodDef GdbMethods[];
@@ -151,7 +148,7 @@ ensure_python_env (struct gdbarch *gdbar
    struct python_env *env = xmalloc (sizeof *env);
  
    /* We should not ever enter Python unless initialized.  */
-  if (!gdb_python_initialized)
+  if ((gdb_python_initialized & GDB_PYTHON_INITIALIZED_BASE) == 0)
      error (_("Python not initialized"));
  
    env->state = PyGILState_Ensure ();
@@ -1235,7 +1232,7 @@ start_type_printers (void)
    struct cleanup *cleanups;
    PyObject *type_module, *func = NULL, *result_obj = NULL;
  
-  if (!gdb_python_initialized)
+  if ((gdb_python_initialized & GDB_PYTHON_INITIALIZED_DIR) == 0)
      return NULL;
  
    cleanups = ensure_python_env (get_current_arch (), current_language);
@@ -1717,7 +1714,7 @@ message == an error message without a st
  
    make_final_cleanup (finalize_python, NULL);
  
-  gdb_python_initialized = 1;
+  gdb_python_initialized = GDB_PYTHON_INITIALIZED_BASE;
    return;
  
   fail:
@@ -1807,6 +1804,8 @@ finish_python_initialization (void)
       variable.  */
  
    do_cleanups (cleanup);
+
+  gdb_python_initialized |= GDB_PYTHON_INITIALIZED_DIR;
    return;
  
   fail: