From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 4779 invoked by alias); 3 Sep 2012 03:44:32 -0000 Received: (qmail 4769 invoked by uid 22791); 3 Sep 2012 03:44:30 -0000 X-SWARE-Spam-Status: No, hits=-2.8 required=5.0 tests=AWL,BAYES_00,RP_MATCHES_RCVD,TW_RG X-Spam-Check-By: sourceware.org Received: from server-nat-6.cs.umd.edu (HELO bacon.cs.umd.edu) (128.8.127.149) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Mon, 03 Sep 2012 03:44:16 +0000 Received: from [192.168.2.10] (pool-72-83-72-24.washdc.east.verizon.net [72.83.72.24]) (Authenticated sender: khooyp) by bacon.cs.umd.edu (Postfix) with ESMTPSA id D4DF8B40228; Sun, 2 Sep 2012 23:44:11 -0400 (EDT) From: Khoo Yit Phang Content-Type: multipart/mixed; boundary=Apple-Mail-39--1030536635 Subject: [RFC] Make python/lib/gdb and submodules proper Python modules Date: Mon, 03 Sep 2012 03:44:00 -0000 Message-Id: Cc: Khoo Yit Phang To: GDB Patches Mime-Version: 1.0 (Apple Message framework v1084) X-CSD-MailScanner-ID: D4DF8B40228.A2289 X-CSD-MailScanner: Found to be clean X-CSD-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-50, required 5, autolearn=not spam, ALL_TRUSTED -50.00) X-CSD-MailScanner-From: khooyp@cs.umd.edu X-CSD-MailScanner-Watermark: 1347248652.71544@ITVAF3I7k2jbqdKqAH7CrQ Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org X-SW-Source: 2012-09/txt/msg00009.txt.bz2 --Apple-Mail-39--1030536635 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Content-length: 1305 Hi, I found it strange that python/lib/gdb and its submodules aren't truly Pyth= on modules (e.g., the python/lib/gdb/__init__.py had to explicitly refer to= the gdb module), so I've put together a patch that makes them proper Pytho= n modules. This patch works by splitting out the gdb module into _gdb for the native c= ode extensions (as recommended by PEP8), setting up sys.path in finish_pyth= on_initialization and finally loading the gdb module, as well as switching = from using exec to reload/__import__ (which updates/creates Python modules)= . Also, I've made a few other related changes: I removed gdb.PYTHONDIR whic= h isn't necessary anymore since the same information is available via gdb._= _file__, and moved the Python code from finish_python_initialization to pyt= hon/lib/gdb/__init__.py. One caveat about this patch is that the data-directory must contain the gdb= module, otherwise Python will not be fully initialized (with a warning) an= d Python support will be very limited (only the _gdb module will be availab= le). I've ran the testsuite on Ubuntu 11.04 with Python 2.7.1 and Python 2.4.6 a= nd found no regressions (you may need the patch I recently posted that adds= -nx to a few tests to avoid spurious results). What do you think? Thanks, Yit September 2, 2012 --Apple-Mail-39--1030536635 Content-Disposition: attachment; filename=python-gdb-native-module.txt Content-Type: text/plain; name="python-gdb-native-module.txt" Content-Transfer-Encoding: quoted-printable Content-length: 13640 Refactor Python "gdb" module into a proper Python package, by introducing a= new "_gdb" module for code implemented in C. gdb/ChangeLog: 2012-09-02 Khoo Yit Phang Refactor Python "gdb" module into a proper Python package, by introducing a new "_gdb" module for code implemented in C, and using reload/__import__ instead of exec. * python/lib/gdb/__init__.py: Import * from _gdb. (GdbOutputFile, sys.stdout, GdbOutputErrorFile, sys.stderr, prompt_hook, sys.argv): Moved from finish_python_initialization. (pretty_printers): Moved from _initialize_python. (packages, auto_load_packages): New list and function replacing module_dict and auto-loading code, using __file__ instead of gdb.PYTHONDIR and reload/__import__ instead of exec. (GdbSetPythonDirectory): Replacing function of the same name from finish_python_initialization, using __file__ instead of gdb.PYTHONDIR and reload/__import__ instead of exec, as well as call auto_load_packages. * python/py-prettyprint.c (find_pretty_printer_from_gdb): Check gdb_python_module and not gdb_module. * python/python-internal.h (gdb_python_module): Declare. * python/python.c (gdb_python_module): New global. (before_prompt_hook): Check gdb_python_module and not gdb_module. (_initialize_python): Rename gdb module to _gdb. Remove gdb.PYTHONDIR. Move gdb.pretty_printer to lib/gdb/__init__.py. (finish_python_initialization): Move Python code to lib/gdb/__init__.py; instead, set up sys.path and import gdb into __main__. gdb/testsuite/ChangeLog: 2012-09-02 Khoo Yit Phang Refactor Python "gdb" module into a proper Python package, by introducing a new "_gdb" module for code implemented in C. * testsuite/gdb.python/python.exp (Test stderr location): Update module location of GDB-specific sys.stderr. (Test stdout location): Ditto for sys.stdout. diff --git a/gdb/python/lib/gdb/__init__.py b/gdb/python/lib/gdb/__init__.py --- a/gdb/python/lib/gdb/__init__.py +++ b/gdb/python/lib/gdb/__init__.py @@ -14,28 +14,110 @@ # along with this program. If not, see . =20 import traceback +import os +import sys +import _gdb + +from _gdb import * + +class GdbOutputFile: + def close(self): + # Do nothing. + return None + + def isatty(self): + return False + + def write(self, s): + write(s, stream=3DSTDOUT) + + def writelines(self, iterable): + for line in iterable: + self.write(line) + + def flush(self): + flush() + +sys.stdout =3D GdbOutputFile() + +class GdbOutputErrorFile: + def close(self): + # Do nothing. + return None + + def isatty(self): + return False + + def write(self, s): + write(s, stream=3DSTDERR) + + def writelines(self, iterable): + for line in iterable: + self.write(line) + + def flush(self): + flush() + +sys.stderr =3D GdbOutputErrorFile() + +# Default prompt hook does nothing. +prompt_hook =3D None + +# Ensure that sys.argv is set to something. +# We do not use PySys_SetArgvEx because it did not appear until 2.6.6. +sys.argv =3D [''] + +# Initial pretty printers. +pretty_printers =3D [] =20 # Auto-load all functions/commands. =20 -# Modules to auto-load, and the paths where those modules exist. +# Packages to auto-load. =20 -module_dict =3D { - 'gdb.function': os.path.join(gdb.PYTHONDIR, 'gdb', 'function'), - 'gdb.command': os.path.join(gdb.PYTHONDIR, 'gdb', 'command') -} +packages =3D [ + 'function', + 'command' +] =20 -# Iterate the dictionary, collating the Python files in each module +# pkgutil.iter_modules is not available prior to Python 2.6. Instead, +# manually iterate the list, collating the Python files in each module # path. Construct the module name, and import. =20 -for module, location in module_dict.iteritems(): - if os.path.exists(location): - py_files =3D filter(lambda x: x.endswith('.py') and x !=3D '__init__.= py', - os.listdir(location)) +def auto_load_packages(): + for package in packages: + location =3D os.path.join(os.path.dirname(__file__), package) + if os.path.exists(location): + py_files =3D filter(lambda x: x.endswith('.py') and x !=3D '__init__= .py', + os.listdir(location)) =20 - for py_file in py_files: - # Construct from foo.py, gdb.module.foo - py_file =3D module + '.' + py_file[:-3] - try: - exec('import ' + py_file) - except: - print >> sys.stderr, traceback.format_exc() + for py_file in py_files: + # Construct from foo.py, gdb.module.foo + modname =3D "%s.%s.%s" % ( __name__, package, py_file[:-3] ) + try: + if modname in sys.modules: + # reload modules with duplicate names + reload(__import__(modname)) + else: + __import__(modname) + except: + print >> sys.stderr, traceback.format_exc() + +auto_load_packages() + +def GdbSetPythonDirectory(dir): + """Update sys.path, reload gdb and auto-load packages.""" + + # Packages are identified by __init__.{py,pyc,pyd,pyo}. + if os.path.basename(__file__).startswith('__init__.py'): + old_dir =3D os.path.dirname(os.path.dirname(__file__)) + else: + old_dir =3D os.path.dirname(__file__) + + if old_dir in sys.path: + sys.path.remove(old_dir) + sys.path.insert(0, dir) + + # note that reload overwrites the gdb module without deleting existing + # attributes + reload(__import__(__name__)) + auto_load_packages() diff --git a/gdb/python/py-prettyprint.c b/gdb/python/py-prettyprint.c --- a/gdb/python/py-prettyprint.c +++ b/gdb/python/py-prettyprint.c @@ -162,9 +162,10 @@ PyObject *function; =20 /* Fetch the global pretty printer list. */ - if (! PyObject_HasAttrString (gdb_module, "pretty_printers")) + if (gdb_python_module =3D=3D NULL + || ! PyObject_HasAttrString (gdb_python_module, "pretty_printers")) Py_RETURN_NONE; - pp_list =3D PyObject_GetAttrString (gdb_module, "pretty_printers"); + pp_list =3D PyObject_GetAttrString (gdb_python_module, "pretty_printers"= ); if (pp_list =3D=3D NULL || ! PyList_Check (pp_list)) { Py_XDECREF (pp_list); diff --git a/gdb/python/python-internal.h b/gdb/python/python-internal.h --- a/gdb/python/python-internal.h +++ b/gdb/python/python-internal.h @@ -114,6 +114,7 @@ struct inferior; =20 extern PyObject *gdb_module; +extern PyObject *gdb_python_module; extern PyTypeObject value_object_type; extern PyTypeObject block_object_type; extern PyTypeObject symbol_object_type; diff --git a/gdb/python/python.c b/gdb/python/python.c --- a/gdb/python/python.c +++ b/gdb/python/python.c @@ -74,6 +74,7 @@ static PyMethodDef GdbMethods[]; =20 PyObject *gdb_module; +PyObject *gdb_python_module; =20 /* Some string constants we may wish to use. */ PyObject *gdbpy_to_string_cst; @@ -879,11 +880,11 @@ =20 cleanup =3D ensure_python_env (get_current_arch (), current_language); =20 - if (PyObject_HasAttrString (gdb_module, "prompt_hook")) + if (gdb_python_module && PyObject_HasAttrString (gdb_python_module, "pro= mpt_hook")) { PyObject *hook; =20 - hook =3D PyObject_GetAttrString (gdb_module, "prompt_hook"); + hook =3D PyObject_GetAttrString (gdb_python_module, "prompt_hook"); if (hook =3D=3D NULL) goto fail; =20 @@ -1358,7 +1359,7 @@ Py_Initialize (); PyEval_InitThreads (); =20 - gdb_module =3D Py_InitModule ("gdb", GdbMethods); + gdb_module =3D Py_InitModule ("_gdb", GdbMethods); =20 /* The casts to (char*) are for python 2.4. */ PyModule_AddStringConstant (gdb_module, "VERSION", (char*) version); @@ -1370,17 +1371,6 @@ PyModule_AddIntConstant (gdb_module, "STDOUT", 0); PyModule_AddIntConstant (gdb_module, "STDERR", 1); PyModule_AddIntConstant (gdb_module, "STDLOG", 2); -=20=20 - /* gdb.parameter ("data-directory") doesn't necessarily exist when the p= ython - script below is run (depending on order of _initialize_* functions). - Define the initial value of gdb.PYTHONDIR here. */ - { - char *gdb_pythondir; - - gdb_pythondir =3D concat (gdb_datadir, SLASH_STRING, "python", NULL); - PyModule_AddStringConstant (gdb_module, "PYTHONDIR", gdb_pythondir); - xfree (gdb_pythondir); - } =20 gdbpy_gdb_error =3D PyErr_NewException ("gdb.error", PyExc_RuntimeError,= NULL); PyModule_AddObject (gdb_module, "error", gdbpy_gdb_error); @@ -1425,9 +1415,6 @@ =20 observer_attach_before_prompt (before_prompt_hook); =20 - PyRun_SimpleString ("import gdb"); - PyRun_SimpleString ("gdb.pretty_printers =3D []"); - gdbpy_to_string_cst =3D PyString_FromString ("to_string"); gdbpy_children_cst =3D PyString_FromString ("children"); gdbpy_display_hint_cst =3D PyString_FromString ("display_hint"); @@ -1452,88 +1439,67 @@ void finish_python_initialization (void) { + PyObject *m; + char *gdb_pythondir; + PyObject *sys_path; struct cleanup *cleanup; =20 cleanup =3D ensure_python_env (get_current_arch (), current_language); =20 - PyRun_SimpleString ("\ -import os\n\ -import sys\n\ -\n\ -class GdbOutputFile:\n\ - def close(self):\n\ - # Do nothing.\n\ - return None\n\ -\n\ - def isatty(self):\n\ - return False\n\ -\n\ - def write(self, s):\n\ - gdb.write(s, stream=3Dgdb.STDOUT)\n \ -\n\ - def writelines(self, iterable):\n\ - for line in iterable:\n\ - self.write(line)\n\ -\n\ - def flush(self):\n\ - gdb.flush()\n\ -\n\ -sys.stdout =3D GdbOutputFile()\n\ -\n\ -class GdbOutputErrorFile:\n\ - def close(self):\n\ - # Do nothing.\n\ - return None\n\ -\n\ - def isatty(self):\n\ - return False\n\ -\n\ - def write(self, s):\n\ - gdb.write(s, stream=3Dgdb.STDERR)\n \ -\n\ - def writelines(self, iterable):\n\ - for line in iterable:\n\ - self.write(line)\n \ -\n\ - def flush(self):\n\ - gdb.flush()\n\ -\n\ -sys.stderr =3D GdbOutputErrorFile()\n\ -\n\ -# Ideally this would live in the gdb module, but it's intentionally writte= n\n\ -# in python, and we need this to bootstrap the gdb module.\n\ -\n\ -def GdbSetPythonDirectory (dir):\n\ - \"Set gdb.PYTHONDIR and update sys.path,etc.\"\n\ - old_dir =3D gdb.PYTHONDIR\n\ - gdb.PYTHONDIR =3D dir\n\ - # GDB's python scripts are stored inside gdb.PYTHONDIR. So insert\n\ - # that directory name at the start of sys.path to allow the Python\n\ - # interpreter to find them.\n\ - if old_dir in sys.path:\n\ - sys.path.remove (old_dir)\n\ - sys.path.insert (0, gdb.PYTHONDIR)\n\ -\n\ - # Tell python where to find submodules of gdb.\n\ - gdb.__path__ =3D [os.path.join (gdb.PYTHONDIR, 'gdb')]\n\ -\n\ - # The gdb module is implemented in C rather than in Python. As a result= ,\n\ - # the associated __init.py__ script is not not executed by default when\= n\ - # the gdb module gets imported. Execute that script manually if it\n\ - # exists.\n\ - ipy =3D os.path.join (gdb.PYTHONDIR, 'gdb', '__init__.py')\n\ - if os.path.exists (ipy):\n\ - execfile (ipy)\n\ -\n\ -# Install the default gdb.PYTHONDIR.\n\ -GdbSetPythonDirectory (gdb.PYTHONDIR)\n\ -# Default prompt hook does nothing.\n\ -prompt_hook =3D None\n\ -# Ensure that sys.argv is set to something.\n\ -# We do not use PySys_SetArgvEx because it did not appear until 2.6.6.\n\ -sys.argv =3D ['']\n\ -"); + /* Add the initial data-directory to sys.path. */ =20 + gdb_pythondir =3D concat (gdb_datadir, SLASH_STRING, "python", NULL); + make_cleanup (xfree, gdb_pythondir); + + sys_path =3D PySys_GetObject ("path"); + + if (sys_path && PyList_Check (sys_path)) + { + PyObject *pythondir; + int err; + + pythondir =3D PyString_FromString (gdb_pythondir); + if (pythondir =3D=3D NULL) + goto fail; + + err =3D PyList_Insert (sys_path, 0, pythondir); + if (err) + goto fail; + + Py_DECREF (pythondir); + } + else + PySys_SetPath (gdb_pythondir); + + /* Import the gdb module to finish the initialization, and + add it to __main__ for convenience. */ + m =3D PyImport_AddModule ("__main__"); + if (m =3D=3D NULL) + goto fail; + + gdb_python_module =3D PyImport_ImportModule ("gdb"); + if (gdb_python_module =3D=3D NULL) + { + gdbpy_print_stack (); + warning (_("Could not load the Python gdb module from `%s'."), + gdb_pythondir); + warning (_("Limited Python support is available from the _gdb module= .")); + do_cleanups (cleanup); + return; + } + + if (PyModule_AddObject (m, "gdb", gdb_python_module)) + goto fail; + + /* Keep the reference to gdb_python_module since it is in a global + variable. */ + + do_cleanups (cleanup); + return; + + fail: + gdbpy_print_stack (); + warning (_("internal error: Unhandled Python exception")); do_cleanups (cleanup); } =20 diff --git a/gdb/testsuite/gdb.python/python.exp b/gdb/testsuite/gdb.python= /python.exp --- a/gdb/testsuite/gdb.python/python.exp +++ b/gdb/testsuite/gdb.python/python.exp @@ -183,8 +183,8 @@ gdb_test {python print symtab[0]} ",func2" "stop at comma in linespec" =20 # gdb.write -gdb_test "python print sys.stderr" ".*__main__.GdbOutputErrorFile instance= at.*" "Test stderr location" -gdb_test "python print sys.stdout" ".*__main__.GdbOutputFile instance at.*= " "Test stdout location" +gdb_test "python print sys.stderr" ".*gdb.GdbOutputErrorFile instance at.*= " "Test stderr location" +gdb_test "python print sys.stdout" ".*gdb.GdbOutputFile instance at.*" "Te= st stdout location" gdb_test "python gdb.write(\"Foo\\n\")" "Foo" "Test default write" gdb_test "python gdb.write(\"Error stream\\n\", stream=3Dgdb.STDERR)" "Err= or stream" "Test stderr write" gdb_test "python gdb.write(\"Normal stream\\n\", stream=3Dgdb.STDOUT)" "No= rmal stream" "Test stdout write" --Apple-Mail-39--1030536635--