From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 31217 invoked by alias); 12 Sep 2012 20:00:11 -0000 Received: (qmail 31157 invoked by uid 22791); 12 Sep 2012 20:00:07 -0000 X-SWARE-Spam-Status: No, hits=-3.7 required=5.0 tests=BAYES_00,KHOP_THREADED,RP_MATCHES_RCVD,TW_RG X-Spam-Check-By: sourceware.org Received: from server-nat-6.cs.umd.edu (HELO bacon.cs.umd.edu) (128.8.127.149) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Wed, 12 Sep 2012 19:59:50 +0000 Received: from [10.109.163.21] (129-2-129-152.wireless.umd.edu [129.2.129.152]) (Authenticated sender: khooyp) by bacon.cs.umd.edu (Postfix) with ESMTPSA id 794B3B403DA; Wed, 12 Sep 2012 15:59:46 -0400 (EDT) Subject: Re: [RFC] Make python/lib/gdb and submodules proper Python modules Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: multipart/mixed; boundary=Apple-Mail-4--194404340 From: Khoo Yit Phang In-Reply-To: <87obldbom1.fsf@fleche.redhat.com> Date: Wed, 12 Sep 2012 20:00:00 -0000 Cc: Khoo Yit Phang , GDB Patches Message-Id: <446971D4-9F73-4CD9-B307-C22D4C079305@cs.umd.edu> References: <87obldbom1.fsf@fleche.redhat.com> To: Tom Tromey X-CSD-MailScanner-ID: 794B3B403DA.A45C6 X-CSD-MailScanner: Found to be clean X-CSD-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-50, required 5, autolearn=not spam, ALL_TRUSTED -50.00) X-CSD-MailScanner-From: khooyp@cs.umd.edu X-CSD-MailScanner-Watermark: 1348084786.87511@8euiUkSe7tL6mr8CY8nIwQ Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org X-SW-Source: 2012-09/txt/msg00227.txt.bz2 --Apple-Mail-4--194404340 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Content-length: 412 Hi Tom, Thanks for the review. I've attached an updated patch that restores PYTHOND= IR and fixes the formatting. Also, is there a particular preferred style fo= r the Python code? Most of the code under gdb/python/lib/gdb seem to confor= m to PEP 8 (no tabs, 4-space indents, no space before parentheses) rather t= han the GNU style. So I've reformatted the "gdb" module to PEP 8 too. Yit September 12, 2012 --Apple-Mail-4--194404340 Content-Disposition: attachment; filename=python-gdb-native-module.txt Content-Type: text/plain; name="python-gdb-native-module.txt" Content-Transfer-Encoding: quoted-printable Content-length: 14017 2012-09-12 Khoo Yit Phang Refactor Python "gdb" module into a proper Python package, by introducing a new "_gdb" module for code implemented in C, and using reload/__import__ instead of exec. * python/lib/gdb/__init__.py: Import * from _gdb. (GdbOutputFile, sys.stdout, GdbOutputErrorFile, sys.stderr, prompt_hook, sys.argv): Moved from finish_python_initialization. (pretty_printers, PYTHONDIR): Moved from _initialize_python. (packages, auto_load_packages): New list and function replacing module_dict and auto-loading code, using __file__ instead of gdb.PYTHONDIR and reload/__import__ instead of exec. (GdbSetPythonDirectory): Replacing function of the same name from finish_python_initialization, using reload/__import__ instead of exec, as well as call auto_load_packages. * python/py-prettyprint.c (find_pretty_printer_from_gdb): Check gdb_python_module and not gdb_module. * python/python-internal.h (gdb_python_module): Declare. * python/python.c (gdb_python_module): New global. (before_prompt_hook): Check gdb_python_module and not gdb_module. (_initialize_python): Rename gdb module to _gdb. Move gdb.PYTHONDIR and gdb.pretty_printer to lib/gdb/__init__.py. (finish_python_initialization): Move Python code to lib/gdb/__init__.py; instead, set up sys.path and import gdb into __main__. diff --git a/gdb/python/lib/gdb/__init__.py b/gdb/python/lib/gdb/__init__.py --- a/gdb/python/lib/gdb/__init__.py +++ b/gdb/python/lib/gdb/__init__.py @@ -14,28 +14,113 @@ # along with this program. If not, see . =20 import traceback +import os +import sys +import _gdb + +from _gdb import * + +class GdbOutputFile: + def close(self): + # Do nothing. + return None + + def isatty(self): + return False + + def write(self, s): + write(s, stream=3DSTDOUT) + + def writelines(self, iterable): + for line in iterable: + self.write(line) + + def flush(self): + flush() + +sys.stdout =3D GdbOutputFile() + +class GdbOutputErrorFile: + def close(self): + # Do nothing. + return None + + def isatty(self): + return False + + def write(self, s): + write(s, stream=3DSTDERR) + + def writelines(self, iterable): + for line in iterable: + self.write(line) + + def flush(self): + flush() + +sys.stderr =3D GdbOutputErrorFile() + +# Default prompt hook does nothing. +prompt_hook =3D None + +# Ensure that sys.argv is set to something. +# We do not use PySys_SetArgvEx because it did not appear until 2.6.6. +sys.argv =3D [''] + +# Initial pretty printers. +pretty_printers =3D [] + +# Convenience variable to GDB's python directory +PYTHONDIR =3D os.path.dirname(os.path.dirname(__file__)) =20 # Auto-load all functions/commands. =20 -# Modules to auto-load, and the paths where those modules exist. +# Packages to auto-load. =20 -module_dict =3D { - 'gdb.function': os.path.join(gdb.PYTHONDIR, 'gdb', 'function'), - 'gdb.command': os.path.join(gdb.PYTHONDIR, 'gdb', 'command') -} +packages =3D [ + 'function', + 'command' +] =20 -# Iterate the dictionary, collating the Python files in each module +# pkgutil.iter_modules is not available prior to Python 2.6. Instead, +# manually iterate the list, collating the Python files in each module # path. Construct the module name, and import. =20 -for module, location in module_dict.iteritems(): - if os.path.exists(location): - py_files =3D filter(lambda x: x.endswith('.py') and x !=3D '__init__.= py', - os.listdir(location)) +def auto_load_packages(): + for package in packages: + location =3D os.path.join(os.path.dirname(__file__), package) + if os.path.exists(location): + py_files =3D filter(lambda x: x.endswith('.py') + and x !=3D '__init__.py', + os.listdir(location)) =20 - for py_file in py_files: - # Construct from foo.py, gdb.module.foo - py_file =3D module + '.' + py_file[:-3] - try: - exec('import ' + py_file) - except: - print >> sys.stderr, traceback.format_exc() + for py_file in py_files: + # Construct from foo.py, gdb.module.foo + modname =3D "%s.%s.%s" % ( __name__, package, py_file[:-3]= ) + try: + if modname in sys.modules: + # reload modules with duplicate names + reload(__import__(modname)) + else: + __import__(modname) + except: + print >> sys.stderr, traceback.format_exc() + +auto_load_packages() + +def GdbSetPythonDirectory(dir): + """Update sys.path, reload gdb and auto-load packages.""" + global PYTHONDIR + + try: + sys.path.remove(PYTHONDIR) + except ValueError: + pass + sys.path.insert(0, dir) + + PYTHONDIR =3D dir + + # note that reload overwrites the gdb module without deleting existing + # attributes + reload(__import__(__name__)) + auto_load_packages() diff --git a/gdb/python/py-prettyprint.c b/gdb/python/py-prettyprint.c --- a/gdb/python/py-prettyprint.c +++ b/gdb/python/py-prettyprint.c @@ -162,9 +162,10 @@ PyObject *function; =20 /* Fetch the global pretty printer list. */ - if (! PyObject_HasAttrString (gdb_module, "pretty_printers")) + if (gdb_python_module =3D=3D NULL + || ! PyObject_HasAttrString (gdb_python_module, "pretty_printers")) Py_RETURN_NONE; - pp_list =3D PyObject_GetAttrString (gdb_module, "pretty_printers"); + pp_list =3D PyObject_GetAttrString (gdb_python_module, "pretty_printers"= ); if (pp_list =3D=3D NULL || ! PyList_Check (pp_list)) { Py_XDECREF (pp_list); diff --git a/gdb/python/python-internal.h b/gdb/python/python-internal.h --- a/gdb/python/python-internal.h +++ b/gdb/python/python-internal.h @@ -114,6 +114,7 @@ struct inferior; =20 extern PyObject *gdb_module; +extern PyObject *gdb_python_module; extern PyTypeObject value_object_type; extern PyTypeObject block_object_type; extern PyTypeObject symbol_object_type; diff --git a/gdb/python/python.c b/gdb/python/python.c --- a/gdb/python/python.c +++ b/gdb/python/python.c @@ -74,6 +74,7 @@ static PyMethodDef GdbMethods[]; =20 PyObject *gdb_module; +PyObject *gdb_python_module; =20 /* Some string constants we may wish to use. */ PyObject *gdbpy_to_string_cst; @@ -879,11 +880,12 @@ =20 cleanup =3D ensure_python_env (get_current_arch (), current_language); =20 - if (PyObject_HasAttrString (gdb_module, "prompt_hook")) + if (gdb_python_module + && PyObject_HasAttrString (gdb_python_module, "prompt_hook")) { PyObject *hook; =20 - hook =3D PyObject_GetAttrString (gdb_module, "prompt_hook"); + hook =3D PyObject_GetAttrString (gdb_python_module, "prompt_hook"); if (hook =3D=3D NULL) goto fail; =20 @@ -1358,7 +1360,7 @@ Py_Initialize (); PyEval_InitThreads (); =20 - gdb_module =3D Py_InitModule ("gdb", GdbMethods); + gdb_module =3D Py_InitModule ("_gdb", GdbMethods); =20 /* The casts to (char*) are for python 2.4. */ PyModule_AddStringConstant (gdb_module, "VERSION", (char*) version); @@ -1370,17 +1372,6 @@ PyModule_AddIntConstant (gdb_module, "STDOUT", 0); PyModule_AddIntConstant (gdb_module, "STDERR", 1); PyModule_AddIntConstant (gdb_module, "STDLOG", 2); -=20=20 - /* gdb.parameter ("data-directory") doesn't necessarily exist when the p= ython - script below is run (depending on order of _initialize_* functions). - Define the initial value of gdb.PYTHONDIR here. */ - { - char *gdb_pythondir; - - gdb_pythondir =3D concat (gdb_datadir, SLASH_STRING, "python", NULL); - PyModule_AddStringConstant (gdb_module, "PYTHONDIR", gdb_pythondir); - xfree (gdb_pythondir); - } =20 gdbpy_gdb_error =3D PyErr_NewException ("gdb.error", PyExc_RuntimeError,= NULL); PyModule_AddObject (gdb_module, "error", gdbpy_gdb_error); @@ -1425,9 +1416,6 @@ =20 observer_attach_before_prompt (before_prompt_hook); =20 - PyRun_SimpleString ("import gdb"); - PyRun_SimpleString ("gdb.pretty_printers =3D []"); - gdbpy_to_string_cst =3D PyString_FromString ("to_string"); gdbpy_children_cst =3D PyString_FromString ("children"); gdbpy_display_hint_cst =3D PyString_FromString ("display_hint"); @@ -1452,88 +1440,67 @@ void finish_python_initialization (void) { + PyObject *m; + char *gdb_pythondir; + PyObject *sys_path; struct cleanup *cleanup; =20 cleanup =3D ensure_python_env (get_current_arch (), current_language); =20 - PyRun_SimpleString ("\ -import os\n\ -import sys\n\ -\n\ -class GdbOutputFile:\n\ - def close(self):\n\ - # Do nothing.\n\ - return None\n\ -\n\ - def isatty(self):\n\ - return False\n\ -\n\ - def write(self, s):\n\ - gdb.write(s, stream=3Dgdb.STDOUT)\n \ -\n\ - def writelines(self, iterable):\n\ - for line in iterable:\n\ - self.write(line)\n\ -\n\ - def flush(self):\n\ - gdb.flush()\n\ -\n\ -sys.stdout =3D GdbOutputFile()\n\ -\n\ -class GdbOutputErrorFile:\n\ - def close(self):\n\ - # Do nothing.\n\ - return None\n\ -\n\ - def isatty(self):\n\ - return False\n\ -\n\ - def write(self, s):\n\ - gdb.write(s, stream=3Dgdb.STDERR)\n \ -\n\ - def writelines(self, iterable):\n\ - for line in iterable:\n\ - self.write(line)\n \ -\n\ - def flush(self):\n\ - gdb.flush()\n\ -\n\ -sys.stderr =3D GdbOutputErrorFile()\n\ -\n\ -# Ideally this would live in the gdb module, but it's intentionally writte= n\n\ -# in python, and we need this to bootstrap the gdb module.\n\ -\n\ -def GdbSetPythonDirectory (dir):\n\ - \"Set gdb.PYTHONDIR and update sys.path,etc.\"\n\ - old_dir =3D gdb.PYTHONDIR\n\ - gdb.PYTHONDIR =3D dir\n\ - # GDB's python scripts are stored inside gdb.PYTHONDIR. So insert\n\ - # that directory name at the start of sys.path to allow the Python\n\ - # interpreter to find them.\n\ - if old_dir in sys.path:\n\ - sys.path.remove (old_dir)\n\ - sys.path.insert (0, gdb.PYTHONDIR)\n\ -\n\ - # Tell python where to find submodules of gdb.\n\ - gdb.__path__ =3D [os.path.join (gdb.PYTHONDIR, 'gdb')]\n\ -\n\ - # The gdb module is implemented in C rather than in Python. As a result= ,\n\ - # the associated __init.py__ script is not not executed by default when\= n\ - # the gdb module gets imported. Execute that script manually if it\n\ - # exists.\n\ - ipy =3D os.path.join (gdb.PYTHONDIR, 'gdb', '__init__.py')\n\ - if os.path.exists (ipy):\n\ - execfile (ipy)\n\ -\n\ -# Install the default gdb.PYTHONDIR.\n\ -GdbSetPythonDirectory (gdb.PYTHONDIR)\n\ -# Default prompt hook does nothing.\n\ -prompt_hook =3D None\n\ -# Ensure that sys.argv is set to something.\n\ -# We do not use PySys_SetArgvEx because it did not appear until 2.6.6.\n\ -sys.argv =3D ['']\n\ -"); + /* Add the initial data-directory to sys.path. */ =20 + gdb_pythondir =3D concat (gdb_datadir, SLASH_STRING, "python", NULL); + make_cleanup (xfree, gdb_pythondir); + + sys_path =3D PySys_GetObject ("path"); + + if (sys_path && PyList_Check (sys_path)) + { + PyObject *pythondir; + int err; + + pythondir =3D PyString_FromString (gdb_pythondir); + if (pythondir =3D=3D NULL) + goto fail; + + err =3D PyList_Insert (sys_path, 0, pythondir); + if (err) + goto fail; + + Py_DECREF (pythondir); + } + else + PySys_SetPath (gdb_pythondir); + + /* Import the gdb module to finish the initialization, and + add it to __main__ for convenience. */ + m =3D PyImport_AddModule ("__main__"); + if (m =3D=3D NULL) + goto fail; + + gdb_python_module =3D PyImport_ImportModule ("gdb"); + if (gdb_python_module =3D=3D NULL) + { + gdbpy_print_stack (); + warning (_("Could not load the Python gdb module from `%s'."), + gdb_pythondir); + warning (_("Limited Python support is available from the _gdb module= .")); + do_cleanups (cleanup); + return; + } + + if (PyModule_AddObject (m, "gdb", gdb_python_module)) + goto fail; + + /* Keep the reference to gdb_python_module since it is in a global + variable. */ + + do_cleanups (cleanup); + return; + + fail: + gdbpy_print_stack (); + warning (_("internal error: Unhandled Python exception")); do_cleanups (cleanup); } =20 diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog --- a/gdb/testsuite/ChangeLog +++ b/gdb/testsuite/ChangeLog @@ -51,6 +51,14 @@ =20 * gdb.base/info-macros.c: Fix whitespace. =20 +2012-09-02 Khoo Yit Phang + + Refactor Python "gdb" module into a proper Python package, by + introducing a new "_gdb" module for code implemented in C. + * testsuite/gdb.python/python.exp (Test stderr location): Update + module location of GDB-specific sys.stderr. + (Test stdout location): Ditto for sys.stdout. + 2012-08-28 Pedro Alves =20 PR gdb/14428 diff --git a/gdb/testsuite/gdb.python/python.exp b/gdb/testsuite/gdb.python= /python.exp --- a/gdb/testsuite/gdb.python/python.exp +++ b/gdb/testsuite/gdb.python/python.exp @@ -183,8 +183,8 @@ gdb_test {python print symtab[0]} ",func2" "stop at comma in linespec" =20 # gdb.write -gdb_test "python print sys.stderr" ".*__main__.GdbOutputErrorFile instance= at.*" "Test stderr location" -gdb_test "python print sys.stdout" ".*__main__.GdbOutputFile instance at.*= " "Test stdout location" +gdb_test "python print sys.stderr" ".*gdb.GdbOutputErrorFile instance at.*= " "Test stderr location" +gdb_test "python print sys.stdout" ".*gdb.GdbOutputFile instance at.*" "Te= st stdout location" gdb_test "python gdb.write(\"Foo\\n\")" "Foo" "Test default write" gdb_test "python gdb.write(\"Error stream\\n\", stream=3Dgdb.STDERR)" "Err= or stream" "Test stderr write" gdb_test "python gdb.write(\"Normal stream\\n\", stream=3Dgdb.STDOUT)" "No= rmal stream" "Test stdout write" --Apple-Mail-4--194404340--