[PATCH] gdb/python: prevent conversion from pointer to bool in to_python()

[PATCH] gdb/python: prevent conversion from pointer to bool in to_python()

[Jan Vrany]

While converting some Python code to use new "safety" APIs I realized
that calling to_python() with a pointer like (for example):

   to_python (symtab->compunit ().objfile ())

calls to_python(bool) which is unlikely to be what the user wants.
This commit changes the code to make it a compile-time error.

After trying several approaches, the least intrusive seemed to be
handling bool specially in to_python() template for integral types and
removing to_python(bool) altogether. This way, the above example fails
with:

    error: no matching function for call to ‘to_python(objfile*)’

Tested on x86_64-linux.
---
 gdb/python/py-safety.h | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/gdb/python/py-safety.h b/gdb/python/py-safety.h
index 3294f38c8b6..9414951b9ba 100644
--- a/gdb/python/py-safety.h
+++ b/gdb/python/py-safety.h
@@ -71,17 +71,13 @@ namespace safety_details
    used in gdb, these are all Python-facing and will return NULL on
    error.  */
 
-static inline PyObject *
-to_python (bool value)
-{
-  return PyBool_FromLong (value);
-}
-
 template<typename T, typename = gdb::Requires<std::is_integral<T>>>
 static inline PyObject *
 to_python (T value)
 {
-  if constexpr (std::is_signed<T>::value)
+  if constexpr (std::is_same<T, bool>::value)
+    return PyBool_FromLong (value);
+  else if constexpr (std::is_signed<T>::value)
     return gdb_py_object_from_longest (value).release ();
   else
     return gdb_py_object_from_ulongest (value).release ();
-- 
2.53.0

Re: [PATCH] gdb/python: prevent conversion from pointer to bool in to_python()

[Tom Tromey]

>>>>> "Jan" == Jan Vrany <jan@vrany.io> writes:

Jan> While converting some Python code to use new "safety" APIs I realized
Jan> that calling to_python() with a pointer like (for example):

Jan>    to_python (symtab->compunit ().objfile ())

Jan> calls to_python(bool) which is unlikely to be what the user wants.
Jan> This commit changes the code to make it a compile-time error.

Jan> After trying several approaches, the least intrusive seemed to be
Jan> handling bool specially in to_python() template for integral types and
Jan> removing to_python(bool) altogether. This way, the above example fails
Jan> with:

Jan>     error: no matching function for call to ‘to_python(objfile*)’

Reading this made me wonder if we'd prefer a template function that
accepts pointer types but then causes a compile error.

But I think this approach is fine too.

Approved-By: Tom Tromey <tom@tromey.com>

Tom