[RFA] ax-gdb: Do not treat enums and bools as integers.

Joel Brobecker
2012-03-08T21:01:15Z

This patch fixes a problem when using gdb + gdbserver, and trying
to break on a function when one of the (enum) parameters is equal
to a certain value, and the size of that enum is 1 byte.

(gdb) break mixed.adb:15 if light = green
Breakpoint 2 at 0x402d5a: file mixed.adb, line 15.
(gdb) cont
Continuing.
[Inferior 1 (process 9742) exited normally]

The debugger should have stopped once when our function was call
with light set to green.

Here is what happens: Because we're using a recent GDBserver,
GDB hands off the evaluation of the condition to GDBserver, by
providing it in the Z0 packet. This is what GDB sends:

$Z0,402d5a,1;X13,26000622100223ff1c16100219162022011327#cf

I decoded the condition as follow:

260006 reg 6 -> push
2210 const8 0x10 -> push
02 add (stack now has 1 element equal to reg6 + 16)
23ff1c const16 0xff1c
1610 ext 16 (sign extend 16 bits)
02 add (stack now has 1 element equal to reg6 + 16 - 228)
19 ref32: Pop as addr, push 32bit value at addr.
1620 ext 32 (sign extend 32 bits)
2201 const8 0x01
13 equal
27 end

The beginning of the agent expression can be explained by the address
of symbol "light":

(gdb) info addr light
Symbol "light" is a variable at frame base reg $rbp offset 16+-228.

However, the mistake is the "ext 32" operation (extend 32 bits),
because our variable is *not* 32bits, only 8:

(gdb) print light'size
$5 = 8

But the reason why GDB decides to use a 32bit extension is because
it overrides the symbol's type with a plain integer type in
ax-gdb.c:gen_usual_unary...

/* If the value is an enum or a bool, call it an integer. */
case TYPE_CODE_ENUM:
case TYPE_CODE_BOOL:
value->type = builtin_type (exp->gdbarch)->builtin_int;
break;

... before calling require_rvalue. And of course, that causes the
generator to generate a sizeof(int) extension of the result.

One way to fix this would be to use an integer type of the correct
size, but I do not understand why this is necessary. The two routines
that use that information to generate the opcode down the line are
gen_fetch (for a memory value), or gen_extend (for a register value).
And they both have handling of enums and bools.

So the fix we elected to implement was simply to remove that code.

gdb/ChangeLog:

* ax-gdb.c (gen_usual_unary): Remove special handling of
enum and bool types.

Tested on x86_64-linux. I'll try to come up with an example that
reproduces the problem without using the proprietary testcase.

OK to commit?

---
gdb/ax-gdb.c | 6 ------
1 files changed, 0 insertions(+), 6 deletions(-)

diff --git a/gdb/ax-gdb.c b/gdb/ax-gdb.c
index 739677f..37882be 100644
--- a/gdb/ax-gdb.c
+++ b/gdb/ax-gdb.c
@@ -878,12 +878,6 @@ gen_usual_unary (struct expression *exp, struct agent_expr *ax,
case TYPE_CODE_STRUCT:
case TYPE_CODE_UNION:
return;
-
- /* If the value is an enum or a bool, call it an integer. */
- case TYPE_CODE_ENUM:
- case TYPE_CODE_BOOL:
- value->type = builtin_type (exp->gdbarch)->builtin_int;
- break;
}

/* If the value is an lvalue, dereference it. */
--
1.7.1