Re: [PATCH] Fix call functions command bug in 64-bit programs for AIX

Ulrich Weigand via Gdb-patches
2022-11-14T19:11:24Z

>(gdb) call num2print (2, 3, 4, 6)
>val in regno = 3 via buf is 2 and *addr is 0, regsize = 8
>val in regno = 4 via buf is 4629700416936869888 and *addr is 1077936128, regsize = 8
>val in regno = 5 via buf is 17179869184 and *addr is 4, regsize = 8
>val in regno = 6 via buf is 25769803776 and *addr is 6, regsize = 8....

Thanks!

This indicates that value is already placed incorrectly
into the register cache, before store_register was even
called.

Looking at rs6000_push_dummy_call in rs6000-aix-tdep.c, I see:

> /* Argument can fit in one register. No problem. */
> gdb_byte word[PPC_MAX_REGISTER_SIZE];
>
> memset (word, 0, reg_size);
> memcpy (word, value_contents (arg).data (), len);
> regcache->cooked_write (tdep->ppc_gp0_regnum + 3 +ii, word);

which places an "int" argument in the *high* bytes of a register
on a 64-bit system. This seems clearly wrong, and would be the
root cause of the problem you're seeing.

I think you'll need to have a closer look at this function and
make sure it implements the AIX ABI correctly, in particular
also on 64-bit systems. (You might want to have a look at the
ppc64_sysv_abi_push_dummy_call routine for comparison, which
implements the Linux 64-bit ABI; I understand this is similar
to the AIX ABI.)

I assume that once you've fixed the push_dummy_call implementation,
no changes to the store_register / fetch_register routines will
be needed at all.

Bye,
Ulrich