Re: [RFA] Skip the "red zone" on AMD64

[Andrew Cagney <ac131313@redhat.com>]

 > This is pretty obvious patch that doesn't harm anything, but anyway - 
OK to apply?

Hmm,

> Andreas Schwab told me that:
> Michal Ludvig <mludvig@suse.cz> writes:
> 
> |> 2003-08-07  Michal Ludvig  <mludvig@suse.cz>
> |>     * x86-64-tdep.c (x86_64_push_arguments): Skip the red zone.
> 
> Please add an empty line after the heading.
> 
> Done.
> 
> Otherwise OK.
> 
> Thanks, committed to 6.0 and head.

The PowerOpen (a.k.a. AIX) and (as I only just discovered) PPC64 ABIs 
make use of the stack beyond the top-of-stack (SP) address.  A very long 
standing bug is that GDB needs to skip that area before allocating stack 
space.

This architecture's `red zone' sounds like the same [mis-]feature.  If 
it is then the posted change won't fix the problem :-(  The return 
structure, already allocated on the stack, will smash that stack area.

I think this code:

   /* Ensure that the initial SP is correctly aligned.  */
   {
     CORE_ADDR old_sp = read_sp ();
     if (gdbarch_frame_align_p (current_gdbarch))
       {
         /* NOTE: cagney/2002-09-18:

            On a RISC architecture, a void parameterless generic dummy
            frame (i.e., no parameters, no result) typically does not
            need to push anything the stack and hence can leave SP and
            FP.  Similarly, a frameless (possibly leaf) function does
            not push anything on the stack and, hence, that too can
            leave FP and SP unchanged.  As a consequence, a sequence of
            void parameterless generic dummy frame calls to frameless
            functions will create a sequence of effectively identical
            frames (SP, FP and TOS and PC the same).  This, not
            suprisingly, results in what appears to be a stack in an
            infinite loop --- when GDB tries to find a generic dummy
            frame on the internal dummy frame stack, it will always
            find the first one.

            To avoid this problem, the code below always grows the
            stack.  That way, two dummy frames can never be identical.
            It does burn a few bytes of stack but that is a small price
            to pay :-).  */
         sp = gdbarch_frame_align (current_gdbarch, old_sp);
         if (sp == old_sp)
           {
             if (INNER_THAN (1, 2))
               /* Stack grows down.  */
               sp = gdbarch_frame_align (current_gdbarch, old_sp - 1);
             else
               /* Stack grows up.  */
               sp = gdbarch_frame_align (current_gdbarch, old_sp + 1);
           }
         gdb_assert ((INNER_THAN (1, 2) && sp <= old_sp)
                     || (INNER_THAN (2, 1) && sp >= old_sp));
       }

so that it uses a new architecture method (size_of_frame_red_zone? 
default 1, assert > 0, doco) to pad out the stack before using it.

If this is the case, can you please revert the patch and post something 
more along the above lines.  Is there code in the testsuite that 
fails/passes with the fix?

Andrew

PS: In case you're wondering.  The above code was only added a few 
months ago.  Prior to that there wasn't anywhere to put the fix.