Re: PATCH: Extend gdb remote protocol for AVX

[Mark Kettenis <mark.kettenis@xs4all.nl>]

> Date: Thu, 18 Sep 2008 10:27:28 -0700
> From: "H.J. Lu" <hongjiu.lu@intel.com>
> 
> Hi,
> 
> Intel AVX extends 128bit XMM registers to 256bit YMM registers. I
> am enclosing a propose to add YMM register support in gdb.  Since
> there is no AVX hardware, we can only implement the remote debug
> with AVX emulator.
> 
> This patch extends gdb remote protocol for AVX, based on Daniel's
> patch to auto-detect ia32 and x86-64 executables:
> 
> http://sources.redhat.com/ml/gdb-patches/2006-11/msg00056.html
> 
> I tested it by setting x86_sse_unit to avx in gdbserver/i387-fp.c.  OK
> to install?

Had some time to learn about AVX yesterday.  I noticed that the %ymm
registers partially overlap with the %xmm registers, and that while
Intel obviously is trying to deprecate the old SSE stuff, the
instructions will still be present.  As such, I think the goal:

> 1. Only display YMM registers, no XMM registers if the execution
> environment supports AVX, independent of executables.

is wrong.  People should still be able to debug traditional SSE code
even if the execution environment supports AVX.  Since the following
goals follow from #1:

> 2. Native:
>    a. Check native AVX support at run-time.
>    b. Use AVX registers only if native environment supports AVX.
>    Otherwise use XMM registers.
> 3. Remote:
>    a. Check remote AVX support when setting up connection.
>    b. Use AVX registers only if remote environment supports AVX.
>    Otherwise use XMM registers.

I disagree with those as well.

We probably need to play pseudo-register tricks to make sure %xmm and
%ymm share the data for the lower 128 bits in the register cache, and
perhaps some option to choose between showing %xmm, %ymm or both in
the "info registers" output.