PATCH: 1/6: Add AVX support

H.J. Lu
2010-03-04T18:02:28Z

AVX registers are saved and restored via the XSAVE extended state. The
extended control register 0 (the XFEATURE_ENABLED_MASK register), XCR0,
is used to determine which states, x87, SSE, AVX, ... are supported
in the XSAVE extended state. XCR0 can be read with the new "xgetbv"
instruction. The xstate_bv field at byte offset 512 in the XSAVE
extended state indicates what states the current process is in. If
the feature bit is cleared, the corresponding registers should be read as
0. If we update a register, we should set the corresponding feature
bit.

We added PTRACE_GETREGSET and PTRACE_SETREGSET to Linux kernel to
fetch and store AVX registers with ptrace. Linux kernel also stores
XCR0 at the first 8 bytes of the software usable bytes, starting at
byte offset 464.

There are total 6 patches to add AVX support for Linux. The first
patch to provide AVX XML target decriptions is at

http://sourceware.org/ml/gdb-patches/2010-03/msg00092.html

They support:

1. Backward compatible. If AVX isn't supported, SSE will be used.
2. Forward compatible. If new state beyond AVX is supported in
the XSAVE extended state, only AVX state will be used.
3. XMM pseudo register. When AVX is available, $xmmX can be used
to access the lower 128bit of $ymmX.
4. Remote gdb protocol extension. GDB will send

x86:xstate=BYTES:xcr0=VALUE

in qSupported request packet to indicate that GDB supports x86 XSAVE
extended state. BYTES specifies the maximum size in bytes of x86 XSAVE
extended state GDB supports. VALUE specifies the maximum value of XCR0
GDB supports. Gdbserver will select the best target description
supported by GDB, based on BYTES and VALUE. The older gdbserver will
always return SSE target.

To support AVX on other OSes, the following changes are needed:

1. Kernel support to get/set the XSAVE extended state.
2. Provide target to_read_description to return SSE or AVX target
description.
3. Update gdbarch_core_read_description to return SSE or AVX target
description based on contents of core dump.

H.J.