Process record does not support instruction 0xc5 at address...

Process record does not support instruction 0xc5 at address...

[Philippe Proulx]

Hello,

Is there any known solution or workaround for this problem when
using the GDB `record` command:

    Process record does not support instruction 0xc5 at address 0x7ffff7dee8a7.
    Process record: failed to record execution log.

    Program stopped.
    0x00007ffff7dee8a7 in _dl_runtime_resolve_avx () from
/lib64/ld-linux-x86-64.so.2

Thank you,

Philippe Proulx

Re: Process record does not support instruction 0xc5 at address...

[dwk]

I used to run into this all the time with SIGSEGV, SIGINT, instruction
0xcc, instruction 0xf4, etc. It seems to have been fixed when I upgraded
gdb at one point. I am currently using gdb 7.7.1 on x86-64. Your mileage
may vary.

On Sun, Aug 14, 2016 at 5:30 PM, Philippe Proulx
<eeppeliteloop@gmail.com> wrote:
> Hello,
>
> Is there any known solution or workaround for this problem when
> using the GDB `record` command:
>
>     Process record does not support instruction 0xc5 at address 0x7ffff7dee8a7.
>     Process record: failed to record execution log.
>
>     Program stopped.
>     0x00007ffff7dee8a7 in _dl_runtime_resolve_avx () from
> /lib64/ld-linux-x86-64.so.2
>
> Thank you,
>
> Philippe Proulx

Re: Process record does not support instruction 0xc5 at address...

[Philippe Proulx]

On Sun, Aug 14, 2016 at 5:42 PM, dwk <dwks42@gmail.com> wrote:
> I used to run into this all the time with SIGSEGV, SIGINT, instruction
> 0xcc, instruction 0xf4, etc. It seems to have been fixed when I upgraded
> gdb at one point. I am currently using gdb 7.7.1 on x86-64. Your mileage
> may vary.

I'm on 7.11.1 by the way.

Processor is Intel(R) Core(TM) i7-3520M with the following flags:

fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36
clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc
aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2
ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer
aes xsave avx f16c rdrand lahf_lm epb tpr_shadow vnmi flexpriority ept
vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts

Phil

>
> On Sun, Aug 14, 2016 at 5:30 PM, Philippe Proulx
> <eeppeliteloop@gmail.com> wrote:
>> Hello,
>>
>> Is there any known solution or workaround for this problem when
>> using the GDB `record` command:
>>
>>     Process record does not support instruction 0xc5 at address 0x7ffff7dee8a7.
>>     Process record: failed to record execution log.
>>
>>     Program stopped.
>>     0x00007ffff7dee8a7 in _dl_runtime_resolve_avx () from
>> /lib64/ld-linux-x86-64.so.2
>>
>> Thank you,
>>
>> Philippe Proulx

Re: Process record does not support instruction 0xc5 at address...

[dwk]

Some context. I found that on an illegal instruction, or when jumping to
an address where there was no mapped instruction, process record would
fail in this way. It looks like 0xc5 is an undefined instruction in
64-bit mode (see http://ref.x86asm.net/coder64.html#xC5) so you appear
to be running to the exact same issue. My work-around was to write a
script which would disable record, single-step to the next instruction,
then re-enable record. However, I was intentionally adding undefined
instructions and hoping for a SIGILL.

The deeper issue here might be that some AVX instructions are not
supported by process record. Thus, I suggest you disable the AVX versions
of functions in libc, libm, and ld.so. libm in particular makes heavy
use of IFUNCs -- indirect symbols which are supposed to be called once to
determine the real target function. Typically it looks at the supported
features of your CPU and selects the appropriate function (AVX optimized,
SSE optimized, SSE3 optimized, etc). It can't hurt to try disabling
this. You can recompile libc (thus ld.so), or hack __init_cpu_features
and thus __cpu_features at runtime (see e.g. strcmp).

Oh, you can also try setting LD_BIND_NOW=1 so that symbols are all
resolved immediately at load-time and _dl_runtime_resolve will never be
called later, unless you dlopen of course.

On Sun, Aug 14, 2016 at 6:36 PM, Philippe Proulx
<eeppeliteloop@gmail.com> wrote:
> On Sun, Aug 14, 2016 at 5:42 PM, dwk <dwks42@gmail.com> wrote:
>> I used to run into this all the time with SIGSEGV, SIGINT, instruction
>> 0xcc, instruction 0xf4, etc. It seems to have been fixed when I upgraded
>> gdb at one point. I am currently using gdb 7.7.1 on x86-64. Your mileage
>> may vary.
>
> I'm on 7.11.1 by the way.
>
> Processor is Intel(R) Core(TM) i7-3520M with the following flags:
>
> fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36
> clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
> constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc
> aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2
> ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer
> aes xsave avx f16c rdrand lahf_lm epb tpr_shadow vnmi flexpriority ept
> vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
>
> Phil
>
>>
>> On Sun, Aug 14, 2016 at 5:30 PM, Philippe Proulx
>> <eeppeliteloop@gmail.com> wrote:
>>> Hello,
>>>
>>> Is there any known solution or workaround for this problem when
>>> using the GDB `record` command:
>>>
>>>     Process record does not support instruction 0xc5 at address 0x7ffff7dee8a7.
>>>     Process record: failed to record execution log.
>>>
>>>     Program stopped.
>>>     0x00007ffff7dee8a7 in _dl_runtime_resolve_avx () from
>>> /lib64/ld-linux-x86-64.so.2
>>>
>>> Thank you,
>>>
>>> Philippe Proulx

Re: Process record does not support instruction 0xc5 at address...

[Philippe Proulx]

On Sun, Aug 14, 2016 at 7:08 PM, dwk <dwks42@gmail.com> wrote:
> Some context. I found that on an illegal instruction, or when jumping to
> an address where there was no mapped instruction, process record would
> fail in this way. It looks like 0xc5 is an undefined instruction in
> 64-bit mode (see http://ref.x86asm.net/coder64.html#xC5) so you appear
> to be running to the exact same issue.

Hmm, I disassembled `ld-linux-x86-64.so.2` and found those in said function:

  158a7:       c5 fd 7f 04 24          vmovdqa %ymm0,(%rsp)
  158ac:       c5 fd 7f 4c 24 20       vmovdqa %ymm1,0x20(%rsp)
  158b2:       c5 fd 7f 54 24 40       vmovdqa %ymm2,0x40(%rsp)
  158b8:       c5 fd 7f 5c 24 60       vmovdqa %ymm3,0x60(%rsp)
  158be:       c5 fd 7f a4 24 80 00    vmovdqa %ymm4,0x80(%rsp)

0x5c is the VEX prefix [1]. From [1]:

    The VEX prefix's initial-byte values, C4h and C5h, are the same as
    the opcodes of the LDS and LES instructions. These instructions are
    not supported in 64-bit mode.

and:

    The AVX instruction set is the first instruction set extension to use
    the VEX coding scheme.

So this is not illegal, and I believe it's just that the GDB
instruction interpreter
does not support this set (yet) [2].

> My work-around was to write a
> script which would disable record, single-step to the next instruction,
> then re-enable record. However, I was intentionally adding undefined
> instructions and hoping for a SIGILL.
>
> The deeper issue here might be that some AVX instructions are not
> supported by process record. Thus, I suggest you disable the AVX versions
> of functions in libc, libm, and ld.so.

This is a somewhat heavy solution but it could be done, possibly with -mno-avx.

> libm in particular makes heavy
> use of IFUNCs -- indirect symbols which are supposed to be called once to
> determine the real target function. Typically it looks at the supported
> features of your CPU and selects the appropriate function (AVX optimized,
> SSE optimized, SSE3 optimized, etc). It can't hurt to try disabling
> this. You can recompile libc (thus ld.so), or hack __init_cpu_features
> and thus __cpu_features at runtime (see e.g. strcmp).
>
> Oh, you can also try setting LD_BIND_NOW=1 so that symbols are all
> resolved immediately at load-time and _dl_runtime_resolve will never be
> called later, unless you dlopen of course.

This works. Thanks! Excellent workaround in the meantime, it did not occured
to me.

[1] https://en.wikipedia.org/wiki/VEX_prefix
[2] https://www.google.com/url?q=https%3A%2F%2Fsourceware.org%2Fgit%2Fgitweb.cgi%3Fp%3Dbinutils-gdb.git%3Ba%3Dblob%3Bf%3Dgdb%2Fi386-tdep.c%3Bh%3D92987af8101ccde68c892699b9a8dee30e9dc9c9%3Bhb%3DHEAD%23l5906&sa=D&sntz=1&usg=AFQjCNG4XN_OosfwyfBmUa9qISDBWVVpbQ

Phil

>
> On Sun, Aug 14, 2016 at 6:36 PM, Philippe Proulx
> <eeppeliteloop@gmail.com> wrote:
>> On Sun, Aug 14, 2016 at 5:42 PM, dwk <dwks42@gmail.com> wrote:
>>> I used to run into this all the time with SIGSEGV, SIGINT, instruction
>>> 0xcc, instruction 0xf4, etc. It seems to have been fixed when I upgraded
>>> gdb at one point. I am currently using gdb 7.7.1 on x86-64. Your mileage
>>> may vary.
>>
>> I'm on 7.11.1 by the way.
>>
>> Processor is Intel(R) Core(TM) i7-3520M with the following flags:
>>
>> fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36
>> clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
>> constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc
>> aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2
>> ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer
>> aes xsave avx f16c rdrand lahf_lm epb tpr_shadow vnmi flexpriority ept
>> vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
>>
>> Phil
>>
>>>
>>> On Sun, Aug 14, 2016 at 5:30 PM, Philippe Proulx
>>> <eeppeliteloop@gmail.com> wrote:
>>>> Hello,
>>>>
>>>> Is there any known solution or workaround for this problem when
>>>> using the GDB `record` command:
>>>>
>>>>     Process record does not support instruction 0xc5 at address 0x7ffff7dee8a7.
>>>>     Process record: failed to record execution log.
>>>>
>>>>     Program stopped.
>>>>     0x00007ffff7dee8a7 in _dl_runtime_resolve_avx () from
>>>> /lib64/ld-linux-x86-64.so.2
>>>>
>>>> Thank you,
>>>>
>>>> Philippe Proulx