* Re: Help Translating Message from MIPS Simulator
2017-04-16 14:26 ` Joel Sherrill
@ 2017-04-17 14:26 ` Joel Sherrill
2017-04-17 14:30 ` Maciej W. Rozycki
1 sibling, 0 replies; 4+ messages in thread
From: Joel Sherrill @ 2017-04-17 14:26 UTC (permalink / raw)
To: Maciej W. Rozycki; +Cc: gdb
On 4/16/2017 9:26 AM, Joel Sherrill wrote:
>
>
> On 4/15/2017 6:44 PM, Maciej W. Rozycki wrote:
>> On Sun, 16 Apr 2017, Joel Sherrill wrote:
>>
>>> We have some RTEMS tests failing on the mips simulator
>>> in gdb. We are using the jmr3904 configuration and the
>>> run ends with this message:
>>>
>>> HILO: MFHI: MF at 0x88015698 following OP at 0x88000464 corrupted by MT at
>>> 0x88003c68
>>>
>>> 0x88000464 does appear to be in a reasonable location
>>> inside the test.
>>>
>>> How do I translate the rest to get an idea about the fault?
>>
>> I'll give some background information.
>>
>> In MIPS architecture HILO or HI/LO is the integer multiply-divide (MD)
>> unit's accumulator aka the HI and LO register pair. For widening
>> multiplication LO holds the low part of the product and HI holds the
>> corresponding high part. For division LO holds the quotient and HI holds
>> the remainder.
>>
>> These registers are not a part of the general ALU and therefore special
>> operations have been defined to retrieve and also to store data there:
>> MFHI and MFLO (Move From HI/LO) are the read instructions and MTHI and
>> MTLO (Move To HI/LO) are the write instructions for the HI and the LO
>> register respectively.
>>
>> In older architecture revisions the MTHI and MTLO instructions are only
>> really useful for context switches as data placed there cannot be further
>> used, except to read it back with MFHI or MFLO. Consequently MTHI and
>> MTLO are seldom used and those architecture revisions do not have hardware
>> interlocks implemented for them, requiring a sufficient number of other
>> instructions to be executed between a MFHI or MFLO and a following MTHI or
>> MTLO for predictable results to be produced. Otherwise the MTHI/MTLO
>> operation may (and generally will) corrupt data retrieved with MFHI/MFLO.
>
> This is an old MIPS being simulated. It is the TX3904.
> So all that applies.
>
>
>> NB later architecture revisions have integer multiply-accumulate
>> instructions which use HI/LO as one of inputs, making MTHI and MTLO more
>> useful, and they do implement hardware interlocks for them.
>>
>> So the message above means that the result of a MFHI or MFLO operation
>> (MF) at 0x88015698 executed after an MD unit operation (OP) at 0x88000464
>> has been corrupted by a MTHI or MTLO operation (MT) at 0x88003c68. And I
>> believe that what I wrote above makes the HILO and MFHI prefixes obvious.
>
> Thanks for the explanation.
>
> 0x88000464 is in the test code.
>
> 0x88003c68 is in the device driver. It is executing in the same thread
> as the test code. This is a single threaded test with no context switches
> before the failure.
>
> 0x88003c54 <+88>: beq a2,a3,0x88003c7c <i2c_bus_transfer+128>
> 0x88003c58 <+92>: addiu v1,v1,12
> 0x88003c5c <+96>: lhu v0,2(v1)
> 0x88003c60 <+100>: andi t0,v0,0x4000
> 0x88003c64 <+104>: bnez t0,0x88003c2c <i2c_bus_transfer+48>
> 0x88003c68 <+108>: mtlo t3
> 0x88003c6c <+112>: move t1,a3
>
> The code for i2c_bus_transfer is in pure C and it looks like gcc generated
> that mtlo. I don't see a mthi or any mfhi/lo instructions in the method.
>
> 0x88015698 is the mfhi instruction in the outer level of the RTEMS
> interrupt processing code. This is the source:
>
> mflo t0
> STREG t8, R_T8*R_SZ(sp)
> STREG t0, R_MDLO*R_SZ(sp)
> STREG t9, R_T9*R_SZ(sp)
> mfhi t0 <===================
> STREG gp, R_GP*R_SZ(sp)
> STREG t0, R_MDHI*R_SZ(sp)
> STREG fp, R_FP*R_SZ(sp)
>
>
> The first two are in C. The last was obviously is in assembly.
Thinking on this more, I think this is a false positive. The
code flag is saving hi/lo at the beginning of an interrupt
and will restore it at the end of the interrupt. This is a
perfectly save and proper.
I think the check is great for single-threaded (and likely
multi-threaded) code with no interrupts. But it doesn't know
that this is actually saving the CPU context and will later
restore it.
What do you think?
> Based on your description, I think gcc is using this as a scratch register
> and shouldn't. In case we are using the wrong compiler options, this is
> what we use:
>
> -march=r3900 -Wa,-xgot -G0
>
> I will file a gcc ticket and cc you on it if that looks like the explanation.
>
> Thanks.
Thanks again.
--joel
>> HTH,
>>
>> Maciej
>>
>
> --joel
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Help Translating Message from MIPS Simulator
2017-04-16 14:26 ` Joel Sherrill
2017-04-17 14:26 ` Joel Sherrill
@ 2017-04-17 14:30 ` Maciej W. Rozycki
1 sibling, 0 replies; 4+ messages in thread
From: Maciej W. Rozycki @ 2017-04-17 14:30 UTC (permalink / raw)
To: Joel Sherrill; +Cc: Maciej W. Rozycki, gdb
On Sun, 16 Apr 2017, Joel Sherrill wrote:
> Thanks for the explanation.
>
> 0x88000464 is in the test code.
>
> 0x88003c68 is in the device driver. It is executing in the same thread
> as the test code. This is a single threaded test with no context switches
> before the failure.
>
> 0x88003c54 <+88>: beq a2,a3,0x88003c7c <i2c_bus_transfer+128>
> 0x88003c58 <+92>: addiu v1,v1,12
> 0x88003c5c <+96>: lhu v0,2(v1)
> 0x88003c60 <+100>: andi t0,v0,0x4000
> 0x88003c64 <+104>: bnez t0,0x88003c2c <i2c_bus_transfer+48>
> 0x88003c68 <+108>: mtlo t3
> 0x88003c6c <+112>: move t1,a3
>
> The code for i2c_bus_transfer is in pure C and it looks like gcc generated
> that mtlo. I don't see a mthi or any mfhi/lo instructions in the method.
>
> 0x88015698 is the mfhi instruction in the outer level of the RTEMS
> interrupt processing code. This is the source:
>
> mflo t0
> STREG t8, R_T8*R_SZ(sp)
> STREG t0, R_MDLO*R_SZ(sp)
> STREG t9, R_T9*R_SZ(sp)
> mfhi t0 <===================
> STREG gp, R_GP*R_SZ(sp)
> STREG t0, R_MDHI*R_SZ(sp)
> STREG fp, R_FP*R_SZ(sp)
>
>
> The first two are in C. The last was obviously is in assembly.
>
> Based on your description, I think gcc is using this as a scratch register
> and shouldn't. In case we are using the wrong compiler options, this is
> what we use:
>
> -march=r3900 -Wa,-xgot -G0
>
> I will file a gcc ticket and cc you on it if that looks like the explanation.
Thanks for quoting your specific code, which made the issue clearer to
me. I wrote the description from memory and an inaccuracy has crept in;
sorry about that. I have now double-checked old documentation to be sure.
There are actually two HI/LO hazards, which I'll illustrate with code
examples:
1. Write after read anti-dependency -- if HI or LO is written (with
MTHI or MTLO respectively or an MD operation) after a read too soon,
then data read is unpredictable:
MFHI $4
...
MTHI $5 # or DIVU, etc.; unpredictable?
There have to be at least 2 instructions between MFHI and MTHI here or
the result placed in $5 by MFHI is unpredictable. Same with MFLO/MTLO.
2. Write after write output dependency -- if an MD operation is followed
by a write to HI or LO with no intervening read, then data subsequently
read from the other register from the HI/LO pair is unpredictable:
MULTU $4, $5 # or DIVU, etc.
... # code not containing MFHI or MFLO
MTLO $6
... # code not containing MFHI
MFHI $7 # unpredictable
Likewise with MTHI followed by MFLO. There has to be a read from
either HI or LO after all MD operations for data written by subsequent
MTLO/MTHI not to interfere with the other accumulator's register.
Here obviously #2 is the case, and GCC can indeed decide to use HI or LO
as a scratch if it has found no other temporary register available and
concluded using HI or LO would be cheaper than spilling a static register
onto the stack. This is not a bug by itself, however care has to be taken
not to clobber data.
What happens here is I suspect has GCC scheduled an MD operation and
produced code that based on some condition concluded it does not need the
result of that operation (in the original R2000/R3000 processors the MD
unit runs in parallel to the main pipeline, without causing a stall, so
the scheduler may well have say filled an unused branch delay slot with an
MD operation; I suspect the TX3904 operates similarly). Later GCC has
decided to use LO as a scratch. Now if GCC has considered HI dead at that
point too, then the unpredictability of data held there does not matter.
Now you can see the unpredictability because your simulator reports it
and of course a subsequent MTHI in context restoration will write that
unpredictable data back to HI, however it does not matter for program
execution and with real hardware you won't observe it, except maybe with a
debugger.
So while such code can indeed be a result of a bug in GCC, it cannot be
told without actually seeing relevant code around. Mind that HI/LO are
call-clobbered in regular MIPS ABIs, so examining the current function
only and then further delimited by any nested function calls made is
normally enough to find out.
I suggest to file a GCC bug indeed, so as to further investigate the case
as by staring at GCC I cannot immediately tell if hazard #2 above is
handled correctly (hazard #1 certainly is, as is in GAS too, in its
`reorder' mode).
Maciej
^ permalink raw reply [flat|nested] 4+ messages in thread