Re: Help Translating Message from MIPS Simulator

Maciej W. Rozycki
2017-04-17T14:30:48Z

On Sun, 16 Apr 2017, Joel Sherrill wrote:

> Thanks for the explanation.
>
> 0x88000464 is in the test code.
>
> 0x88003c68 is in the device driver. It is executing in the same thread
> as the test code. This is a single threaded test with no context switches
> before the failure.
>
> 0x88003c54 <+88>: beq a2,a3,0x88003c7c <i2c_bus_transfer+128>
> 0x88003c58 <+92>: addiu v1,v1,12
> 0x88003c5c <+96>: lhu v0,2(v1)
> 0x88003c60 <+100>: andi t0,v0,0x4000
> 0x88003c64 <+104>: bnez t0,0x88003c2c <i2c_bus_transfer+48>
> 0x88003c68 <+108>: mtlo t3
> 0x88003c6c <+112>: move t1,a3
>
> The code for i2c_bus_transfer is in pure C and it looks like gcc generated
> that mtlo. I don't see a mthi or any mfhi/lo instructions in the method.
>
> 0x88015698 is the mfhi instruction in the outer level of the RTEMS
> interrupt processing code. This is the source:
>
> mflo t0
> STREG t8, R_T8*R_SZ(sp)
> STREG t0, R_MDLO*R_SZ(sp)
> STREG t9, R_T9*R_SZ(sp)
> mfhi t0 <===================
> STREG gp, R_GP*R_SZ(sp)
> STREG t0, R_MDHI*R_SZ(sp)
> STREG fp, R_FP*R_SZ(sp)
>
>
> The first two are in C. The last was obviously is in assembly.
>
> Based on your description, I think gcc is using this as a scratch register
> and shouldn't. In case we are using the wrong compiler options, this is
> what we use:
>
> -march=r3900 -Wa,-xgot -G0
>
> I will file a gcc ticket and cc you on it if that looks like the explanation.

Thanks for quoting your specific code, which made the issue clearer to
me. I wrote the description from memory and an inaccuracy has crept in;
sorry about that. I have now double-checked old documentation to be sure.

There are actually two HI/LO hazards, which I'll illustrate with code
examples:

1. Write after read anti-dependency -- if HI or LO is written (with
MTHI or MTLO respectively or an MD operation) after a read too soon,
then data read is unpredictable:

MFHI $4
...
MTHI $5 # or DIVU, etc.; unpredictable?

There have to be at least 2 instructions between MFHI and MTHI here or
the result placed in $5 by MFHI is unpredictable. Same with MFLO/MTLO.

2. Write after write output dependency -- if an MD operation is followed
by a write to HI or LO with no intervening read, then data subsequently
read from the other register from the HI/LO pair is unpredictable:

MULTU $4, $5 # or DIVU, etc.
... # code not containing MFHI or MFLO
MTLO $6
... # code not containing MFHI
MFHI $7 # unpredictable

Likewise with MTHI followed by MFLO. There has to be a read from
either HI or LO after all MD operations for data written by subsequent
MTLO/MTHI not to interfere with the other accumulator's register.

Here obviously #2 is the case, and GCC can indeed decide to use HI or LO
as a scratch if it has found no other temporary register available and
concluded using HI or LO would be cheaper than spilling a static register
onto the stack. This is not a bug by itself, however care has to be taken
not to clobber data.

What happens here is I suspect has GCC scheduled an MD operation and
produced code that based on some condition concluded it does not need the
result of that operation (in the original R2000/R3000 processors the MD
unit runs in parallel to the main pipeline, without causing a stall, so
the scheduler may well have say filled an unused branch delay slot with an
MD operation; I suspect the TX3904 operates similarly). Later GCC has
decided to use LO as a scratch. Now if GCC has considered HI dead at that
point too, then the unpredictability of data held there does not matter.

Now you can see the unpredictability because your simulator reports it
and of course a subsequent MTHI in context restoration will write that
unpredictable data back to HI, however it does not matter for program
execution and with real hardware you won't observe it, except maybe with a
debugger.

So while such code can indeed be a result of a bug in GCC, it cannot be
told without actually seeing relevant code around. Mind that HI/LO are
call-clobbered in regular MIPS ABIs, so examining the current function
only and then further delimited by any nested function calls made is
normally enough to find out.

I suggest to file a GCC bug indeed, so as to further investigate the case
as by staring at GCC I cannot immediately tell if hazard #2 above is
handled correctly (hazard #1 certainly is, as is in GAS too, in its
`reorder' mode).

Maciej