From: Tsukasa OI via Gdb-patches <gdb-patches@sourceware.org>
To: Andrew Burgess <aburgess@redhat.com>
Cc: gdb-patches@sourceware.org
Subject: Re: [PING^4 PATCH 0/1] sim/testsuite: PR29596, Fix broken "make check-sim" by trimming excess path from arch
Date: Wed, 12 Oct 2022 14:39:05 +0900 [thread overview]
Message-ID: <a51c493b-e366-bd2f-7815-982cb1f6f3a3@irq.a4lg.com> (raw)
In-Reply-To: <87wn96d3g2.fsf@redhat.com>
On 2022/10/11 20:06, Andrew Burgess wrote:
>
> Hi!
>
> Thanks for looking at this issue.
>
>
> Tsukasa OI <research_trasio@irq.a4lg.com> writes:
>
>> TODO
>> NOTE: the cover letter and subject may change each time I ping.
>
> For a single patch series like this a cover letter really isn't needed.
> All the information you've written in this cover letter would be better
> included in the commit message for the patch. It's all really useful
> info that somebody looking at this patch in the future might want to
> know.
I once had an accident because the commit message missed important
background and made some confusion. But you are right. I might not
stop submitting one patch with cover letter if there's something to
describe (but not suited to commit message) but otherwise, I will follow
that.
>
>>
>> Hello,
>>
>> This patch fixes the problem which make check-sim (testing the simulator)
>> doesn't work on many targets.
>>
>> The patch is a PING (4) of
>> <https://sourceware.org/pipermail/binutils/2022-July/121814.html>
>> <https://sourceware.org/pipermail/gdb-patches/2022-August/191564.html>.
>> Tracker on GitHub:
>> <https://github.com/a4lg/binutils-gdb/wiki/sim_trim_path_from_arch>
>>
>> In this ping, I removed ChangeLog and won't change anymore (unless I miss
>> something).
>>
>>
>>
>> I first reproduced this bug while I'm adding simulator tests to RISC-V
>> but talking with AArch64 (with some upstream tests) would be better.
>>
>>
>>
>> [How to reproduce]
>>
>> 1. Configure Binutils with aarch64-unknown-elf and build it
>> /src/binutils/configure --target=aarch64-unknown-elf && make
>> 2. Run `make check-sim' and confirmed that the simulator tests "pass"
>> 3. Intentionally try to fail the test by modifying
>> `sim/testsuite/aarch64/pass.s'
>> (replace the last line from "pass" to "fail")
>> 4. Run `make check-sim' and "confirmed" that the simulator tests "pass"
>> (it should fail!)
>
> I followed these instructions but I couldn't reproduce the problem you
> are seeing. More below...
>
>>
>>
>>
>> [Analysis]
>>
>> ... Yes, something is going wrong. After the test, you can see the test
>> log in `sim/testsuite/aarch64/allinsn/testrun.log':
>>
>> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>> Test run by builduser on Sat Aug 27 01:04:31 2022
>> Target is aarch64-unknown-elf
>> Host is x86_64-pc-linux-gnu
>>
>> === tests ===
>>
>> Schedule of variations:
>> unix
>>
>> Running target unix
>> Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target.
>> Using /usr/share/dejagnu/config/unix.exp as generic interface file for target.
>> Using /src/binutils/sim/testsuite/config/default.exp as tool-and-target-specific interface file.
>> WARNING: Assuming target board is the local machine (which is probably wrong).
>> You may need to set your DEJAGNU environment variable.
>> Running /src/binutils/sim/testsuite/./aarch64/allinsn.exp ...
>
> I don't see this, I see:
>
> Running ../../src.dev-1/sim/testsuite/aarch64/allinsn.exp ...
>
>> Can't find a compatible assembler
>> Executing on host: false /src/binutils/sim/testsuite/lib/compilercheck.c -E -o testsuite/./aarch64/allinsn/compilercheck.x (timeout = 300)
>> spawn -ignore SIGHUP false /src/binutils/sim/testsuite/lib/compilercheck.c -E -o testsuite/./aarch64/allinsn/compilercheck.x
>> compiler exited with status 1
>> Can't find a compatible C compiler
>> Testing adds.s on machine aarch64.
>> UNTESTED: ./aarch64/adds.s
>> Testing addv.s on machine aarch64.
>> UNTESTED: ./aarch64/addv.s
>> [...]
>> Testing pass.s on machine aarch64.
>> UNTESTED: ./aarch64/pass.s
>
> And here I see:
>
> Testing pass.s on machine aarch64.
> Executing on host: /tmp/build/./gas/as-new ../../src/sim/testsuite/aarch64/pass.s -I../../src/sim/testsuite/aarch64 -o /tmp/build/sim/pass.s.o (timeout = 300)
> spawn -ignore SIGHUP /tmp/build/./gas/as-new ../../src/sim/testsuite/aarch64/pass.s -I../../src/sim/testsuite/aarch64 -o /tmp/build/sim/pass.s.o
> Executing on host: /tmp/build/./ld/ld-new /tmp/build/sim/pass.s.o -o /tmp/build/sim/pass.s.x (timeout = 300)
> spawn -ignore SIGHUP /tmp/build/./ld/ld-new /tmp/build/sim/pass.s.o -o /tmp/build/sim/pass.s.x
> ./aarch64/run /tmp/build/sim/pass.s.x
> spawn ./aarch64/run /tmp/build/sim/pass.s.x
> fail
>
> which is what we'd expect.
>
> I'm using commit d2d69057a2c, which is current HEAD of master.
>
> Any idea why I might be seeing different results to you?
>
> Thanks,
> Andrew
Wow, interesting. This is 100% reproducible on my environment (even
with commit d2d69057a2c) so... it feels like a kind of version-related
issue (most likely Tcl version). Yes, it seems you are running "make
check-sim" correctly and... I would like to see your...
- Output of "runtest --version"
- $(builddir)/sim/Makefile
- $(builddir)/sim/site.exp
My "runtest --version" result (Ubuntu 22.04.1 LTS x86_64) is:
> DejaGnu version 1.6.2
> Expect version 5.45.4
> Tcl version 8.6
Thanks,
Tsukasa
>
>> [...]
>> Testing xtn.s on machine aarch64.
>> UNTESTED: ./aarch64/xtn.s
>> testcase /src/binutils/sim/testsuite/./aarch64/allinsn.exp completed in 0 seconds
>>
>> === Summary ===
>>
>> # of untested testcases 27
>> runtest completed at Sat Aug 27 01:04:31 2022
>> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>
>>
>> We clearly find that the test runner didn't find recently built assembler
>> (gas/as-new). That assembler (and the linker) is supposed to be used
>> because the simulator itself (sim/Makefile) sets its configuration.
>>
>> Quoting `sim/site.exp' and `sim/site-sim-config.exp':
>> set AS_FOR_TARGET_AARCH64 "/build/binutils/./gas/as-new"
>> set LD_FOR_TARGET_AARCH64 "/build/binutils/./ld/ld-new"
>>
>>
>> At last, we find `sim/testsuite/lib/sim-defs.exp'.
>>
>> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>> proc sim_init_toolchain {} {
>> [...]
>> set arch [sim_arch]
>> set ARCH [string map {- _} [string toupper $arch]]
>> foreach var {AS LD CC} {
>> set var_for_target "${var}_FOR_TARGET"
>> global $var_for_target
>> set var_for_target_arch "${var_for_target}_${ARCH}"
>> global $var_for_target_arch
>>
>> if [info exists $var_for_target_arch] {
>> set $var_for_target [set $var_for_target_arch]
>> } else {
>> set $var_for_target ""
>> }
>> [...]
>> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>
>>
>> In here, it extracts {AS,LD,CC}_FOR_TARGET_AARCH64 and sets proper
>> {AS,LD,CC}_FOR_TARGET. At least, it is supposed to do so.
>>
>> However, this block doesn't work because the `arch' variable returned by
>> the `sim_arch' function is "./aarch64". That is supposed to be "aarch64".
>>
>> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>> proc sim_arch {} {
>> global subdir
>> set arch "$subdir"
>> while { [file dirname $arch] != "." } {
>> set arch [file dirname $arch]
>> }
>> return "$arch"
>> }
>> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>
>>
>>
>> [Fix: Cause and Patch Details]
>>
>> `sim_arch' function is simple. Until the dirname part of the "arch" is not
>> "." (current directory), it trims the filename part.
>>
>> For instance, if subdir is "A/B/C/D", this function returns "A". However,
>> if subdir is "./A/B/C/D", this function returns "./A", not "A".
>> In fact, actual subdir value here is "./aarch64" and we will get
>> "./aarch64" for [sim_arch]. As a result, it fails to extract proper
>> assembler/linker/compiler for given target.
>>
>> To deal with it, I added another "file tail" function call before returning.
>>
>>
>>
>> [Analysis: After the Patch]
>>
>> After this patch, `sim/testsuite/aarch64/allinsn/testrun.log' will
>> look like this:
>>
>> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>> Test run by builduser on Sat Aug 27 01:34:29 2022
>> Target is aarch64-unknown-elf
>> Host is x86_64-pc-linux-gnu
>>
>> === tests ===
>>
>> Schedule of variations:
>> unix
>>
>> Running target unix
>> Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target.
>> Using /usr/share/dejagnu/config/unix.exp as generic interface file for target.
>> Using /src/binutils/sim/testsuite/config/default.exp as tool-and-target-specific interface file.
>> WARNING: Assuming target board is the local machine (which is probably wrong).
>> You may need to set your DEJAGNU environment variable.
>> Running /src/binutils/sim/testsuite/./aarch64/allinsn.exp ...
>> Found a compatible assembler
>> Can't execute C compiler
>> Testing adds.s on machine aarch64.
>> Executing on host: /build/binutils/./gas/as-new /src/binutils/sim/testsuite/./aarch64/adds.s -I/src/binutils/sim/testsuite/./aarch64 -o testsuite/./aarch64/allinsn/adds.s.o (timeout = 300)
>> spawn -ignore SIGHUP /build/binutils/./gas/as-new /src/binutils/sim/testsuite/./aarch64/adds.s -I/src/binutils/sim/testsuite/./aarch64 -o testsuite/./aarch64/allinsn/adds.s.o
>> Executing on host: /build/binutils/./ld/ld-new testsuite/./aarch64/allinsn/adds.s.o -o testsuite/./aarch64/allinsn/adds.s.x (timeout = 300)
>> spawn -ignore SIGHUP /build/binutils/./ld/ld-new testsuite/./aarch64/allinsn/adds.s.o -o testsuite/./aarch64/allinsn/adds.s.x
>> ./aarch64/run testsuite/./aarch64/allinsn/adds.s.x
>> spawn ./aarch64/run testsuite/./aarch64/allinsn/adds.s.x
>> pass
>> PASS: aarch64 adds.s
>> [...]
>> Testing pass.s on machine aarch64.
>> Executing on host: /build/binutils/./gas/as-new /src/binutils/sim/testsuite/./aarch64/pass.s -I/src/binutils/sim/testsuite/./aarch64 -o testsuite/./aarch64/allinsn/pass.s.o (timeout = 300)
>> spawn -ignore SIGHUP /build/binutils/./gas/as-new /src/binutils/sim/testsuite/./aarch64/pass.s -I/src/binutils/sim/testsuite/./aarch64 -o testsuite/./aarch64/allinsn/pass.s.o
>> Executing on host: /build/binutils/./ld/ld-new testsuite/./aarch64/allinsn/pass.s.o -o testsuite/./aarch64/allinsn/pass.s.x (timeout = 300)
>> spawn -ignore SIGHUP /build/binutils/./ld/ld-new testsuite/./aarch64/allinsn/pass.s.o -o testsuite/./aarch64/allinsn/pass.s.x
>> ./aarch64/run testsuite/./aarch64/allinsn/pass.s.x
>> spawn ./aarch64/run testsuite/./aarch64/allinsn/pass.s.x
>> fail
>> status: 0
>> output: fail
>>
>> pattern: pass
>>
>> FAIL: aarch64 pass.s (execution)
>> [...]
>> Testing xtn.s on machine aarch64.
>> Executing on host: /build/binutils/./gas/as-new /src/binutils/sim/testsuite/./aarch64/xtn.s -I/src/binutils/sim/testsuite/./aarch64 -o testsuite/./aarch64/allinsn/xtn.s.o (timeout = 300)
>> spawn -ignore SIGHUP /build/binutils/./gas/as-new /src/binutils/sim/testsuite/./aarch64/xtn.s -I/src/binutils/sim/testsuite/./aarch64 -o testsuite/./aarch64/allinsn/xtn.s.o
>> Executing on host: /build/binutils/./ld/ld-new testsuite/./aarch64/allinsn/xtn.s.o -o testsuite/./aarch64/allinsn/xtn.s.x (timeout = 300)
>> spawn -ignore SIGHUP /build/binutils/./ld/ld-new testsuite/./aarch64/allinsn/xtn.s.o -o testsuite/./aarch64/allinsn/xtn.s.x
>> ./aarch64/run testsuite/./aarch64/allinsn/xtn.s.x
>> spawn ./aarch64/run testsuite/./aarch64/allinsn/xtn.s.x
>> pass
>> PASS: aarch64 xtn.s
>> testcase /src/binutils/sim/testsuite/./aarch64/allinsn.exp completed in 0 seconds
>>
>> === Summary ===
>>
>> # of expected passes 26
>> # of unexpected failures 1
>> runtest completed at Sat Aug 27 01:34:29 2022
>> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>
>>
>> OK, we "successfully" failed. If we restore `sim/testsuite/aarch64/pass.s',
>> it completely succeeds.
>>
>> This patch trims "./" part of "./arch" using the `file tail' function.
>> Even if the subdir input is "aarch64", it returns "aarch64".
>> With this patch, we can now successfully test the simulator.
>>
>>
>> Regards,
>> Tsukasa
>>
>>
>>
>>
>> Tsukasa OI (1):
>> sim/testsuite: PR29596, Trim extra path from arch
>>
>> sim/testsuite/lib/sim-defs.exp | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>>
>> base-commit: c10a862f17847bc9c50d680c87b87dc51ae4b95e
>> --
>> 2.34.1
>
next prev parent reply other threads:[~2022-10-12 5:39 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <28df7073ec5a8f601ba47c9d2f7d4a7a0ce08753.1657795051.git.research_trasio@irq.a4lg.com>
2022-08-27 1:53 ` [RESEND PATCH 0/1] sim/testsuite: Trim extra " Tsukasa OI via Gdb-patches
2022-08-27 1:53 ` [RESEND PATCH 1/1] " Tsukasa OI via Gdb-patches
2022-09-14 10:56 ` [PING^2 PATCH 0/1] " Tsukasa OI via Gdb-patches
2022-09-14 10:56 ` [PING^2 PATCH 1/1] " Tsukasa OI via Gdb-patches
2022-09-21 16:02 ` [PING^3 PATCH 0/1] sim/testsuite: Fix broken "make check-sim" by trimming excess " Tsukasa OI via Gdb-patches
2022-09-21 16:02 ` [PING^3 PATCH 1/1] sim/testsuite: Trim extra " Tsukasa OI via Gdb-patches
2022-10-09 3:55 ` [PING^4 PATCH 0/1] sim/testsuite: PR29596, Fix broken "make check-sim" by trimming excess " Tsukasa OI via Gdb-patches
2022-10-09 3:55 ` [PING^4 PATCH 1/1] sim/testsuite: PR29596, Trim extra " Tsukasa OI via Gdb-patches
2022-10-11 11:06 ` [PING^4 PATCH 0/1] sim/testsuite: PR29596, Fix broken "make check-sim" by trimming excess " Andrew Burgess via Gdb-patches
2022-10-12 5:39 ` Tsukasa OI via Gdb-patches [this message]
2022-10-16 13:01 ` Tsukasa OI via Gdb-patches
2022-10-23 19:16 ` [RESEND PATCH 0/1] sim/testsuite: Trim extra " Mike Frysinger via Gdb-patches
2022-10-24 6:31 ` Tsukasa OI via Gdb-patches
2022-10-24 7:33 ` Mike Frysinger via Gdb-patches
2022-10-23 19:16 ` [PATCH] sim: testsuite: tweak parallel find invocation [PR sim/29596] Mike Frysinger via Gdb-patches
2022-10-23 19:44 ` [PATCH] sim: testsuite: update ignored .exp files " Mike Frysinger via Gdb-patches
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a51c493b-e366-bd2f-7815-982cb1f6f3a3@irq.a4lg.com \
--to=gdb-patches@sourceware.org \
--cc=aburgess@redhat.com \
--cc=research_trasio@irq.a4lg.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox