How to run multiple target boards

[Tom de Vries <tdevries@suse.de>]

[ was: Re: [committed][gdb/testsuite] Fix incorrect string concat in
jit-elf.exp ]

On 13-05-2020 15:45, Simon Marchi wrote:
> FWIW, I just learned it's possible to pass a list of target boards to --target_board:
> 
>   --target_board='unix native-gdbserver native-extended-gdbserver'
> 
> And it gives a combined result at the end.  So I'll make myself an alias to test on all
> the commonly used boards, probably:
> 
> - cc-with-debug-names.exp
> - cc-with-dwz.exp
> - cc-with-dwz-m.exp
> - cc-with-gdb-index.exp
> - debug-types.exp
> - dwarf4-gdb-index.exp
> - fission-dwp.exp
> - fission.exp
> - native-extended-gdbserver.exp
> - native-gdbserver.exp
> - native-stdio-gdbserver.exp
> - readnow.exp
> 
> When working on a single test, it's usually not to long to run them all.  Of course, it's
> not really practical to run the complete testsuite twice (before and after) for each board
> for every change we do...

Hi Simon,

I just tried this out.

My test scripts use make check, but I didn't manage to make that work
yet, so I tried out using runtest directly (and I may be missing
something obvious, given that I haven't used this before):
...
$ cd build/gdb/testsuite
$ runtest gdb.base/gold-gdb-index.exp --target_board='cc-with-gdb-index
 unix'
...

It seems however that the global settings (CC_FOR_TARGET etc) from the
first board stay active in the unix board:
...
Schedule of variations:
    cc-with-gdb-index
    unix

Running target cc-with-gdb-index
Using
/data/gdb_versions/devel/src/gdb/testsuite/boards/../boards/cc-with-gdb-index.exp
as board description file for target.
Using
/data/gdb_versions/devel/src/gdb/testsuite/boards/../boards/cc-with-tweaks.exp
as board description file for target.
Using
/data/gdb_versions/devel/src/gdb/testsuite/boards/../boards/local-board.exp
as board description file for target.
Using /usr/share/dejagnu/config/unix.exp as generic interface file for
target.
Using /data/gdb_versions/devel/src/gdb/testsuite/config/unix.exp as
tool-and-target-specific interface file.
Running
/data/gdb_versions/devel/src/gdb/testsuite/gdb.base/gold-gdb-index.exp ...
gdb compile failed, Error while writing index for
`/data/gdb_versions/devel/build/gdb/testsuite/outputs/gdb.base/gold-gdb-index/.tmp/gold-gdb-index':
Cannot use an index to create the index

                === gdb Summary for cc-with-gdb-index ===

# of untested testcases         1
Running target unix
Using /usr/share/dejagnu/baseboards/unix.exp as board description file
for target.
Using /usr/share/dejagnu/config/unix.exp as generic interface file for
target.
Using /data/gdb_versions/devel/src/gdb/testsuite/config/unix.exp as
tool-and-target-specific interface file.
Running
/data/gdb_versions/devel/src/gdb/testsuite/gdb.base/gold-gdb-index.exp ...
gdb compile failed, Error while writing index for
`/data/gdb_versions/devel/build/gdb/testsuite/outputs/gdb.base/gold-gdb-index/.tmp/gold-gdb-index':
Cannot use an index to create the index

                === gdb Summary for unix ===

# of untested testcases         1

                === gdb Summary ===

# of untested testcases         2
/data/gdb_versions/devel/build/gdb/gdb version  10.0.50.20200525-git -nw
-nx -data-directory
/data/gdb_versions/devel/build/gdb/testsuite/../data-directory  -ex "set
sysroot"
...

I managed to make this work by unsetting some vars in base-config.exp:
...
$ cat gdb/testsuite/config/base-config.exp
if { [info exists CC_FOR_TARGET] } {
    unset CC_FOR_TARGET
}
if { [info exists CXX_FOR_TARGET] } {
    unset CXX_FOR_TARGET
}
...
after which I get:
...
                === gdb Summary ===

# of expected passes            4
# of untested testcases         1
...

I'm not sure yet if this is the proper way to fix this.

Thanks,
- Tom