Re: [PATCH 1/1] [gdb/contrib] Add make-check-all.sh

[Andrew Burgess via Gdb-patches <gdb-patches@sourceware.org>]

Tom de Vries via Gdb-patches <gdb-patches@sourceware.org> writes:

> Add script gdb/contrib/make-check-all.sh, that's intended to function as a
> drop-in replacement of make check, but excercising all host/target boards in
> gdb/testsuite/boards.
>
> Shell-checked and tested on x86_64-linux.

Hi Tom,

Thanks for putting this together, I think this could be really useful.

I'm not a fan of the way you've split the patch description into email
0/1 and not included it with this commit.  I think there's lots of
useful information in there, and I'd much rather have the whole
description included in the commit message -- it's much easier to find
then rather than having to hunt on the mailing list in the future.

> ---
>  gdb/contrib/make-check-all.sh | 255 ++++++++++++++++++++++++++++++++++
>  1 file changed, 255 insertions(+)
>  create mode 100755 gdb/contrib/make-check-all.sh
>
> diff --git a/gdb/contrib/make-check-all.sh b/gdb/contrib/make-check-all.sh
> new file mode 100755
> index 00000000000..1befe418b81
> --- /dev/null
> +++ b/gdb/contrib/make-check-all.sh
> @@ -0,0 +1,255 @@
> +#!/bin/bash
> +
> +# Copyright (C) 2023 Free Software Foundation, Inc.
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 3 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
> +
> +# Run make check with all boards from gdb/testsuite/boards.

I think you need some kind of usage test here too.  Imagine a new
developer finds this file and looks inside.  Ideally, I think they
should know how to use it, and what it will do for them without having
to read any of the actual script content.

On implementation, I wonder if it would be useful to provide a mechanism
by which the gdb.sum and gdb.log files for each test run could be
preserved?

Thanks,
Andrew

> +
> +host_boards=(
> +    local-remote-host
> +    local-remote-host-notty
> +)
> +
> +gdbserver_boards=(
> +    native-extended-gdbserver
> +    native-gdbserver
> +    native-stdio-gdbserver
> +)
> +
> +remote_gdbserver_boards=(
> +    remote-gdbserver-on-localhost
> +    remote-stdio-gdbserver
> +)
> +
> +host_target_boards=(
> +    local-remote-host-native
> +)
> +
> +target_boards=(
> +    cc-with-gdb-index
> +    cc-with-debug-names
> +    cc-with-dwz
> +    cc-with-dwz-m
> +    cc-with-gnu-debuglink
> +    debug-types
> +    dwarf4-gdb-index
> +    dwarf64
> +    fission
> +    fission-dwp
> +    gold
> +    gold-gdb-index
> +    readnow
> +    stabs
> +)
> +
> +rtf_for_board ()
> +{
> +    local b
> +    b="$1"
> +
> +    case $b in
> +	local-remote-host-native)
> +	    mkdir -p "$tmpdir/$b"
> +	    rtf=(
> +		"${rtf[@]}"
> +		"HOST_DIR=$tmpdir/$b"
> +	    )
> +	    ;;
> +	remote-stdio-gdbserver)
> +	    rtf=(
> +		"${rtf[@]}"
> +		"REMOTE_HOSTNAME=localhost"
> +	    )
> +	    if [ "$target_user" != "" ]; then
> +		rtf=(
> +		    "${rtf[@]}"
> +		    "REMOTE_USERNAME=$target_user"
> +		)
> +	    else
> +		rtf=(
> +		    "${rtf[@]}"
> +		    "REMOTE_USERNAME=$USER"
> +		)
> +	    fi
> +	    ;;
> +	remote-gdbserver-on-localhost)
> +	    if [ "$target_user" != "" ]; then
> +		rtf=(
> +		    "${rtf[@]}"
> +		    "REMOTE_TARGET_USERNAME=$target_user"
> +		)
> +	    fi
> +	    ;;
> +	local-remote-host|local-remote-host-notty)
> +	    if [ "$host_user" != "" ]; then
> +		rtf=(
> +		    "${rtf[@]}"
> +		    "REMOTE_HOST_USERNAME=$host_user"
> +		)
> +	    else
> +		rtf=(
> +		    "${rtf[@]}"
> +		    "REMOTE_HOST_USERNAME=$USER"
> +		)
> +	    fi
> +	    ;;
> +	*)
> +	    ;;
> +    esac
> +}
> +
> +summary ()
> +{
> +    if $verbose; then
> +	cat
> +    else
> +	grep -E "^(#|FAIL:|ERROR:|WARNING:)" \
> +	    | sort -u
> +    fi
> +}
> +
> +do_tests ()
> +{
> +    if $debug; then
> +	echo "RTF: ${rtf[*]}"
> +    fi
> +
> +    if $dry_run; then
> +	return
> +    fi
> +
> +    make check \
> +	 RUNTESTFLAGS="${rtf[*]} ${tests[*]}" \
> +	 2>&1 \
> +	| summary
> +}
> +
> +parse_args ()
> +{
> +    debug=false
> +    keep=false
> +    verbose=false
> +    dry_run=false
> +
> +    host_user=""
> +    target_user=""
> +
> +    while [ $# -gt 0 ]; do
> +	case "$1" in
> +	    --debug)
> +		debug=true
> +		;;
> +	    --keep)
> +		keep=true
> +		;;
> +	    --verbose)
> +		verbose=true
> +		;;
> +	    --dry-run)
> +		dry_run=true
> +		;;
> +	    --host-user)
> +		shift
> +		host_user="$1"
> +		;;
> +	    --target-user)
> +		shift
> +		target_user="$1"
> +		;;
> +	    *)
> +		break
> +		;;
> +	esac
> +	shift
> +    done
> +
> +    tests=("$@")
> +}
> +
> +main ()
> +{
> +    local b
> +    local h
> +
> +    parse_args "$@"
> +
> +    tmpdir=$(mktemp -d)
> +
> +    if $debug; then
> +	echo "TESTS: ${tests[*]}"
> +    fi
> +
> +    if true; then
> +	echo "LOCAL:"
> +	rtf=()
> +	do_tests
> +    fi
> +
> +    if true; then
> +	for b in "${target_boards[@]}"; do
> +	    echo "TARGET BOARD: $b"
> +	    rtf=(
> +		--target_board="$b"
> +	    )
> +	    rtf_for_board "$b"
> +	    do_tests
> +	done
> +    fi
> +
> +    if true; then
> +	for b in "${gdbserver_boards[@]}" "${remote_gdbserver_boards[@]}"; do
> +	    echo "TARGET BOARD: $b"
> +	    rtf=(
> +		--target_board="$b"
> +	    )
> +	    rtf_for_board "$b"
> +	    do_tests
> +	done
> +    fi
> +
> +    if true; then
> +	for h in "${host_boards[@]}"; do
> +	    for b in "${remote_gdbserver_boards[@]}"; do
> +		echo "HOST BOARD: $h, TARGET BOARD: $b"
> +		rtf=(
> +		    --host_board="$h"
> +		    --target_board="$b"
> +		)
> +		rtf_for_board "$h"
> +		rtf_for_board "$b"
> +		do_tests
> +	    done
> +	done
> +    fi
> +
> +    if true; then
> +	for b in "${host_target_boards[@]}"; do
> +	    echo "HOST/TARGET BOARD: $b"
> +	    rtf=(
> +		--host_board="$b"
> +		--target_board="$b"
> +	    )
> +	    rtf_for_board "$b"
> +	    do_tests
> +	done
> +    fi
> +
> +    if $keep; then
> +	echo "keeping tmp dir $tmpdir"
> +    else
> +	rm -Rf "$tmpdir"
> +    fi
> +}
> +
> +main "$@"
> -- 
> 2.35.3