From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from simark.ca by simark.ca with LMTP id IRbHEYiu4Gp8WS8AWB0awg (envelope-from ) for ; Tue, 27 Oct 2026 10:00:40 -0400 Authentication-Results: simark.ca; dkim=pass (2048-bit key; unprotected) header.d=vrany.io header.i=@vrany.io header.a=rsa-sha256 header.s=protonmail header.b=C5B8VARy; dkim-atps=neutral Received: by simark.ca (Postfix, from userid 112) id 44A2E378003; Tue, 27 Oct 2026 10:00:40 -0400 (EDT) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-25) on simark.ca X-Spam-Level: X-Spam-Status: No, score=-5.4 required=5.0 tests=ARC_SIGNED,ARC_VALID,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=unavailable autolearn_force=no version=4.0.1 Received: from vm01.sourceware.org (vm01.sourceware.org [IPv6:2620:52:6:3111::32]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by simark.ca (Postfix) with ESMTPS id 347CA378002 for ; Tue, 27 Oct 2026 10:00:39 -0400 (EDT) Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C3ECD4BA798C for ; Tue, 27 Oct 2026 14:00:37 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C3ECD4BA798C Authentication-Results: sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=vrany.io header.i=@vrany.io header.a=rsa-sha256 header.s=protonmail header.b=C5B8VARy Received: from mail-106116.protonmail.ch (mail-106116.protonmail.ch [79.135.106.116]) by sourceware.org (Postfix) with ESMTPS id 02EF84BA799C for ; Tue, 27 Oct 2026 14:00:01 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 02EF84BA799C Authentication-Results: sourceware.org; dmarc=pass (p=quarantine dis=none) header.from=vrany.io Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=vrany.io ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 02EF84BA799C Authentication-Results: sourceware.org; arc=none smtp.remote-ip=79.135.106.116 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1793109608; cv=none; b=exk+2kC1+bTXAqE2Dk6DMEcVG0GKDxWVzI0hNHEmflyBs6zZT3X5nokTEfSfYEI49CbVL2lb7V9q5wdHTWBM4+1vY8nzDQgRXK//0Kk4cUCLmXIR54kfJW3QJPSRGkTsRVlHnEA9mVHv9oREX5TG+RYJlHd4woK549jDAXrQ2ys= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1793109608; c=relaxed/simple; bh=IUyqQJcRtjkhVeCG9tmYnfZGaYZGHuTzd+ti7dgAGtI=; h=DKIM-Signature:Date:To:From:Subject:Message-ID:MIME-Version; b=WekOk4WveKyqFRzDnSRZPRCPqVMOgXmBuwgaJG60TfMaSoAP6DuNs8ktJhT+OoQzkyx21SAo+k17ee6yDz/1ySFanf6dOJqezR/yCjyzc5nQAYh7YyOW71GmLxp8+mwoadjMdZ9+igbUcrT/QaP/lXf1D24lxaBoTTS5AhNEbp0= ARC-Authentication-Results: i=1; sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=vrany.io header.i=@vrany.io header.a=rsa-sha256 header.s=protonmail header.b=C5B8VARy DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 02EF84BA799C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vrany.io; s=protonmail; t=1793109597; x=1793368797; bh=Gq4tE+vDgsuPzkFCTMP8BbTbce5riflxJ1mw/R13sv0=; h=Date:To:From:Cc:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector; b=C5B8VARyCWkSyYnxg61ZNf2vgNyhGMyh7dBBZ167gV45qTzLEF3Zr8mY2bktag7Nt 1w7j/WuACtcEhiqJEC8SXDz3/1LWCgmSpaeJOlBtGXKrSnE3tch8WPf+Lf5tvumAxU 8DQNxmGaV0LhIK+bRd55Bpac96yHN9TqSQHW8lyhymvrvVkiI4nVD+ka/fOtuVMKuT yquzA/C+mCPnrOm8wYWEXNNcMIYPtLqvG93vx8YndpDRbLZxSa+hWbpou5qfdLr/8B CBk+5yYDpqJt2c8fobyEju3csHskbpl3SmfkPrSQppZSYDRtNhsqJ7cbrEoioE3y9S DJo+5Z8+IrctQ== Date: Tue, 27 Oct 2026 13:59:52 +0000 To: gdb-patches@sourceware.org, simark@simark.ca From: Jan Vrany Cc: Jan Vrany Subject: [PATCH v2] gdb/testsuite: add support for running tests using QEMU Message-ID: <20261027135827.3868769-1-jan@vrany.io> Feedback-ID: 40767693:user:proton X-Pm-Message-ID: cf79c0a821feb718235c5c96ce630f0f9d3b3646 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gdb-patches-bounces~public-inbox=simark.ca@sourceware.org > I think that is really cool, I am in favor of adding it. Naming nit: > perhaps name it qemu-user, I could see us also adding a qemu-system > board to test bare metal. For example, I saw that qemu now has an AVR > port, I would like to make a board based on that. OK, I renamed it. I just tried to save me some typing :-) > > Do you pass a -L to the qemu user binary, or set QEMU_LD_PREFIX? I > found it necessary for the right dynamic linker to be found: > > $ qemu-aarch64 ./a.out > qemu-aarch64: Could not open '/lib/ld-linux-aarch64.so.1': No such f= ile or directory = /0.0s > $ qemu-aarch64 -L /usr/aarch64-linux-gnu ./a.out > Hello! > > If it's always going to be required to pass one or the other, could the > board require that the user specify it, or could it try to guess it > (e.g. "/usr/$triplet")? AFAIK no. If no QEMU_LD_PREFIX is set nor -L passed, QEMU uses default, compiled-in path (if it exists). The value of that default depends, for example in recent Debian it is /usr/gnemul/qemu-aarch64 (/usr/gnemul/qemu-r= iscv64, ...). On my system I have things setup so the "minimal sysroot" is in that it "just works". If one wants to use different prefix, he/she can either set QEMU_LD_PREFIX or QEMU_FLAGS accordingly. In v2 (see below) I have added comment in the qemu-user.exp to clarify this. I have also added a way to force static compilation - this still allows running large set of tests without need to for dynamic linker. > > I ran your patch through Claude, most of the important comments were > about robustnessof the "non-happy" paths, which in my experience is very > important when you're trying to use the board and figuring things out. > I would suggest giving all the failure modes a try to see if the error > reported is sensible and the user experience is good. > Thanks. I hope I addressed all issues in v2 except following: > > - gdb/testsuite/boards/qemu.exp:179: `exit 0` terminates the runtest > process, not just the current test. In a serial run, all the > remaining tests are skipped silently. Use `unsupported "..."` with a > message and `return -1` instead; gdb_run_cmd already handles a > non-zero return (lib/gdb.exp:549). This is intentional. My understanding is that in GDB testsuite there's no distinction between test "setup" and tests itself in a sense that if setup fails, then the tests are meaningless. So in one fail to connect to the target, the failure is noted and testing proceeds, leading to inevitable timeouts and tests are recorded as failures. The "exit 0" here is to avoid these timeouts for tests that cannot work. It may cause some tests that could work to be silently skipped but seems to me as an unfortunate consequence of lack of setup-test-teardown structure in GDB tests. If simply returning a failure and let the rest of tests to timeout is preferable, I have no problem. Thanks! Jan -- 8< -- gdb/testsuite: add support for running tests using QEMU While debugging failing RISC-V tests, I found myself once again in a situation where running tests easily using QEMU usermode emulation would be useful. Since this is something that resurfaces every now and then [1], I decided to have another stab at it. This commit adds a new "board" file - qemu-user.exp - that allows running tests using QEMU usermode emulation. The main goal is to make it easier to test changes on multiple different architectures. However using QEMU to run tests is no substitute for running tests on real hardware (or in a VM). See limitations below. The implementation based on simavr.exp and on an earlier patch by Zeck S [2] and somewhat simplified. Unlike previous attempts, test programs are no compiled statically by default in order to support broader set of testcases. A static compilation can be forced by setting CFLAGS_FOR_TARGET=3D-static. It also supports more targets (new targets should be easy to add). Not all test that pass on real hardware pass when run under usermode QEMU. Some failures stem from limitations of QEMU's GDB stub - it does not support non-stop mode, multiple threads, hardware watchpoints. In some other cases GDB crashes, perhaps due to a bug in either GDB or QEMU. I have made no attempt to investigate these failures in detail. Running all tests in gdb.base on x86_64 Linux using qemu-aarch64 I got: # of unexpected core files=091 # of expected passes=0966182 # of unexpected failures=09558 # of expected failures=0920 # of known failures=09=0929 # of untested testcases=0914 # of unresolved testcases=094 # of unsupported tests=09146 # of paths in test names=097 # of duplicate test names=093 Running same set of tests using qemu-x86_64 I got: # of unexpected core files=091 # of expected passes=0964988 # of unexpected failures=092527 # of expected failures=0920 # of known failures=09=0929 # of untested testcases=098 # of unresolved testcases=095 # of unsupported tests=09174 # of paths in test names=097 # of duplicate test names=095 [1]: https://inbox.sourceware.org/gdb/938d2b5574b94d9584b758eee3c0c064@ti.c= om/ [2]: https://inbox.sourceware.org/gdb-patches/20231204024634.419445-3-zeck6= 54321@gmail.com/ Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=3D33590 --- gdb/testsuite/boards/qemu-user.exp | 235 +++++++++++++++++++++++++++++ 1 file changed, 235 insertions(+) create mode 100644 gdb/testsuite/boards/qemu-user.exp diff --git a/gdb/testsuite/boards/qemu-user.exp b/gdb/testsuite/boards/qemu= -user.exp new file mode 100644 index 00000000000..286940f16a4 --- /dev/null +++ b/gdb/testsuite/boards/qemu-user.exp @@ -0,0 +1,235 @@ +# Copyright 2026 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# This file is a dejagnu "board file" and is used to run the testsuite +# using QEMU's usermode emulation. Use --target to specify desired target. +# +# To use specific QEMU, pass custom options to QEMU, target toolchain to u= se or +# to pass custom options to the compiler, set environment variable QEMU, +# QEMU_FLAGS, CROSS or CFLAGS_FOR_TARGET (see examples below). +# +# Note that to execute dynamically linked programs with qemu-user, a minim= al +# sysroot is required. It contains the target system's dynamic linker and = the +# shared libraries needed by the program. If it is not located at the defa= ult +# path, either set QEMU_LD_PREFIX to the sysroot path or set QEMU_FLAGS to +# "-L /path/to/sysroot". +# +# To avoid need for minimal sysroot altogether, force static compilation b= y +# setting CFLAGS_FOR_TARGET to "-static" (see example below). Note however +# that some tests need dynamic linking (for example, gdb.base/solib-vanish= .exp) +# +# Example usage: +# +# make check RUNTESTFLAGS=3D'--target=3Driscv64-unknown-linux-gnu --ta= rget_board=3Dqemu-user' +# +# make check QEMU=3Dqemu-arm CROSS=3D/opt/gcc-8-armhf/bin/arm-linux-gn= ueabihf- \ +# RUNTESTFLAGS=3D'--target=3Darm-unknown-linux-gnueabihf --target_b= oard=3Dqemu-user' +# +# make check CFLAGS_FOR_TARGET=3D-static \ +# RUNTESTFLAGS=3D'--target=3Daarch64-unknown-linux-gnu --target_boa= rd=3Dqemu-user' +# + +# Load this for "gdb_target_cmd_ext" +load_lib gdbserver-support.exp + +# +# Setup +# +set qemu "" +set qemu_flags "" +set cross "" + +switch -glob $target_triplet { + "x86_64-*-linux-gnu" { +=09set qemu "qemu-x86_64" +=09set cross "x86_64-linux-gnu-" + } + "riscv64-*-linux-gnu" { +=09set qemu "qemu-riscv64" +=09set cross "riscv64-linux-gnu-" + } + "aarch64-*-linux-gnu" { +=09set qemu "qemu-aarch64" +=09set cross "aarch64-linux-gnu-" + } + "arm-*-linux-gnueabihf" { +=09set qemu "qemu-arm" +=09set cross "arm-linux-gnueabihf-" + } + "powerpc-*-linux-gnu" { +=09set qemu "qemu-ppc" +=09set cross "powerpc-linux-gnu-" + } + "powerpc64le-*-linux-gnu" { +=09set qemu "qemu-ppc64le" +=09set cross "powerpc64le-linux-gnu-" + } + # + # Add more here if needed. + # + default { +=09error "Target $target_triplet not yet supported by board" + } +} + +# +# Allow user to specify QEMU binary and/or custom options by setting +# QEMU and/or QEMU_FLAGS environment variables. +# +if [info exists ::env(QEMU)] { + set qemu $::env(QEMU) +} +if [info exists ::env(QEMU_FLAGS)] { + set qemu_flags $::env(QEMU_FLAGS) +} + +# +# Allow user to specify target toolchain by setting CROSS environment vari= able +# +if [info exists ::env(CROSS)] { + set cross $::env(CROSS) +} + +# +# Allow user to pass custom flags to the compiler. +# +if [info exists ::env(CFLAGS_FOR_TARGET)] { + set CFLAGS_FOR_TARGET $::env(CFLAGS_FOR_TARGET) +} + +set qemu_spawn_id "" + +set_board_info compiler "${cross}gcc" +set_board_info c++compiler "${cross}g++" + +set_board_info use_gdb_stub 1 +set_board_info gdb_protocol "remote" +set_board_info gdb,do_reload_on_run 1 +set_board_info gdb,nofileio 1 +set_board_info exit_is_reliable 1 +# QEMU does not support HW watchpoints +set_board_info gdb,no_hardware_watchpoints 1 + +# +# Helper function to run QEMU. Returns the port number on which QEMU +# stub listens or 0 on failure. +# +proc qemu_spawn { file inferior_args } { + global qemu_spawn_id + global qemu + global qemu_flags + + # + # Close any previous QEMU instance. + # + if { $qemu_spawn_id !=3D "" } { +=09verbose -log "qemu: closing previous spawn id $qemu_spawn_id" +=09if [catch { close -i $qemu_spawn_id } !=3D 0] { +=09 warning "qemu: failed to close connection to previous qemu instance= " +=09} + +=09# some tests get QEMU/its gdb stub +=09# into a state where the process doesn't want to die +=09# so -nowait to not hang the entire test run +=09wait -nowait -i $qemu_spawn_id +=09set qemu_spawn_id "" + } + + + # + # Run QEMU. + # + while 1 { +=09set portnum [expr {1024+round(rand()*(65535-1024))}] +=09set cmd "spawn -noecho $qemu $qemu_flags -g $portnum $file $inferior_ar= gs" + +=09verbose -log "Spawning qemu: $cmd" + +=09if {[catch { eval $cmd } err ]} { +=09 verbose -log "Failed to run QEMU: $err" +=09 return 0 +=09} +=09set qemu_spawn_id $spawn_id + +=09# QEMU stub doesn't do inferior I/O through GDB. But we can +=09# talk to the program using QEMU's tty instead. +=09global inferior_spawn_id +=09set inferior_spawn_id $spawn_id + +=09expect { +=09 -i $qemu_spawn_id +=09 -timeout 1 +=09 -re ".*Address already in use.*" { +=09=09# Port already in use. Cleanup and retry. +=09=09verbose -log "Port $portnum is already in use." +=09=09if [catch { close -i $qemu_spawn_id } !=3D 0] { +=09=09 warning "failed to terminate QEMU instance" +=09=09} +=09=09wait -i $qemu_spawn_id +=09=09continue +=09 } +=09 eof { +=09=09verbose -log "QEMU terminated prematurely." +=09=09return 0 +=09 } + +=09} + +=09verbose -log "QEMU spawned with spawn id $qemu_spawn_id, pid [exp_pid $= qemu_spawn_id], stub listening at $portnum" +=09break + } + + return $portnum +} + + +# +# gdb_reload -- load a file into the target. +# +proc gdb_reload { {inferior_args {}} } { + global last_loaded_file + set portnum [qemu_spawn $last_loaded_file $inferior_args] + + if { $portnum =3D=3D 0 } { +=09return 1 + } + + set res [gdb_target_cmd_ext "remote" ":$portnum"] + + if { $res =3D=3D 2 } { +=09unsupported "Cannot connect to QEMU stub (likely because QEMU does not = support non-stop mode)." +=09# QEMU does not support non-stop mode. Exit here to avoid subsequent +=09# failures / timeouts. +=09exit 0 + } + return [expr {$res =3D=3D 0 ? 0 : 1}] +} + +proc mi_gdb_target_load { } { + global last_loaded_file + set portnum [qemu_spawn $last_loaded_file ""] + + if { $portnum =3D=3D 0 } { +=09return -1 + } + + if { [mi_gdb_target_cmd "remote" ":$portnum"] !=3D 0 } { +=09return -1 + } + + return 0 +} + +set GDBFLAGS "${GDBFLAGS} -iex \"set auto-connect-native-target off\"" --=20 2.53.0