From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by sourceware.org (Postfix) with ESMTPS id D2C6F385DC0B for ; Thu, 23 Apr 2020 17:54:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org D2C6F385DC0B Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=embecosm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=andrew.burgess@embecosm.com Received: by mail-wr1-x432.google.com with SMTP id k13so7883000wrw.7 for ; Thu, 23 Apr 2020 10:54:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=embecosm.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jUjHCbiga5VZB3jVtXwwZdiE7YMN1gX/IO16019bcuY=; b=KivuCm+cwM8y5cIbGlAixBOfCKAsQ8iA6YwXIJFi1juHRI1hr7vFVA+ZS7F4NqU2ao j37ctNXKxNOkJhg6Gw98rH0SnPOKwx808OBBUGmY/UBH28ApZkeVCidCS1JtabGgOUbC e79zGUpJQYpMs9jGnp1llQYkMX0vbpBluIVlqm3bwzE1zrE+KSiokbroiNqWoFfEVd2B Q7lPbWCQB5gRV8z4IIEpceSFH+NWFmJYmKT2jQs82dqI+37NkAXvfBUszG7k7/m3rljO g2tvBbaj+D9shBumEPUsAAyJyZ5ijhC2QrPOnQG5siR5AFTqGEoChYdkbfvlGijcSrYb 92OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jUjHCbiga5VZB3jVtXwwZdiE7YMN1gX/IO16019bcuY=; b=mMqiDO+U/QhUDh96zxgyqFLU/33UzaOhofnSzMjnsc5lgZpU5EQSWpAlGn0o8j2kV+ IxZz6n+DYMd3fGLT+0ZOwqPPUOl5yi/Aw5gq9JmEh3J0kcQJEUES32hDtX7RyIwYczbB tJARFr5kNNubSbJczaHNUfmYQYxrfHRt1+K1LpoUrQIs6BSorPs00tBYOJtN5Gi93olV CUxrecexELSfZxYrVfvj5jYVUX80VLo0SgTmsLTWusbXKTHCbhWuXSGA8X+A2KfkmDyg es7pSxLNq13KrHh603ajCh5iLV+poOZj9TSbTYu/qZ3XQKrTiVGI3X6gTf7fnZj9TYy1 rbbw== X-Gm-Message-State: AGi0PuZqluburbB7OvpAT6Rv1DLVSfG0g1Smx2CTQ8x5BX2RScmKbcT5 Cmo451tKjZd+fs3LicYL48Iya2w4O6M= X-Google-Smtp-Source: APiQypIDmS+L0OZqRZiZY5wA/5hddGW+nENECQKKiGc+yQ9cFpsgILiL48kYU+NSWHTupYm62i/6Dg== X-Received: by 2002:a5d:5085:: with SMTP id a5mr6618088wrt.394.1587664444596; Thu, 23 Apr 2020 10:54:04 -0700 (PDT) Received: from localhost (host81-151-181-184.range81-151.btcentralplus.com. [81.151.181.184]) by smtp.gmail.com with ESMTPSA id z22sm4457450wma.20.2020.04.23.10.54.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Apr 2020 10:54:03 -0700 (PDT) From: Andrew Burgess To: gdb-patches@sourceware.org Subject: [PATCH 2/4] gdb/testsuite: Detect and warn if paths are used in test names Date: Thu, 23 Apr 2020 18:53:56 +0100 Message-Id: X-Mailer: git-send-email 2.25.3 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-23.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Apr 2020 17:54:07 -0000 A new library is introduced that hooks into the core of Dejagnu and detects when a test's name includes either the source or build paths. If any offending test names are detected then Dejagnu will print a new result type, '# of paths in test names'. Users should treat this result type just like other bad results types, and aim not to increase this number. Currently for a local run on my machine, I don't see any offending test names, but it is possible that different targets, or different configurations, might currently be breaking the no paths rule. In order to get this working I have needed to wrap two core Dejagnu functions, log_summary, and reset_vars. Relying on core functions that are not part of any API is always going to be risky, given the relatively slow rate of Dejagnu change this is probably OK for now, and we can possibly upstream some changes to Dejagnu that would allow this functionality to be supported in a more official way later on. Currently if the tests are run in parallel mode the new result type is not merged into the combined summary file so users will need to run in non-parallel mode to check this result. A later commit will fix this. gdb/testsuite/ChangeLog: * lib/gdb.exp: Include check-test-names.exp library. * lib/check-test-names.exp: New file. --- gdb/testsuite/ChangeLog | 5 ++ gdb/testsuite/lib/check-test-names.exp | 79 ++++++++++++++++++++++++++ gdb/testsuite/lib/gdb.exp | 1 + 3 files changed, 85 insertions(+) create mode 100644 gdb/testsuite/lib/check-test-names.exp diff --git a/gdb/testsuite/lib/check-test-names.exp b/gdb/testsuite/lib/check-test-names.exp new file mode 100644 index 00000000000..6377ace3fc3 --- /dev/null +++ b/gdb/testsuite/lib/check-test-names.exp @@ -0,0 +1,79 @@ +# Copyright 2020 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# This library provides some protection against the introduction of +# tests that include either the source of build paths in the test +# name. When a test includes the path in its test name it is harder +# to compare results between two runs of GDB from different trees. + +# Count the number of test names that contain either the source path, +# or the build path. +set paths_in_test_names 0 + +# Check if MESSAGE contains either the source path or the build path. +# This will result in test names that can't easily be compared between +# different runs of GDB. +# +# Any offending paths in message cause PATHS_IN_TEST_NAMES to be +# incremented. +proc check_test_names { message } { + global srcdir objdir + global paths_in_test_names + + foreach pattern [list $srcdir $objdir] { + if { [ regexp $pattern $message ] } { + # Count each test just once. + incr paths_in_test_names + return + } + } +} + +# Arrange for CHECK_TEST_NAMES to be called. +set local_record_procs(pass) "check_test_names" +set local_record_procs(fail) "check_test_names" +set local_record_procs(xfail) "check_test_names" +set local_record_procs(kfail) "check_test_names" +set local_record_procs(xpass) "check_test_names" +set local_record_procs(kpass) "check_test_names" +set local_record_procs(unresolved) "check_test_names" +set local_record_procs(untested) "check_test_names" +set local_record_procs(unsupported) "check_test_names" + +# Wrapper around the global Dejagnu LOG_SUMMARY procedure. Prints a +# warning if any tests were found that contained either the source or +# build paths. +rename log_summary original_log_summary +proc log_summary { args } { + global paths_in_test_names + + # If ARGS is the empty list then we don't want to pass a single + # empty string as a parameter here. + eval "original_log_summary $args" + + if { $paths_in_test_names > 0 } { + clone_output "# of paths in test names\t${paths_in_test_names}" + } +} + +# Wrapper around the global Dejagnu RESET_VARS procedure, resets our +# PATHS_IN_TEST_NAMES counter. +rename reset_vars original_reset_vars +proc reset_vars {} { + global paths_in_test_names + + original_reset_vars + set paths_in_test_names 0 +} diff --git a/gdb/testsuite/lib/gdb.exp b/gdb/testsuite/lib/gdb.exp index 8418c3d8753..a294b53f2c3 100644 --- a/gdb/testsuite/lib/gdb.exp +++ b/gdb/testsuite/lib/gdb.exp @@ -29,6 +29,7 @@ load_lib libgloss.exp load_lib cache.exp load_lib gdb-utils.exp load_lib memory.exp +load_lib check-test-names.exp global GDB -- 2.25.3