From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by sourceware.org (Postfix) with ESMTPS id AA46B385DC0B for ; Thu, 23 Apr 2020 17:54:02 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org AA46B385DC0B Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=embecosm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=andrew.burgess@embecosm.com Received: by mail-wr1-x431.google.com with SMTP id x18so7889645wrq.2 for ; Thu, 23 Apr 2020 10:54:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=embecosm.com; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=d6IrIB5I944Ym9imc4/7k8zNCi2oH4ep1jeiK7G8Dt0=; b=bnP8BfEl09mRZe8c8kpn9TFbiN29ez8HztaSX7jyo9/7d6LnxGaFDCn/cCT2zP1udE mrdJZeBotuZoyPjmUWsK/I3q2HEKWJB7LSAq4rZglWPc5s5eJXYdStmTgNCY4KeVJPfT VqiwJXqu3TyM1fzozCaNmADaNGzqK82Pd4HWfFaFdJdMaW1i/nWeLImR3KIj7DcuRCZ1 uy4n5Q9PzsuG3us718Mdj8uZhUEOQwKeoFkuItGzhnOg0xOSFn1TBRHSDjunQ90vnjqO oVvh3xA81A6EX/GjMmUOYnsx+3u1ynLLqhmiaRaSOIjk245+Koyn5dWVsc/fJeOQ2Tm5 FymQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=d6IrIB5I944Ym9imc4/7k8zNCi2oH4ep1jeiK7G8Dt0=; b=mTXkuM2t5f+qQHw7qrLC8H3qyzz/XFZsvEEXeAsi5Cf9pSc9BanMWgG4KZoDadM85x rgzdJCa69TF/LAbpeqdwpeuF+wgC6F0juBlXlXXbLiQ2j6XUIYnxRYA38mfJIWBhHtoV imqMMtQ+naDFZ7jnq/9rPFMjRS6H4OLodKrWWZSWG8z1Q/MNZ1hPE241aTHCD6kcNGBw q31UgSBN7XDQdZlmQNMHjs867ZwuECSXymtbYX3DuHy3T7O3oPrGWarKJy07GkHdTIer SHunZwUv4MYM/jSQjdqeyfhqRbkovcU8ulgg9Ex3kaN3J7qixFIiWRmJAdkbLS4nUoWz vFEA== X-Gm-Message-State: AGi0PuYEI1sVPRH/Pseh+n4rvETwfrkant8XXiNMj1vS8fxCZWGkRyNl lHt3Zc9tQsBQblzqNeEpByTkp5xxgXA= X-Google-Smtp-Source: APiQypLrFaYv/8ng1wlDYmjwoiYvM1YTAb/0Tz81edq5v9QG9Ke0An8dZ7nksqXC796xLQ45m/uMYg== X-Received: by 2002:adf:f34e:: with SMTP id e14mr6141603wrp.193.1587664441565; Thu, 23 Apr 2020 10:54:01 -0700 (PDT) Received: from localhost (host81-151-181-184.range81-151.btcentralplus.com. [81.151.181.184]) by smtp.gmail.com with ESMTPSA id 19sm4573990wmo.3.2020.04.23.10.54.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Apr 2020 10:54:00 -0700 (PDT) From: Andrew Burgess To: gdb-patches@sourceware.org Subject: [PATCH 0/4] Automatic detection of test name problems Date: Thu, 23 Apr 2020 18:53:54 +0100 Message-Id: X-Mailer: git-send-email 2.25.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-13.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Apr 2020 17:54:04 -0000 This started as fixing some test names in gdb.btrace/multi-inferior.exp, but then I wanted to try and prevent these problems creeping back in again. So, this series extends our testing framework so that Dejagnu will now produce two new result types, these are: # of paths in test names # of duplicate test names So, the test gdb.btrace/multi-inferior.exp (without the fix in this series) would produce results like this: === gdb Summary === # of expected passes 10 # of paths in test names 2 This indicates that two test names include either the build or source path, which should really be resolved. Alternatively, the test gdb.ada/fun_overload_menu.exp currently produces results like this: === gdb Summary === # of expected passes 8 # of duplicate test names 2 Indicating that two test names are duplicated. It is expected that users will treat these new results just like FAILs, they should never increase these numbers when adding new tests. It's possible for a test run to include both of the new test result types. I've updated the dg-extract-results scripts so that the new result types are merged over when running the tests in parallel. I know this change needs to go through GCC, but I wanted to discuss these patches here first, before posting the dg-extract-results changes to the gcc list. I'll merge the first patch in this series (the gdb.btrace/multi-inferior.exp fix) relatively soon as obvious. Feedback and thoughts welcome. Thanks, Andrew -- Andrew Burgess (4): gdb/testsuite: Remove build paths from test names gdb/testsuite: Detect and warn if paths are used in test names gdb/testsuite: Detect and warn about duplicate test names contrib: Handle GDB specific test result types contrib/ChangeLog | 5 + contrib/dg-extract-results.py | 4 +- contrib/dg-extract-results.sh | 10 ++ gdb/testsuite/ChangeLog | 17 +++ gdb/testsuite/gdb.btrace/multi-inferior.exp | 6 +- gdb/testsuite/lib/check-test-names.exp | 108 ++++++++++++++++++++ gdb/testsuite/lib/gdb.exp | 1 + 7 files changed, 148 insertions(+), 3 deletions(-) create mode 100644 gdb/testsuite/lib/check-test-names.exp -- 2.25.3