Add python black formatting step to gdb buildbot

Mark Wielaard
2022-07-17T22:11:10Z

[-- Attachment #1: Type: text/plain, Size: 2865 bytes --]

Hi,

On irc Simon requested a python black formatting step for the gdb
buildbot. And I saw Tom just fixed the last few formatting issues. So
I added a "gdb black step" to the gdb-fedora-x86_64 builder (it seems
fine to just run this on one builder instead on all - 19 now! - gdb
builders).

It currently succeeds:
https://builder.sourceware.org/buildbot/#/builders/74/builds/721/steps/5/logs/stdio

And it will sent email if it detects a python file under gdb that it
would reformat.

Note that it will also run for a user try branch, so if you don't have
black locally installed, but want to know if the buildbot would flag
some new python code you can submit a job:

- git checkout -b python-frob
- hack, hack, hack... OK, looks good to submit
- git commit -a -m "Awesome python hack"
- git push origin python-frob:users/mark/try-python-frob
- ... wait for the emails to come in or watch buildbot logs ...

If there are any other jobs that would be beneficial to run on the gdb
sources please let us know, On irc I suggested running ARI
(A.R. Index), but it seem ARI isn't super popular. Also it wouldn't
really be clear what "success" means for an ARI run, or how to report
the results in a useful way.

Cheers,

Mark

BTW. For those coming (virtually) to Cauldron Frank and I will run a little
BoF to discuss how to make these things better and/or more useful.
https://gcc.gnu.org/wiki/cauldron2022

Title: BoF - Sourceware GNU Toolchain Infrastructure and beyond

Presenters: Mark Wielaard and Frank Eigler

Abstract: The Sourceware infrastructure is continually improving. Just
like our other services, such as bugzilla, mailinglists and git repos,
we like to provide zero maintenance infrastructure for tracking and
automation of patches, testing and analyzing testresults. This BoF is
for everybody who likes to discuss (and wants to help with) automating
the infrastructure to make contributing to our projects more fun and
more productive.

Topics to discuss include the shared buildbot [1]. Whether we need
more/other arches/distro support. Which builders are most beneficial
to projects. How buildbot should report issues. Whether to use the
buildbot to automate other tasks like updating documentation,
websites, generate release tars or updating bugzilla. How to use git
user try branches. Taking advantage of the Bunsen [2] testrun cluster
analysis, per-testrun testcase search/browse engines, search
operators, testsuite summary (vs detail) grids. Patch tracking using
patchwork [3] integrated with buildbot and the CICD trybot [4]. How to
use the sourcehut mirror [5]. And anything else that would make you
more productive and happy contributing.

[1] https://builder.sourceware.org/
[2] https://builder.sourceware.org/testruns/
[3] https://patchwork.sourceware.org/
[4] https://sourceware.org/glibc/wiki/CICDDesign.
[5] https://sr.ht/~sourceware/

[-- Attachment #2: 0001-Add-python-black-formatter-step-to-gdb-fedora-x86_64.patch --]
[-- Type: text/x-diff, Size: 4066 bytes --]

From 1b7b630eb56703ed87c94cfb65bc73558ea57bda Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Sun, 17 Jul 2022 23:34:23 +0200
Subject: [PATCH] Add python black formatter step to gdb-fedora-x86_64 builder

Add 'black' to the fedora-latest container file. Add a
gdb_targets_all_black_factory that includes a gdb_black_step for the
gdb-fedora-x86_64 builder.
---
.../containers/Containerfile-fedora-latest | 1 +
builder/master.cfg | 27 ++++++++++++++++++-
2 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/builder/containers/Containerfile-fedora-latest b/builder/containers/Containerfile-fedora-latest
index e2e9eae..e8e4ad4 100644
--- a/builder/containers/Containerfile-fedora-latest
+++ b/builder/containers/Containerfile-fedora-latest
@@ -13,6 +13,7 @@ RUN dnf upgrade -y && \
xmlto xmlto-tex fop poppler-utils libxslt docbook-dtds docbook-style-xsl \
libxml2-devel dos2unix dpkg koji python3-devel python3-koji python3-mock \
python3-rpm python3-pyxdg mailcap openmpi-devel gdb help2man wget xxhash-devel \
+ black \
buildbot-worker && \
dnf clean all

diff --git a/builder/master.cfg b/builder/master.cfg
index d94170a..36ec4ae 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -2098,6 +2098,12 @@ gdb_make_step = steps.Compile(
'all-gdb', 'all-gdbserver'],
name='make',
haltOnFailure=True)
+# python formatter
+gdb_black_step = steps.Test(
+ workdir='binutils-gdb',
+ command=['python3', '-m', 'black', '--check', '--diff', 'gdb'],
+ name='python black formatter',
+ haltOnFailure=False, flunkOnFailure=True)
# Only a small subset of tests that are fast and known to PASS.
gdb_test_exp = ("TESTS= "
"gdb.base/break-always.exp "
@@ -2196,6 +2202,25 @@ gdb_targets_all_factory.addSteps(bunsen_logfile_upload_steps([
(["../gdb-build/gdb/testsuite/gdb.sum", "../gdb-build/gdb/testsuite/gdb.log", "../gdb-build/gdb/config.log"],"gdb")],
tagsuffix="/extended-gdbserver"))

+gdb_targets_all_black_factory = util.BuildFactory()
+gdb_targets_all_black_factory.addStep(gdb_git_step)
+gdb_targets_all_black_factory.addStep(gdb_rm_step)
+gdb_targets_all_black_factory.addStep(gdb_configure_targets_all_step)
+gdb_targets_all_black_factory.addStep(gdb_make_step)
+gdb_targets_all_black_factory.addStep(gdb_black_step)
+gdb_targets_all_black_factory.addStep(gdb_check_step)
+gdb_targets_all_black_factory.addSteps(bunsen_logfile_upload_steps([ # only a few tests being run but still
+ (["../gdb-build/gdb/testsuite/gdb.sum", "../gdb-build/gdb/testsuite/gdb.log", "../gdb-build/gdb/config.log"],"gdb")],
+ tagsuffix="/native"))
+gdb_targets_all_black_factory.addStep(gdb_check_native_gdbserver_step)
+gdb_targets_all_black_factory.addSteps(bunsen_logfile_upload_steps([
+ (["../gdb-build/gdb/testsuite/gdb.sum", "../gdb-build/gdb/testsuite/gdb.log", "../gdb-build/gdb/config.log"],"gdb")],
+ tagsuffix="/gdbserver"))
+gdb_targets_all_black_factory.addStep(gdb_check_native_extended_gdbserver_step)
+gdb_targets_all_black_factory.addSteps(bunsen_logfile_upload_steps([
+ (["../gdb-build/gdb/testsuite/gdb.sum", "../gdb-build/gdb/testsuite/gdb.log", "../gdb-build/gdb/config.log"],"gdb")],
+ tagsuffix="/extended-gdbserver"))
+
# Used for armhf only because of
# https://sourceware.org/bugzilla/show_bug.cgi?id=28561
gdb_factory_make_only = util.BuildFactory()
@@ -2228,7 +2253,7 @@ gdb_fedora_x86_64_builder = util.BuilderConfig(
readContainerFile('fedora-latest')},
workernames=vm_workers,
tags=["gdb", "fedora", "x86_64"],
- factory=gdb_targets_all_factory)
+ factory=gdb_targets_all_black_factory)
c['builders'].append(gdb_fedora_x86_64_builder)

gdb_try_fedora_x86_64_builder = util.BuilderConfig(
--
2.30.2