From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 19299 invoked by alias); 14 Jan 2015 23:43:19 -0000 Mailing-List: contact gdb-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-owner@sourceware.org Received: (qmail 19288 invoked by uid 89); 14 Jan 2015 23:43:18 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-3.0 required=5.0 tests=AWL,BAYES_00,SPF_HELO_PASS,SPF_PASS,T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-GCM-SHA384 encrypted) ESMTPS; Wed, 14 Jan 2015 23:43:07 +0000 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t0ENh65U009215 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Wed, 14 Jan 2015 18:43:06 -0500 Received: from localhost (dhcp-10-15-16-169.yyz.redhat.com [10.15.16.169]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t0ENh5g2011335 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 14 Jan 2015 18:43:06 -0500 From: Sergio Durigan Junior To: GDB Development Subject: [ANNOUNCEMENT] GDB BuildBot X-URL: http://blog.sergiodj.net Date: Wed, 14 Jan 2015 23:43:00 -0000 Message-ID: <87bnm1exo6.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-IsSubscribed: yes X-SW-Source: 2015-01/txt/msg00026.txt.bz2 Hi all, It took a bit more time than I was planning, but finally I am back to announce our BuildBot. I really expect that it will help us improve our codebase and catch regressions/failures sooner (in fact, it already helped me to catch at least 2 regressions). The initial idea was to deploy our BuildBot master directly on sourceware, but unfortunately this was not possible. So, after trying to find other places, I decided to use my own personal server. However, my main goal is to turn this project into something official for the GDB project, and I do hope that using my own server does not give a false sense of "this is Sergio's pet-project". Having said that, the current address for our BuildBot is: You will see many pages there, each one displaying the builds in a particular way. The one that I find most useful is the Waterfall view: You can already see the 8 build configurations (a.k.a. "builders") that I created to test Fedora in a variety of ways; it is obviously possible to create as many builders as we want, and I intend to expand this list very soon (I'm trying to obtain access to a PPC64 Fedora machine). Each build is composed by a number of steps. There are preparation steps (i.e., cleaning up the build directory, updating the git repository), compilation steps (i.e., configure && make && make check, each one with customizable parameters), a step to compare the test results (kindly contributed by Tom Tromey), and another two steps to save the test results in a nice way (details later). The comparison of the test results is done by using a baseline, which is the best testsuite run done so far (think of it as the best gdb.sum we had so far). For each testsuite run, we get its results and compare against this baseline; if something that was PASSing turns to FAIL, it is reported as a regression. If a test that was FAILing turns to PASS, it is added in the baseline. If no regressions are found, then nothing is done; otherwise, an e-mail is sent to with a report of what failed. Therefore, if you are not subscribed to this list, I strongly recommend you to do so! Still on the comparison topic, another thing worth mentioning is the presence of non-deterministic testcases in our testsuite. To deal with this, a special file called "xfail" can be created for each builder, which will contain lines to be ignored in the gdb.sum file (i.e., test failures to be ignored). For now, this file needs to be updated by hand, but I am already thinking about a way to semi-automatize the process. If you want me to update the xfail file for a certain builder, please send me an e-mail (and feel free to include the GDB mailing list in the Cc). After struggling *a lot* trying to find a nice way to store the testsuite results, I came up with a solution using git as a DB-like backend. You can see the test results here: You will notice that, for each builder, there is a git repository. Inside it, I store the gdb.{sum,log} files. When a new commit has been tested by BuildBot, it stored the files in the repository, and commits the result. Additionally, it also creates a tag whose name is composed by a timestamp and the git SHA1 that has been tested from the GDB repository. For example, suppose that I pushed a new patch to the GDB repository, and its SHA1 is 0a1b2c3d. Then, after BuildBot has tested my commit on the builder Fedora-x86_64, it will store the resulting gdb.{sum.log} files on the repository named Fedora-x86_64, commit them, and create a tag named 20150114-182610-0a1b2c3d. The cgit interface's main purpose, for now, is just to allow everybody to have access to the full testsuite logs that were generated when certain commit was tested. It is also not possible (yet) to clone the repositories containing the logs; maybe I will enable it in the future. The configuration files for our BuildBot are currently located at: I intend to submit them soon upstream, and put them under the contrib/ directory. The BuildBot also offers an IRC bot that can stay in the channel and send messages if some regression is found, but I haven't enabled this because it can be a little verbose for some people. However, if the majority wants, I can easily enable it. I am maintaining a wiki page describing the details of our BuildBot here: And I am looking for new buildslaves to add to our setup! If you are interested, take a look at the wiki page above and do not hesitate to contact me. I personally believe that the following setups would help a lot: - ARM - PPC - Some Windows/MinGW machine I am also open for suggestions/improvements, of course. I still think our configuration needs a lot of polishing, and I already have some ideas that I would like to implement (testing other branches, enabling the "Try mode", which will allow people to send their patches to be tested, etc.), but IMNSHO it is good to be used "in production" already. I hope it helps! Before I forget: special thanks to Tom Tromey (for starting to work on a previous GDB BuildBot in 2011), and Jan Kratochvil for valuable feedback during the configuration of the current BuildBot. Cheers, -- Sergio GPG key ID: 0x65FC5E36 Please send encrypted e-mail if possible http://sergiodj.net/