Re: Status of the AArch* builders

Sergio Durigan Junior
2018-10-25T16:26:21Z

On Wednesday, October 24 2018, Alan Hayward wrote:

>> On 17 Oct 2018, at 19:45, Sergio Durigan Junior <sergiodj@redhat.com> wrote:
>>
>>> For now, I have disabled the email notifications. IME there's always
>>> something that we need to tweak in the first days to make sure that the
>>> builders are running fine. So if you could just keep an eye on the
>>> builds and make sure that everything is OK, I'd appreciate.
>>>
>>> After a few days have passed, I will enable the email notifications.
>>> Regular test failures will be sent to the gdb-testers@ ml, and breakages
>>> will be sent to gdb-patches@.
>>
>> For the record, I've now enabled the e-mail notifications for the
>> Aarch64 builders. I've also added them to the list of Try builders, so
>> it's possible to submit try jobs to them.
>>
>
> Currently the AArch64 builders keep going back to failed regressions. This is
> mostly due to *all* the gdb.threads and gdb.server tests being racy on AArch64
> Ubuntu. Keeping it down to 16 threads has reduced the frequency, but not
> removed it.
> I’m currently trying to fix up as many of the AArch64 test failures across the
> whole test suite as I can. I suspect fixing the threaded issue is going to
> take a while longer.

TBH, all builders suffer from this problem. They all have racy tests,
and even though I tried to implement a system to detect such tests and
exclude them from the reports that are sent to gdb-testers, they still
sneak in the reports. That's the main (and sole?) reason why
gdb-testers is currently impossible to follow, and people don't really
read it.

> In the meantime is it possible to update just the AArch64 buildbot scripts so
> they don’t take into account the thread/server tests for regressions? Either
> remove the tests before running, or grep them out of the results. Not sure if
> that’s possible with the way it’s set up (and not sure where to look). This
> would then give us confidence with the rest of the tests. They can be
> reinstated when stable again.

I think it can be done, but it's not so trivial, and I'm busy with other
stuff right now :-/. Sorry about that.

I'll try to take a look at this problem during the weekend. BTW, the
configuration files for the GDB BuildBot live at:

https://git.sergiodj.net/gdb-buildbot.git/

Thanks,

--
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible