On 2/25/24 5:40 AM, Mark Wielaard wrote:
> Please don't be sarcastic. We do the exact same thing with our CI at
> https://builder.sourceware.org/ which gets new commits all the time
> sometimes fetching code for 20 builders at a time in parallel from
> different machines. So it is known to work. We aren't DDoSing
> ourselves. So we just have to figure out why it doesn't seem to work
> for Tomasz.

I think it's not entirely unreasonable for a project to internally place
its own demands on its own resources, with the knowledge of how much it
has in the way of resources...

especially when, per the buildbot logs, you are using persistent git
clones and fetching new changes on every buildbot run.

This is rather different from someone saying that he is building all
your projects on every commit when you didn't ask for that to be done,
and is NOT using git, and as a result of not using git, is getting
throttled by mysterious intricacies of gitweb vs cgit.

In fact, if I'm reading Tomasz correctly, in order to do this as an rpm
specfile build, he has automation hooked up to scan for every commit
landing on the binutils repo, then appending that commit as a gitweb
downloaded patchfile to the `Patch:` section of the spec.

So for example for the binutils-gdb case, that currently stands at 1626
separate gitweb downloaded patch files, and as soon as an email
notification comes in that the gdb repo has gotten a 1627th commit since
the latest tag, all those 1626 gitweb formatted patch files are going to
be downloaded again from scratch, except it is 1627 now.

This is why caching is not just:

> But yes, having a cache certainly helps.

Rather, it is "having a cache is basic politeness unto others".

It doesn't matter whether it works, and it doesn't matter whether
avoiding gitweb specifically and using cgit can make it better. It is
simply rude to do this altogether. Use a cache, or use git clones, or
best of all, use both -- just like builder.sourceware.org does.


Refusing to use a cache is called executing a DDoS campaign.

I am not being sarcastic.... okay, the "clapping" thing was a bit
sarcastic. But the DDoS Linux bit was meant in absolute seriousness.


-- 
Eli Schwartz