Vendor branches on sourceware.org's binutils-gdb repo

Vendor branches on sourceware.org's binutils-gdb repo

[Edjunior Barbosa Machado]

Hi all,

at the time when binutils/gdb was moving to git, there has been some
discussion in the mailing list [1] about the possibility of hosting
vendor branches on sourceware.org's binutils-gdb git repository but, as
far as I understood, there has been no agreement about this policy.

We already maintain community vendor branches for glibc (on
sourceware.org) and gcc (on gnu.org), and we'd like to do the same for
binutils-gdb. The idea is to create separate namespaces, i.e.
ibm/gdb/7.7 and ibm/binutils/2.24. Those branches will only store
patches under GPL and with a proper copyright assignment.

Any comments? Objections?

Thanks and regards,
--
Edjunior Barbosa Machado
Linux Technology Center, IBM Systems & Technology Group

[1] https://sourceware.org/ml/gdb/2013-10/threads.html#00147

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Sergio Durigan Junior]

On Saturday, April 05 2014, Edjunior Barbosa Machado wrote:

> Hi all,

Hey :-).

> We already maintain community vendor branches for glibc (on
> sourceware.org) and gcc (on gnu.org), and we'd like to do the same for
> binutils-gdb. The idea is to create separate namespaces, i.e.
> ibm/gdb/7.7 and ibm/binutils/2.24. Those branches will only store
> patches under GPL and with a proper copyright assignment.
>
> Any comments? Objections?

Hm, just a comment, but nothing major or blocker.  I understand the need
for vendor branches, but I also think that we should make more use of
git's distributed model.  For example, why can't Company X (I am not
criticizing anyone particularly, really) create and maintain its own git
repository, with all the necessary branches there?  Wouldn't that be
better than (a) "polluting" sourceware's repository and (b) putting an
extra pressure on sourceware's infra?

I may be missing some detail here, and if that is the case then I am
sorry for creating unecessary noise, but at first glance I couldn't come
up with a decent answer for my question above.

P.S.: Before I forget, this comment/question really applies to everyone
who is still using sourceware as their "personal" git repo.

P.S. 2: It is also worth mentioning that my intention is *not* to make
people create GitHub accounts and start doing things there.  Aside from
the fact that GitHub uses non-free software to run its services (and
also serves non-free Javascript), IMHO it also encourages people to stay
within its "social network" walls and makes it a little harder to get
advantage from git's distributed protocol.

-- 
Sergio

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Aaro Koskinen]

Hi,

On Sun, Apr 06, 2014 at 03:02:49AM -0300, Sergio Durigan Junior wrote:
> Hm, just a comment, but nothing major or blocker.  I understand the need
> for vendor branches, but I also think that we should make more use of
> git's distributed model.  For example, why can't Company X (I am not
> criticizing anyone particularly, really) create and maintain its own git
> repository, with all the necessary branches there?  Wouldn't that be
> better than (a) "polluting" sourceware's repository and (b) putting an
> extra pressure on sourceware's infra?

I think it's very useful for users to have all vendor branches
in a single repository. At least with glibc this has helped me a lot
(as a user) when identifying and cherry-picking needed fixes to
my own systems.

A.

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Joel Brobecker]

> On Sun, Apr 06, 2014 at 03:02:49AM -0300, Sergio Durigan Junior wrote:
> > Hm, just a comment, but nothing major or blocker.  I understand the need
> > for vendor branches, but I also think that we should make more use of
> > git's distributed model.  For example, why can't Company X (I am not
> > criticizing anyone particularly, really) create and maintain its own git
> > repository, with all the necessary branches there?  Wouldn't that be
> > better than (a) "polluting" sourceware's repository and (b) putting an
> > extra pressure on sourceware's infra?
> 
> I think it's very useful for users to have all vendor branches
> in a single repository. At least with glibc this has helped me a lot
> (as a user) when identifying and cherry-picking needed fixes to
> my own systems.

FWIW: I have found that the extra branches are just making me download
lots of commits that I have no use for, and I suspect that this is the
case for many of us. That's the default behavior, and most users will be
impacted by those. While it's convenient, it is also very easy to pull
a branch from another repository. I won't strongly object to vendor
branches, especially since we already have some, but I think it's
unnecessary. I do strongly suggest, however, that they all hosted under
the same namespace and then split into sub-namespaces (Eg.
"vendor/[vendor-name]/[branch-name]").

-- 
Joel

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Michael Matz]

Hi,

On Sun, 6 Apr 2014, Joel Brobecker wrote:

> > I think it's very useful for users to have all vendor branches in a 
> > single repository. At least with glibc this has helped me a lot (as a 
> > user) when identifying and cherry-picking needed fixes to my own 
> > systems.
> 
> FWIW: I have found that the extra branches are just making me download 
> lots of commits that I have no use for, and I suspect that this is the 
> case for many of us. That's the default behavior, and most users will be 
> impacted by those. While it's convenient, it is also very easy to pull a 
> branch from another repository.

But it's not necessarily easy for the vendor to _host_ that other 
repository.  And IMHO, the current 288 MB for binutils-gdb git objects 
aren't enough to discourage vendor branches (and if you're worried about 
the download size it's equally easy to simply not pull those branches).


Ciao,
Michael.

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Joel Brobecker]

> But it's not necessarily easy for the vendor to _host_ that other 
> repository.  And IMHO, the current 288 MB for binutils-gdb git objects 
> aren't enough to discourage vendor branches (and if you're worried about 
> the download size it's equally easy to simply not pull those branches).

I don't think it's "equally easy" to not pull those branches.
If it is, I'd like to have the recipe for "pull all branches except
some", and I will put it on the GDB wiki. On the other hand, there
are many free services on the web to host git repositories. Lastly,
I find that we have way too many branches in our repository at
the moment, drowning the ones that really matter to most (the release
branches, basically). But then again, I also understand that it makes
it easier for people to find that code if they go looking for it.
Hence the not-so-strong objection, although, this issue can be very
easily addressed by documenting those branches on the GDB website or
wiki. It should be anyway, so that people have an idea of what the
branch is about. I think that's what people did with the Archer project.

-- 
Joel

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Mark Kettenis]

> Date: Mon, 7 Apr 2014 16:27:22 +0200 (CEST)
> From: Michael Matz <matz@suse.de>
> 
> Hi,
> 
> On Sun, 6 Apr 2014, Joel Brobecker wrote:
> 
> > > I think it's very useful for users to have all vendor branches in a 
> > > single repository. At least with glibc this has helped me a lot (as a 
> > > user) when identifying and cherry-picking needed fixes to my own 
> > > systems.
> > 
> > FWIW: I have found that the extra branches are just making me download 
> > lots of commits that I have no use for, and I suspect that this is the 
> > case for many of us. That's the default behavior, and most users will be 
> > impacted by those. While it's convenient, it is also very easy to pull a 
> > branch from another repository.
> 
> But it's not necessarily easy for the vendor to _host_ that other 
> repository.

Really?  Are there really companies that are active in the Free
Software community that don't have the infrastructure to host a
relatively small git repo?

> And IMHO, the current 288 MB for binutils-gdb git objects aren't
> enough to discourage vendor branches (and if you're worried about
> the download size it's equally easy to simply not pull those
> branches).

Size is an issue for me.  I try to support GDB on many platforms, some
of which are somewhat old or low power and don't have a lot of disk
storage.  I'm already running into problems on some of my machines.
And every time git messes up my repo because I run out of disk space
(or just because it doesn't seem to properly implement DWIM) I need to
fetch everything all over again.

So how do I tell git to only clone master and not give me everybody
else's shit?  Last time I tried to do that, it simply didn't work.

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Andreas Schwab]

Mark Kettenis <mark.kettenis@xs4all.nl> writes:

> So how do I tell git to only clone master and not give me everybody
> else's shit?  Last time I tried to do that, it simply didn't work.

git-clone(1):

       --[no-]single-branch
           Clone only the history leading to the tip of a single branch,
           either specified by the --branch option or the primary branch
           remote’s HEAD points at.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

RE: Vendor branches on sourceware.org's binutils-gdb repo

[Matthew Fortune]

> > So how do I tell git to only clone master and not give me everybody
> > else's shit?  Last time I tried to do that, it simply didn't work.
> 
> git-clone(1):
> 
>        --[no-]single-branch
>            Clone only the history leading to the tip of a single branch,
>            either specified by the --branch option or the primary branch
>            remote’s HEAD points at.

If space is a problem and the purpose of the clone is mainly to perform a
build then you may also be interested in --depth. It comes with a health
warning but really reduces the overheads if it is suitable for you:

       --depth <depth>
           Create a shallow clone with a history truncated to the specified
           number of revisions. A shallow repository has a number of
           limitations (you cannot clone or fetch from it, nor push from 
           nor into it), but is adequate if you are only interested in the
           recent history of a large project with a long history, and would 
           want to send in fixes as patches.

Matthew

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Mark Kettenis]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 585 bytes --]

> From: Andreas Schwab <schwab@suse.de>
> Date: Mon, 07 Apr 2014 17:06:08 +0200
> 
> Mark Kettenis <mark.kettenis@xs4all.nl> writes:
> 
> > So how do I tell git to only clone master and not give me everybody
> > else's shit?  Last time I tried to do that, it simply didn't work.
> 
> git-clone(1):
> 
>        --[no-]single-branch
>            Clone only the history leading to the tip of a single branch,
>            either specified by the --branch option or the primary branch
>            remote’s HEAD points at.

But I can't push from a repository cloned like that can I?

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Markus Trippelsdorf]

On 2014.04.07 at 07:47 -0700, Joel Brobecker wrote:
> > But it's not necessarily easy for the vendor to _host_ that other 
> > repository.  And IMHO, the current 288 MB for binutils-gdb git objects 
> > aren't enough to discourage vendor branches (and if you're worried about 
> > the download size it's equally easy to simply not pull those branches).
> 
> I don't think it's "equally easy" to not pull those branches.
> If it is, I'd like to have the recipe for "pull all branches except
> some", and I will put it on the GDB wiki. 

That doesn't work in git AFAIK. What you can do is list all branches
that you like to pull in .git/config. The following example if from my
gcc git tree, but you get the idea:

[remote "origin"]
	url = git://gcc.gnu.org/git/gcc.git
	fetch = refs/heads/master:refs/remotes/origin/master
	fetch = refs/heads/gcc-4_7-branch:refs/remotes/origin/gcc-4_7-branch
	fetch = refs/heads/gcc-4_8-branch:refs/remotes/origin/gcc-4_8-branch

-- 
Markus

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Michael Matz]

Hi,

On Mon, 7 Apr 2014, Mark Kettenis wrote:

> > But it's not necessarily easy for the vendor to _host_ that other 
> > repository.
> 
> Really?  Are there really companies that are active in the Free Software 
> community that don't have the infrastructure to host a relatively small 
> git repo?

It's not the infrastructure.  It's policies.  Some companies have a policy 
of not publishing any code outside except through a heavy process, which 
might include a white-list of where it may be published/hosted.  
sourceware.org is on that white-list already.  It may be very painful to 
extend that white-list.

> > And IMHO, the current 288 MB for binutils-gdb git objects aren't
> > enough to discourage vendor branches (and if you're worried about
> > the download size it's equally easy to simply not pull those
> > branches).
> 
> Size is an issue for me.  I try to support GDB on many platforms, some
> of which are somewhat old or low power and don't have a lot of disk
> storage.  I'm already running into problems on some of my machines.
> And every time git messes up my repo because I run out of disk space
> (or just because it doesn't seem to properly implement DWIM) I need to
> fetch everything all over again.
> 
> So how do I tell git to only clone master and not give me everybody
> else's shit?  Last time I tried to do that, it simply didn't work.

master only is easy:
  git config remote.origin.fetch +refs/heads/master:refs/remotes/origin/master

By default that setting is "+refs/heads/*:refs/remotes/origin/*" which 
fetches everything under refs/heads/ (recursively unfortunately).  
Furthermore the above aren't really filename globs (so e.g. 
refs/heads/[^v]* doesn't work), but you can have multiple such refspecs.

So to retrieve just a subset of branches you can name them all 
individually (git-config --add), or we need to employ some namespace 
scheme where nothing is directly in refs/heads/, but only in 
subdirectories of that.  Then we could have:

 +refs/heads/default/*:refs/remotes/origin/default/*
 +refs/heads/vendor/*:refs/remotes/origin/vendor/*

(Or further subsetted to only some vendors).  I'll admit that naming 
subsets of branches to fetch is not that "easy" without changes in the 
repository structure.


Ciao,
Michael.

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Andreas Schwab]

Mark Kettenis <mark.kettenis@xs4all.nl> writes:

> But I can't push from a repository cloned like that can I?

Yes, you can.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Michael Matz]

Hi,

On Mon, 7 Apr 2014, Mark Kettenis wrote:

> > > So how do I tell git to only clone master and not give me everybody
> > > else's shit?  Last time I tried to do that, it simply didn't work.
> > 
> > git-clone(1):
> > 
> >        --[no-]single-branch
> >            Clone only the history leading to the tip of a single branch,
> >            either specified by the --branch option or the primary branch
> >            remote???s HEAD points at.
> 
> But I can't push from a repository cloned like that can I?

You can.  git fetch will continue to only fetch the named (or default) 
branch, and git push will only push that branch.  It's a normal git 
repository in all respects, just with the fetch= config variable 
restricted to just one head (instead of everything under refs/heads/).


Ciao,
Michael.

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Joel Brobecker]

> It's not the infrastructure.  It's policies.  Some companies have a policy 
> of not publishing any code outside except through a heavy process, which 
> might include a white-list of where it may be published/hosted.  
> sourceware.org is on that white-list already.  It may be very painful to 
> extend that white-list.

Companies that do have such a white list are an acceptable exception,
IMO, and I would be OK with having their branches in our sourceware.org
repository.

-- 
Joel

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Andreas Schwab]

Michael Matz <matz@suse.de> writes:

> So to retrieve just a subset of branches you can name them all 
> individually (git-config --add)

git remote set-branches --add

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 894 bytes --]

On Sat 05 Apr 2014 17:12:09 Edjunior Barbosa Machado wrote:
> at the time when binutils/gdb was moving to git, there has been some
> discussion in the mailing list [1] about the possibility of hosting
> vendor branches on sourceware.org's binutils-gdb git repository but, as
> far as I understood, there has been no agreement about this policy.
> 
> We already maintain community vendor branches for glibc (on
> sourceware.org) and gcc (on gnu.org), and we'd like to do the same for
> binutils-gdb. The idea is to create separate namespaces, i.e.
> ibm/gdb/7.7 and ibm/binutils/2.24. Those branches will only store
> patches under GPL and with a proper copyright assignment.
> 
> Any comments? Objections?

i'd use it for Gentoo if it were available.  i've gotten all the gdb changes 
merged at this point (i just do random back ports), but binutils is another 
story ...
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Matt Rice]

On Mon, Apr 7, 2014 at 7:52 AM, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>> Date: Mon, 7 Apr 2014 16:27:22 +0200 (CEST)
>> From: Michael Matz <matz@suse.de>
>>
>> Hi,
>>
>> On Sun, 6 Apr 2014, Joel Brobecker wrote:
>>
>> > > I think it's very useful for users to have all vendor branches in a
>> > > single repository. At least with glibc this has helped me a lot (as a
>> > > user) when identifying and cherry-picking needed fixes to my own
>> > > systems.
>> >
>> > FWIW: I have found that the extra branches are just making me download
>> > lots of commits that I have no use for, and I suspect that this is the
>> > case for many of us. That's the default behavior, and most users will be
>> > impacted by those. While it's convenient, it is also very easy to pull a
>> > branch from another repository.
>>
>> But it's not necessarily easy for the vendor to _host_ that other
>> repository.
>
> Really?  Are there really companies that are active in the Free
> Software community that don't have the infrastructure to host a
> relatively small git repo?

I generally prefer working in the distributed model, but I think it
makes sense for sourceware to host it, for copyright assignment
purposes
it makes it clear and easy to decide if effort on a branch can be
merged by a third party into master, which happens on occasion.

I would probably be happier if there was a separate repository
binutils+gdb-vendor.git or some such though.

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Joel Brobecker]

> I would probably be happier if there was a separate repository
> binutils+gdb-vendor.git or some such though.

That would definitely adress all the issues I have. Not sure if
the sourceware sysadmins would agree or not to host it.  If it was
accepted, we should probably move the old vendor branches to that
parallel repository as well, and start documenting its existence
on both GDB and binutils websites/wikis.

-- 
Joel

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Stan Shebs]

On 4/5/14 1:12 PM, Edjunior Barbosa Machado wrote:
> Hi all,
> 
> at the time when binutils/gdb was moving to git, there has been some
> discussion in the mailing list [1] about the possibility of hosting
> vendor branches on sourceware.org's binutils-gdb git repository but, as
> far as I understood, there has been no agreement about this policy.

One small point in favor of vendor branches is that it ensures we
have the code already in hand - if a project goes on the back burner,
or changes ownership, or an admin leaves, the vendor repo can disappear
overnight.

On the question of space and activity, are there vendor branches that
are really so much more active than our trunk?  I would expect them
be to relatively quiet most of the time, vendor branches typically
having fewer people doing work on them.

Stan

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Doug Evans]

On Wed, Apr 9, 2014 at 10:58 AM, Stan Shebs <stanshebs@earthlink.net> wrote:
> On 4/5/14 1:12 PM, Edjunior Barbosa Machado wrote:
>> Hi all,
>>
>> at the time when binutils/gdb was moving to git, there has been some
>> discussion in the mailing list [1] about the possibility of hosting
>> vendor branches on sourceware.org's binutils-gdb git repository but, as
>> far as I understood, there has been no agreement about this policy.
>
> One small point in favor of vendor branches is that it ensures we
> have the code already in hand - if a project goes on the back burner,
> or changes ownership, or an admin leaves, the vendor repo can disappear
> overnight.
>
> On the question of space and activity, are there vendor branches that
> are really so much more active than our trunk?  I would expect them
> be to relatively quiet most of the time, vendor branches typically
> having fewer people doing work on them.

fwiw,
I really like irker's reports on #gdb for the trunk.
OTOH, the S/N ratio on #gdb will monotonically drop over time as use
of vendor branches on binutils-gdb scales up.  There are days when the
S/N ratio on #gdb would drop to barely useful with the branches that
are there now.  If it is really hard to *only* show trunk commits on
#gdb, that is, IMO, a strong argument in favor of putting vendor
branches in a different repo.
*If* one really wanted irker even with it having to report commits in
all branches then one *could* have a separate channel, but it's not my
first choice (or possibly second choice even).
OTOH, #gdb has been relatively quiet for vendor-branch commits
recently, maybe this has already been fixed?

I'm ok with a binutils-gdb-vendor repo or some such.  Seems like it
should be trivial for overseers to set up (I'm assuming people won't
want a copy of trunk in this repo with a cron job, or some such, to
keep it up to date, or any other such time commitment from overseers).
 Whether a separate vendor repo uses materially more resources, I
don't know.

Re: Vendor branches on sourceware.org's binutils-gdb repo

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 243 bytes --]

did we decide anything ?  seems like we didn't.
	https://sourceware.org/ml/binutils/2014-04/msg00068.html

i'm debating whether to move Gentoo's branches over like i do
already for glibc:
	https://sourceware.org/git/?p=glibc.git;a=heads
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]