A Proposal to Move to Git

A Proposal to Move to Git

[Tom Tromey]

I'd like to move gdb and binutils from CVS to Git.  I've done much of
the preliminary work and I will do the remainder, including the
inevitable follow-up bug-fixing.

I've read all the previous threads on the topic.  I think I have
internalized the big issues but it is possible that I am missing
something.  I'm sure you'll let me know.


I think we should move specifically to git for two reasons.  First, it
is obviously better than CVS for the majority of work.  Second, point
#1 is borne out by observing that most active gdb developers are
already using git.  (I can't speak for binutils developers, though I
do my rare binutils forays in git as well.)


One principle I'm following in this proposal is to make the minimal
change possible.  That is, I want to focus on the conversion to git.
Often times these discussions veer off into other process changes --
removing ChangeLogs, stuff like that -- but for this change we ought
to concern ourselves solely with the rollout of git, and leave other
changes for a later date.

So, if you have other changes you want to propose, I would appreciate
it if you would hold them until the transition is complete.

Note that there are even some absurd cases of this I am leaving
in-tree; for example the requirements, obsolete with git, to put a
date into a branch name.


The basics of the plan are as outlined by Joseph Myers:

    http://gcc.gnu.org/ml/gcc/2011-03/msg00486.html

For the purposes of this discussion I think you can focus on 6.b -- a
shared gdb+binutils repository.

The reason for a shared repository is simply that binutils and gdb
share a substantial amount of code, mainly BFD, but other things as
well.

This gives the change minimal impact.  It is not zero impact, but:

1. It is superior for all of us to build the whole tree, to avoid
   those (rare) occasions where BFD changes break other parts of the
   build;

2. You can already build just a subset of the tree;

3. This affects just the regular developers, not releases.


I have been using http://sourceware.org/bugzilla/show_bug.cgi?id=14768
to track the to-do items.

My understanding of the task list is:

* Update the gdbadmin scripts.
  I've done this though I have not tested them.

* Update the BFD daily date-updating commit.
  Not done.

* Port log_accum_bugzillafied to git and set up git commit email.
  I've done this and tested the post-receive parts.

* Update DJ's script that auto-merges some changes from GCC.
  Note that I think it will have to continue to merge to the 'src' CVS
  repository, for the benefit of projects left behind.
  Not done.

* Examine gdb and binutils documentation to see what needs to be
  updated.  This means looking at the texinfo manuals, the web sites,
  and the gdb wiki.
  I have patches for this.


Once the infrastructure bits are in place, there is the matter of the
conversion:

* Convert the tree.  We can perhaps reuse parts of the existing
  conversion process for this.  I will try a test conversion at least
  once.  Since the old history is available
  (see http://sourceware.org/ml/binutils/2012-10/msg00407.html),
  we can try to merge it before conversion.
  There seem to be several approach we can take here.  I am
  investigating the options, but I'm interested in your expert
  advice.

* Mark the various converted directories as read-only in CVS.
  This can be done via the commitinfo file.


I'd like to do the final switch around mid-September.  Not sooner,
because I am going to be away for a little while near the end of
August, and I want to be available to fix problems.

Tom

Re: A Proposal to Move to Git

[Andrew Pinski]

On Tue, Aug 20, 2013 at 2:12 PM, Tom Tromey <tromey@redhat.com> wrote:
> I'd like to move gdb and binutils from CVS to Git.  I've done much of
> the preliminary work and I will do the remainder, including the
> inevitable follow-up bug-fixing.

Since newlib is part of the same repo, it would make sense to split
out newlib the same way you are splitting out gdb and binutils.

Thanks,
Andrew Pinski

>
> I've read all the previous threads on the topic.  I think I have
> internalized the big issues but it is possible that I am missing
> something.  I'm sure you'll let me know.
>
>
> I think we should move specifically to git for two reasons.  First, it
> is obviously better than CVS for the majority of work.  Second, point
> #1 is borne out by observing that most active gdb developers are
> already using git.  (I can't speak for binutils developers, though I
> do my rare binutils forays in git as well.)
>
>
> One principle I'm following in this proposal is to make the minimal
> change possible.  That is, I want to focus on the conversion to git.
> Often times these discussions veer off into other process changes --
> removing ChangeLogs, stuff like that -- but for this change we ought
> to concern ourselves solely with the rollout of git, and leave other
> changes for a later date.
>
> So, if you have other changes you want to propose, I would appreciate
> it if you would hold them until the transition is complete.
>
> Note that there are even some absurd cases of this I am leaving
> in-tree; for example the requirements, obsolete with git, to put a
> date into a branch name.
>
>
> The basics of the plan are as outlined by Joseph Myers:
>
>     http://gcc.gnu.org/ml/gcc/2011-03/msg00486.html
>
> For the purposes of this discussion I think you can focus on 6.b -- a
> shared gdb+binutils repository.
>
> The reason for a shared repository is simply that binutils and gdb
> share a substantial amount of code, mainly BFD, but other things as
> well.
>
> This gives the change minimal impact.  It is not zero impact, but:
>
> 1. It is superior for all of us to build the whole tree, to avoid
>    those (rare) occasions where BFD changes break other parts of the
>    build;
>
> 2. You can already build just a subset of the tree;
>
> 3. This affects just the regular developers, not releases.
>
>
> I have been using http://sourceware.org/bugzilla/show_bug.cgi?id=14768
> to track the to-do items.
>
> My understanding of the task list is:
>
> * Update the gdbadmin scripts.
>   I've done this though I have not tested them.
>
> * Update the BFD daily date-updating commit.
>   Not done.
>
> * Port log_accum_bugzillafied to git and set up git commit email.
>   I've done this and tested the post-receive parts.
>
> * Update DJ's script that auto-merges some changes from GCC.
>   Note that I think it will have to continue to merge to the 'src' CVS
>   repository, for the benefit of projects left behind.
>   Not done.
>
> * Examine gdb and binutils documentation to see what needs to be
>   updated.  This means looking at the texinfo manuals, the web sites,
>   and the gdb wiki.
>   I have patches for this.
>
>
> Once the infrastructure bits are in place, there is the matter of the
> conversion:
>
> * Convert the tree.  We can perhaps reuse parts of the existing
>   conversion process for this.  I will try a test conversion at least
>   once.  Since the old history is available
>   (see http://sourceware.org/ml/binutils/2012-10/msg00407.html),
>   we can try to merge it before conversion.
>   There seem to be several approach we can take here.  I am
>   investigating the options, but I'm interested in your expert
>   advice.
>
> * Mark the various converted directories as read-only in CVS.
>   This can be done via the commitinfo file.
>
>
> I'd like to do the final switch around mid-September.  Not sooner,
> because I am going to be away for a little while near the end of
> August, and I want to be available to fix problems.
>
> Tom

Re: A Proposal to Move to Git

[Joseph S. Myers]

On Tue, 20 Aug 2013, Andrew Pinski wrote:

> On Tue, Aug 20, 2013 at 2:12 PM, Tom Tromey <tromey@redhat.com> wrote:
> > I'd like to move gdb and binutils from CVS to Git.  I've done much of
> > the preliminary work and I will do the remainder, including the
> > inevitable follow-up bug-fixing.
> 
> Since newlib is part of the same repo, it would make sense to split
> out newlib the same way you are splitting out gdb and binutils.

Setting up a repository for 6.c in my list of pieces into which to divide 
the src repository is a matter for the newlib maintainers which should not 
have any dependencies in either direction on what any other project does.  
The same applies to each of the other separate pieces I identified: the 
maintainers should decide for themselves if and when to move out of the 
src repository and what version control system to move to if so.

This is about 6.b (binutils+gdb, including sim and some gdbtk pieces but 
not cgen and not tcl, tk or itcl).

http://gcc.gnu.org/ml/gcc/2011-03/msg00486.html

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: A Proposal to Move to Git

[Joel Brobecker]

I've glanced at the plan, and I have to say that I am pretty excited
if this becomes reality :). Thanks for doing this, Tom!

> I'd like to do the final switch around mid-September.  Not sooner,
> because I am going to be away for a little while near the end of
> August, and I want to be available to fix problems.

Also, I think it would be better to just get the 7.7 release out
the door with CVS, allowing us then a few months to transition to
git without impacting any release.  Chance are that, if switching
over to git, I'll rewrite all release scripts rather than adapt them,
as a lot of stuff could be done better, as well as completely
automated instead of semi-manual.

-- 
Joel

Re: A Proposal to Move to Git

[Paul_Koning]

On Aug 20, 2013, at 5:12 PM, Tom Tromey <tromey@redhat.com> wrote:

> I'd like to move gdb and binutils from CVS to Git.  ...

I like it.  I'd say the same (somewhat more strongly) if you had proposed Subversion.  But the main point is to go to any available real version control system, and git certainly qualifies.

	paul

Re: A Proposal to Move to Git

[Phil Muldoon]

On 20/08/13 22:12, Tom Tromey wrote:
> I'd like to move gdb and binutils from CVS to Git.  I've done much of
> the preliminary work and I will do the remainder, including the
> inevitable follow-up bug-fixing.

I have no technical points, other than a hearty approval of this move.

git is stable, well used, and far more efficient than equivalent
source control versions.  There are many reasons, which I won't go
into here (but a quick search on the net will tell you why).

Also there are many, many transition guides for CVS users, one example
of which is here:

https://www.kernel.org/pub/software/scm/git/docs/v1.4.4.4/cvs-migration.html

That's not definitive, but an example.

Anyway I look forward to the transition.  And thanks for doing this.

Cheers,

Phil

Re: A Proposal to Move to Git

[Eli Zaretskii]

> From: Tom Tromey <tromey@redhat.com>
> CC: Binutils Development <binutils@sourceware.org>
> Date: Tue, 20 Aug 2013 15:12:41 -0600
> 
> I'd like to move gdb and binutils from CVS to Git.  I've done much of
> the preliminary work and I will do the remainder, including the
> inevitable follow-up bug-fixing.

Thanks for doing this.

> Once the infrastructure bits are in place, there is the matter of the
> conversion:
> 
> * Convert the tree.  We can perhaps reuse parts of the existing
>   conversion process for this.  I will try a test conversion at least
>   once.  Since the old history is available
>   (see http://sourceware.org/ml/binutils/2012-10/msg00407.html),
>   we can try to merge it before conversion.
>   There seem to be several approach we can take here.  I am
>   investigating the options, but I'm interested in your expert
>   advice.
> 
> * Mark the various converted directories as read-only in CVS.
>   This can be done via the commitinfo file.

What about possible issues related to the differences in how CVS and
git do things?  A few things come to mind:

 . Tags.  CVS tags file revisions, while git tags the whole tree.  Can
   this cause loss of information?

 . Files/directories that are deleted or moved/renamed.  Will the
   information from the Attic be extracted, and will git be able to
   intuit renaming and handle deleted files correctly?  (One type of
   file which we constantly rename is ChangeLog.)

 . How will the committers be mapped?  AFAIU, CVS and git represent
   committers differently.

 . Which branches will be converted?  Do we want to convert all or
   just some, and if the latter, which ones will be left out?

 . Finally, what about some QA, once the initial conversion is done?
   Can we run some scripts to be sure nothing is lost?

It's quite possible that some, or maybe even all of these issues are
already solved by the existing conversion procedure.  But I thought
I'd raise these (and other similar) issues, since I don't think they
can be postponed for later -- once we convert and start working with
git, the history that we have at that point will be practically
immutable, so whatever will be missed during the conversion will never
happen.  Therefore, I think we need to be aware of any such issues and
review them carefully before we decide how to handle them.

Re: A Proposal to Move to Git

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>  . How will the committers be mapped?

I have a complete mapping from accounts to real names.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: A Proposal to Move to Git

[Eli Zaretskii]

> From: Andreas Schwab <schwab@suse.de>
> Cc: Tom Tromey <tromey@redhat.com>, gdb@sourceware.org,	binutils@sourceware.org
> Date: Wed, 21 Aug 2013 17:32:33 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >  . How will the committers be mapped?
> 
> I have a complete mapping from accounts to real names.

Thanks.  If this means that your mapping can be (or already is) used
for the conversion, then I guess it's one concern less.

Re: A Proposal to Move to Git

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@suse.de>
>> Cc: Tom Tromey <tromey@redhat.com>, gdb@sourceware.org,	binutils@sourceware.org
>> Date: Wed, 21 Aug 2013 17:32:33 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >  . How will the committers be mapped?
>> 
>> I have a complete mapping from accounts to real names.
>
> Thanks.  If this means that your mapping can be (or already is) used

I'm using it for <git://repo.or.cz/src>.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: A Proposal to Move to Git

[Steinar Bang]

>>>>> Eli Zaretskii <eliz@gnu.org>:

>  . Files/directories that are deleted or moved/renamed.  Will the
>    information from the Attic be extracted, and will git be able to
>    intuit renaming and handle deleted files correctly?  (One type of
>    file which we constantly rename is ChangeLog.)

If you mean follow the history with "git blame" and "git log" across
renames, then the answer is: it depends on how that rename is
represented in the CVS repository.

If a file is removed with one name, and the exact same file is added
with a different name, git itself should be able to detect this as a
rename, no matter what the conversion tool does.

But there are several ways to represent a rename in CVS... issue #1 for
cvs2svn (from which cvs2git derives) lists the possible ways a rename
can be done, and discusses the heuristics to detect this.  I see that
this issue is still open:
 http://cvs2svn.tigris.org/issues/show_bug.cgi?id=1

Re: A Proposal to Move to Git

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Wed, 21 Aug 2013 20:40:04 +0200
> Cc: binutils@sourceware.org
> 
> >>>>> Eli Zaretskii <eliz@gnu.org>:
> 
> >  . Files/directories that are deleted or moved/renamed.  Will the
> >    information from the Attic be extracted, and will git be able to
> >    intuit renaming and handle deleted files correctly?  (One type of
> >    file which we constantly rename is ChangeLog.)
> 
> If you mean follow the history with "git blame" and "git log" across
> renames, then the answer is: it depends on how that rename is
> represented in the CVS repository.

Yes, and I would think we should consider extracting as much
information from CVS about these renames as practically possible,
because having a discontinuous history is very inconvenient when you
need to do some forensics.

Re: A Proposal to Move to Git

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> Yes, and I would think we should consider extracting as much
> information from CVS about these renames as practically possible,

This amount is basically zero.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: A Proposal to Move to Git

[Florian Weimer]

* Eli Zaretskii:

>  . Files/directories that are deleted or moved/renamed.  Will the
>    information from the Attic be extracted, and will git be able to
>    intuit renaming and handle deleted files correctly?

Git uses contents-based heuristics to reconstruct renames.  The
downside is that it is impossible to make sure that a rename is
recognized by Git.  The advantage is that users cannot forget to
register renames on the version control side.

The CVS checkout reconstruction has to be sufficiently accurate and
preserve at a copy of the file at each intermediate checkout version.
Such a correct conversion is usually quite difficult to achieve (CVS
history is usually quite bad, due to bugs and user errors), but it's
obviously desirable for other reasons.

Re: A Proposal to Move to Git

[Steve Ellcey]

On Tue, 2013-08-20 at 15:12 -0600, Tom Tromey wrote:

> I'd like to do the final switch around mid-September.  Not sooner,
> because I am going to be away for a little while near the end of
> August, and I want to be available to fix problems.
> 
> Tom

There was a comment on the gdb list about releasing GDB 7.7 before the
transition, I wonder if it would make sense to release a Binutils 2.24
as well?  In general, I am in favor of the transition and I don't think
doing (or not doing) a binutils 2.24 release should be a blocker for the
move to git, but I wouldn't mind seeing a new release before the
transition.

Steve Ellcey
sellcey@mips.com

Re: A Proposal to Move to Git

[Joel Brobecker]

> There was a comment on the gdb list about releasing GDB 7.7 before the
> transition, I wonder if it would make sense to release a Binutils 2.24
> as well?  In general, I am in favor of the transition and I don't think
> doing (or not doing) a binutils 2.24 release should be a blocker for the
> move to git, but I wouldn't mind seeing a new release before the
> transition.

I think it should depend on when 2.24 is scheduled to be released.
2.23 was released end of Oct 2012. Tristan, are you planning on
releasing 2.24 around that time as well? And what do you think of
the suggestion to wait for the release? IIRC, you were telling me
that a lot of the process was manual, so perhaps a switch to git
wouldn't impact you as much for producing the release?

Thanks,
-- 
Joel

Re: A Proposal to Move to Git

[Tristan Gingold]

On Aug 21, 2013, at 5:50 PM, Joel Brobecker wrote:

>> There was a comment on the gdb list about releasing GDB 7.7 before the
>> transition, I wonder if it would make sense to release a Binutils 2.24
>> as well?  In general, I am in favor of the transition and I don't think
>> doing (or not doing) a binutils 2.24 release should be a blocker for the
>> move to git, but I wouldn't mind seeing a new release before the
>> transition.
> 
> I think it should depend on when 2.24 is scheduled to be released.
> 2.23 was released end of Oct 2012. Tristan, are you planning on
> releasing 2.24 around that time as well? And what do you think of
> the suggestion to wait for the release? IIRC, you were telling me
> that a lot of the process was manual, so perhaps a switch to git
> wouldn't impact you as much for producing the release?

Yes, 2.24 should be released soon.
But I'd prefer to use the same SCM for the 2.24.x releases, so
I'd prefer to move to git before 2.24 (if this is possible).

Tristan.

Re: A Proposal to Move to Git

[Richard Earnshaw]

On 21/08/13 16:37, Steve Ellcey wrote:
> On Tue, 2013-08-20 at 15:12 -0600, Tom Tromey wrote:
> 
>> I'd like to do the final switch around mid-September.  Not sooner,
>> because I am going to be away for a little while near the end of
>> August, and I want to be available to fix problems.
>>
>> Tom
> 
> There was a comment on the gdb list about releasing GDB 7.7 before the
> transition, I wonder if it would make sense to release a Binutils 2.24
> as well?  In general, I am in favor of the transition and I don't think
> doing (or not doing) a binutils 2.24 release should be a blocker for the
> move to git, but I wouldn't mind seeing a new release before the
> transition.

If you switch after the releases have been made you also need to
consider how you'll handle 'dot' releases should they be necessary.  Are
they going to come from the CVS source base and then be re-imported into
GIT?  Or, are you going to use a different process to handle them from
that used to produce the main release?

R.

Re: A Proposal to Move to Git

[Joel Brobecker]

> If you switch after the releases have been made you also need to
> consider how you'll handle 'dot' releases should they be necessary.  Are
> they going to come from the CVS source base and then be re-imported into
> GIT?  Or, are you going to use a different process to handle them from
> that used to produce the main release?

It's a good point. I thought we'd keep using CVS for 7.7.x while
transitioning to git for "HEAD". On the other hand, I like Tristan's
confidence, so I'd be OK with switching to git before we create
the 7.7 branch. Also, why force ourselves to use two systems, one
of them being phased out, when we can use just one?

-- 
Joel

Re: A Proposal to Move to Git

[Fred Cooke]

Using two systems in parallel seems to be a terrible idea. One nice
thing with Git is that the release branch, once imported, could be
practiced on locally for honing the new process.

On Thu, Aug 22, 2013 at 4:25 PM, Joel Brobecker <brobecker@adacore.com> wrote:
>> If you switch after the releases have been made you also need to
>> consider how you'll handle 'dot' releases should they be necessary.  Are
>> they going to come from the CVS source base and then be re-imported into
>> GIT?  Or, are you going to use a different process to handle them from
>> that used to produce the main release?
>
> It's a good point. I thought we'd keep using CVS for 7.7.x while
> transitioning to git for "HEAD". On the other hand, I like Tristan's
> confidence, so I'd be OK with switching to git before we create
> the 7.7 branch. Also, why force ourselves to use two systems, one
> of them being phased out, when we can use just one?
>
> --
> Joel

Re: A Proposal to Move to Git

[Mark Kettenis]

> From: Tom Tromey <tromey@redhat.com>
> CC: Binutils Development <binutils@sourceware.org>
> Date: Tue, 20 Aug 2013 15:12:41 -0600
> 
> I'd like to move gdb and binutils from CVS to Git.  I've done much of
> the preliminary work and I will do the remainder, including the
> inevitable follow-up bug-fixing.

I still *hate* git.  Hate it even more now that I've used it for a
project.  It gets in my way; it never does what I want, at least not
by default.  It's slow; updating a tree takes significantly more time
than with cvs as it insists that I stash my local changes first.  It's
buggy; I had it just sit there and spin forever on a git pull on
several occasions.  It encourages the annoying "send the gazillian
diffs in a series again, just because I've fixed a single spelling
mistake in one of them" behaviour.  But it seems I'm the only one in
the world who doesn't "git" it, so I'll shut up.

Except for one tiny question.  How much free space do I need to clone
the future gdb repo and do useful work with it?  Some of my machines
don't have a lot of free disk space.  Sparse checkouts don't really
work.  And it seems you can't do any real development from a shallow
clone.  So if the answer to the my question is that I need at least 1G
of disk space, then there is a problem.

Re: A Proposal to Move to Git

[Fred Cooke]

Binutils, as is, @ 13d568a676e8e6038195199b1f005cdcc88d47f5 is 252meg
including the checkout, of which 82meg is the .git dir, and the
balance the checkout. It's VERY efficient.

On Thu, Aug 22, 2013 at 10:10 PM, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>> From: Tom Tromey <tromey@redhat.com>
>> CC: Binutils Development <binutils@sourceware.org>
>> Date: Tue, 20 Aug 2013 15:12:41 -0600
>>
>> I'd like to move gdb and binutils from CVS to Git.  I've done much of
>> the preliminary work and I will do the remainder, including the
>> inevitable follow-up bug-fixing.
>
> I still *hate* git.  Hate it even more now that I've used it for a
> project.  It gets in my way; it never does what I want, at least not
> by default.  It's slow; updating a tree takes significantly more time
> than with cvs as it insists that I stash my local changes first.  It's
> buggy; I had it just sit there and spin forever on a git pull on
> several occasions.  It encourages the annoying "send the gazillian
> diffs in a series again, just because I've fixed a single spelling
> mistake in one of them" behaviour.  But it seems I'm the only one in
> the world who doesn't "git" it, so I'll shut up.
>
> Except for one tiny question.  How much free space do I need to clone
> the future gdb repo and do useful work with it?  Some of my machines
> don't have a lot of free disk space.  Sparse checkouts don't really
> work.  And it seems you can't do any real development from a shallow
> clone.  So if the answer to the my question is that I need at least 1G
> of disk space, then there is a problem.

Re: A Proposal to Move to Git

[David Miller]

From: Mark Kettenis <mark.kettenis@xs4all.nl>
Date: Thu, 22 Aug 2013 22:10:47 +0200 (CEST)

> So if the answer to the my question is that I need at least 1G
> of disk space, then there is a problem.

It'll probably be on the order of 125MB, something like that.

Re: A Proposal to Move to Git

[Andrew Pinski]

On Thu, Aug 22, 2013 at 1:22 PM, David Miller <davem@davemloft.net> wrote:
> From: Mark Kettenis <mark.kettenis@xs4all.nl>
> Date: Thu, 22 Aug 2013 22:10:47 +0200 (CEST)
>
>> So if the answer to the my question is that I need at least 1G
>> of disk space, then there is a problem.
>
> It'll probably be on the order of 125MB, something like that.

You can also try with "--depth 1" which should reduce the size of the
.git directory.

Thanks,
Andrew Pinski

Re: A Proposal to Move to Git

[Andreas Schwab]

Mark Kettenis <mark.kettenis@xs4all.nl> writes:

> Except for one tiny question.  How much free space do I need to clone
> the future gdb repo and do useful work with it?

The size of the complete history is about half the size of the working
directory.

$ du -sh .git/objects/ .
270M    .git/objects/
529M    .

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: A Proposal to Move to Git

[Tom Tromey]

Mark> Except for one tiny question.  How much free space do I need to clone
Mark> the future gdb repo and do useful work with it?  Some of my machines
Mark> don't have a lot of free disk space.  Sparse checkouts don't really
Mark> work.  And it seems you can't do any real development from a shallow
Mark> clone.  So if the answer to the my question is that I need at least 1G
Mark> of disk space, then there is a problem.

gdb.git is 372M.  That's the bare repository, so no checkout.  It may be
a bit bigger when we mix in binutils, though the final numbers will all
be different for a variety of reasons.  I think 400M is an ok rough
estimate.


I think you have several options here.

I believe you can push from a --single-branch clone.  One of these
weighed in at 304M for me.  That was a checkout, not a bare repository.

Note that a gdb+binutils checkout from CVS is 321M.  So I think that,
whatever the issue is, it will be at the margin.


Alternatively, you can adopt a more laborious approach by doing a
shallow clone on a space-constrained device, and shipping the results to
a push-capable repository.  I've done this at times (not due to space
but because I couldn't conveniently push from the machine I was hacking
on).


Or, maybe you can run git-cvs locally and cvs commit from your space
constrained device into your git repository.  I have never tried this.

Tom

Re: A Proposal to Move to Git

[H.J. Lu]

I assume git repo has both binutils and gdb. Is that possible to
only checkout binutils portion?

-- 
H.J.

Re: A Proposal to Move to Git

[Joel Brobecker]

> I assume git repo has both binutils and gdb. Is that possible to
> only checkout binutils portion?

AFAIK, partial checkouts are not possible with git. That's one of
the reasons I felt it was so tough to get out of CVS.

-- 
Joel

Re: A Proposal to Move to Git

[H.J. Lu]

On Fri, Aug 23, 2013 at 8:55 AM, Joel Brobecker <brobecker@adacore.com> wrote:
>> I assume git repo has both binutils and gdb. Is that possible to
>> only checkout binutils portion?
>
> AFAIK, partial checkouts are not possible with git. That's one of
> the reasons I felt it was so tough to get out of CVS.
>

What is wrong with git mirror?  I only use CVS for checkin and use git
mirror for everything else.  The only minor inconvenience I run into is the
git mirror delay.


-- 
H.J.

Re: A Proposal to Move to Git

[Joel Brobecker]

> What is wrong with git mirror?  I only use CVS for checkin and use git
> mirror for everything else.  The only minor inconvenience I run into is the
> git mirror delay.

Git mirror is an improvement, but still an archaic solution compared
to the power of a git repository. We will have to get out of CVS
eventually, as the tool gets less and less used, so let's do it sooner
rather than later. I don't think that losing partial checkouts is
that big a price to pay.

-- 
Joel

Re: A Proposal to Move to Git

[Paul_Koning]

I've heard that the answer may be better in recent versions.   If not, you can consider Subversion instead, since it certainly has the ability.

	paul

On Aug 23, 2013, at 11:55 AM, Joel Brobecker <brobecker@adacore.com> wrote:

>> I assume git repo has both binutils and gdb. Is that possible to
>> only checkout binutils portion?
> 
> AFAIK, partial checkouts are not possible with git. That's one of
> the reasons I felt it was so tough to get out of CVS.
> 
> -- 
> Joel

Re: A Proposal to Move to Git

[H.J. Lu]

On Fri, Aug 23, 2013 at 9:03 AM,  <Paul_Koning@dell.com> wrote:
> I've heard that the answer may be better in recent versions.   If not, you can consider Subversion instead, since it certainly has the ability.
>

Given that we have git mirror, I don't see the need to switch
to subversion from CVS.  I don't use CVS for my binutils work
anyway.


-- 
H.J.

Re: A Proposal to Move to Git

[NightStrike]

On Fri, Aug 23, 2013 at 12:05 PM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Fri, Aug 23, 2013 at 9:03 AM,  <Paul_Koning@dell.com> wrote:
>> I've heard that the answer may be better in recent versions.   If not, you can consider Subversion instead, since it certainly has the ability.
>>
>
> Given that we have git mirror, I don't see the need to switch
> to subversion from CVS.  I don't use CVS for my binutils work
> anyway.

Subversion handles merging a lot better than CVS (svn merge
--reintegrate works very well).  It was written by people that worked
on CVS with CVS in mind, and so it's trivial to switch over (both in
terms of the repository as well as the users, ie the learning curve is
virtually nonexistent.)  Git users can easily use git-svn, which means
you can satisfy mostly everyone, and still be using current, well
maintained software.

Obviously, the die has already been cast, but there is something nice
about having a centralized server with decentralized clients.  Kind of
the best of both worlds.  I'm guessing nobody likes that idea, though
:)

Re: A Proposal to Move to Git

[asmwarrior]

On 2013-8-27 18:00, Steinar Bang wrote:
>>>>>> NightStrike <nightstrike@gmail.com>:
> 
>> ... Git users can easily use git-svn, which means you can satisfy
>> mostly everyone, and still be using current, well maintained software.
> 
> git-svn is broken on the Win32 platform.  It can get you a git checkout
> with the full history.  The problems start when you wish to push your
> changes back into subversion.  Then you have to either do a merge or a
> rebase against the branch tracking the remote subversion repository, and
> you end up getting a conflict in all files you have changes in.
> 
> When I tried it, the conflicts with merge, were the entire files (which
> is pretty useless. With rebase and with merge using the
> ignore-space-change option, the conflicts were just my changed lines,
> but with CRLF/LF issues.
> 
> Ie. I got a conflict on all of my changes whether they were a conflict
> or not.
> 
> I tried switching off core.autocrlf in git and I tried convincing
> git-svn that the svn checkout shouldn't do CRLF/LF translation, but I
> had no luck with that.
> 
> My experiments were in msysgit 1.8.3-preview20130601.
> 
> (For the record: if someone have a procedure that can make this a
> non-issue I would be happy to be proved wrong)
> 
I'm using the same msysgit and TortoiseGit, I can use git-svn without such conflict issue.
I can successfully commit changes to Codeblocks' SVN repo
http://sourceforge.net/projects/codeblocks/ by git-svn.
My setting of local git is: auto crlf is true, safe crlf is false.

Yuanhui Zhang

Re: A Proposal to Move to Git

[Matt Rice]

On Fri, Aug 23, 2013 at 8:40 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> I assume git repo has both binutils and gdb. Is that possible to
> only checkout binutils portion?
>

you can do this with sparse checkouts, but note that it is something
you will have to do on the client side, as far as I know there is no
way to configure this from the server side/on cloning, and your .git
dir will contain history for both..

(sorry H.J. if you receive this twice)

Re: A Proposal to Move to Git

[H.J. Lu]

On Fri, Aug 23, 2013 at 9:24 AM, Matt Rice <ratmice@gmail.com> wrote:
> On Fri, Aug 23, 2013 at 8:40 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
>> I assume git repo has both binutils and gdb. Is that possible to
>> only checkout binutils portion?
>>
>
> you can do this with sparse checkouts, but note that it is something
> you will have to do on the client side, as far as I know there is no
> way to configure this from the server side/on cloning, and your .git
> dir will contain history for both..
>

Does sparse checkout work for binuitls/gdb git repo?
Has anyone tried?


-- 
H.J.

Re: A Proposal to Move to Git

[Matt Rice]

On Fri, Aug 23, 2013 at 9:37 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Fri, Aug 23, 2013 at 9:24 AM, Matt Rice <ratmice@gmail.com> wrote:
>> On Fri, Aug 23, 2013 at 8:40 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
>>> I assume git repo has both binutils and gdb. Is that possible to
>>> only checkout binutils portion?
>>>
>>
>> you can do this with sparse checkouts, but note that it is something
>> you will have to do on the client side, as far as I know there is no
>> way to configure this from the server side/on cloning, and your .git
>> dir will contain history for both..
>>
>
> Does sparse checkout work for binuitls/gdb git repo?
> Has anyone tried?

It did the last time that I tried, and was even useful for doing an
uberbaum tree using the gcc git mirror, this may make maintaining the
files existing in both repositories easier, but since it required a
different conversion of the repositories than the current mirrors, I
haven't done so recently.

Re: A Proposal to Move to Git

[H.J. Lu]

On Fri, Aug 23, 2013 at 9:47 AM, Matt Rice <ratmice@gmail.com> wrote:
> On Fri, Aug 23, 2013 at 9:37 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
>> On Fri, Aug 23, 2013 at 9:24 AM, Matt Rice <ratmice@gmail.com> wrote:
>>> On Fri, Aug 23, 2013 at 8:40 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
>>>> I assume git repo has both binutils and gdb. Is that possible to
>>>> only checkout binutils portion?
>>>>
>>>
>>> you can do this with sparse checkouts, but note that it is something
>>> you will have to do on the client side, as far as I know there is no
>>> way to configure this from the server side/on cloning, and your .git
>>> dir will contain history for both..
>>>
>>
>> Does sparse checkout work for binuitls/gdb git repo?
>> Has anyone tried?
>
> It did the last time that I tried, and was even useful for doing an
> uberbaum tree using the gcc git mirror, this may make maintaining the
> files existing in both repositories easier, but since it required a
> different conversion of the repositories than the current mirrors, I
> haven't done so recently.

We need to provide scripts to enable sparse checkout for
binutils and gdb.

-- 
H.J.

Re: A Proposal to Move to Git

[Doug Evans]

On Fri, Aug 23, 2013 at 10:51 AM, Steinar Bang <sb@dod.no> wrote:
>>>>>> "H.J. Lu" <hjl.tools@gmail.com>:
>
>> I assume git repo has both binutils and gdb. Is that possible to only
>> checkout binutils portion?
>
> Not as such.  There is something similar called sparse checkouts.
> Eg. if your source is organized like
>  binutils/
>  gdb/
> you can opt to checkout only
>  binutils/
> in your working directory.
>
> But it still will be organized as a binutils/ subdirectory at the top
> level.
>
> Ie. if you clone your binutils-only version of gdb+binutils like so
>  git clone git://somegitrepo.org/gdbbinutils.git binutils
> you will have to do
>  cd binutils/binutils
> to get to he interesting stuff...
>
> Also the "user interface" for sparse checkouts kinda sucks:
>  - You set a git property
>  - You add lines to a text file saying what should be checked out
>  - You either do an initial checkout, or you use the little used command
>    git-read-tree to update the working directory
>
> After this, git can be used normally (merge, add, commit, push).
>
> If disk usage is an issue, I guess it could be useful...?  You would
> still have the entire git repo, though.
>

Not that this is necessarily one of HJ's issues, but it's too bad
"binutils" is both the name of the package and a directory within it.
Otherwise I can imagine all-binutils, install-binutils, check-binutils
in the top level makefile doing the right thing (e.g. being wrappers
for all-gas+all-ld+..., etc.).

Re: A Proposal to Move to Git

[Alan Modra]

On Thu, Aug 22, 2013 at 10:10:47PM +0200, Mark Kettenis wrote:
> > From: Tom Tromey <tromey@redhat.com>
> > CC: Binutils Development <binutils@sourceware.org>
> > Date: Tue, 20 Aug 2013 15:12:41 -0600
> > 
> > I'd like to move gdb and binutils from CVS to Git.  I've done much of
> > the preliminary work and I will do the remainder, including the
> > inevitable follow-up bug-fixing.
> 
> I still *hate* git.

I'm an old dinosaur too. :)  I've only just started really using git,
but I'm getting to like it..

> Except for one tiny question.  How much free space do I need to clone

A lot less than you need to make CVS reasonable.  As a maintainer I
find I often need to dig through old patches.  With CVS that means
you need a local copy of the whole repository, without which
operations like cvs -D"some date" -D"some date + 2 minutes" take
forever.

-- 
Alan Modra
Australia Development Lab, IBM

Re: A Proposal to Move to Git

[Hans-Peter Nilsson]

First and foremost, thanks for setting this in motion.

On Tue, 20 Aug 2013, Tom Tromey wrote:
> The basics of the plan are as outlined by Joseph Myers:
>
>     http://gcc.gnu.org/ml/gcc/2011-03/msg00486.html
>
> For the purposes of this discussion I think you can focus on 6.b -- a
> shared gdb+binutils repository.
>
> The reason for a shared repository is simply that binutils and gdb
> share a substantial amount of code, mainly BFD, but other things as
> well.
>
> This gives the change minimal impact.  It is not zero impact, but:
>
> 1. It is superior for all of us to build the whole tree, to avoid
>    those (rare) occasions where BFD changes break other parts of the
>    build;

Sorry, but I have to react on that.  I waited a bit, but I
really must say it: you say "all of us" like a the majority, but
I doubt that there are more of "all of you" than "all of us"
working on trees checked out as separate projects.  Also, how is
that a change worth mentioning?  *Your* work-flow isn't changed,
as you already check out gdb+binutils.  I can accept the
inconvenience of the changed default thing that happens for "all
of us" remaining, and having to adjust my work-flow somewhat,
but I refuse to smile and call it good.  For example, we'll now
each by default (when just going "make" after configure, not
"make all-foo all-bar all-baz" for the bits in our project) be
exposed to breakages in "the other" project, breakages unrelated
to "our own" project.  Ok, now that I've got that out, over to
some more constructive comments.

The work-flow changes for those of us that *want* to still build
with separate trees (like, with auto-testers) should be written
down and supported, not just implied and left on our own.  I see
changes to the src-release scripts, so let's make that the
number one supported way of generating a separate tree: through
intermediate release-like tar-balls.  (No, not git sparse
checkouts.  Too many local steps that *will change as projects
add files*, and also needing something like git-1.7.10 to be
bug-free.)  But, lots of modules are unnamed in src-release, for
example the "sim" module.  Let's add those in CVSROOT/modules
that are missing from src-release but are still maintained in
the src tree (e.g. excluding dejagnu) as part of a migrated
project.  I see a couple of strange ones in the src-release
script that are not in CVSROOT/modules (like separate gas and
gas+binutils - without ld or gprof), so a pristine-ness-argument
can't be used as a counter-argument, please, thank you. :)

At first I thought there was a huge inconvenience to projects
left in the cold with CVS, but it it's less huge, it seems they
will still be able to continue as they were - *except* they
won't be able to update the shared files like configure.ac.
That's a bit more than inconvenient.  Thus, there should be a
migration path offered as part of this effort and it's not
someone elses issue.  I guess someone will dislike the following
suggestion, but how about separate per-directory repos, using
git submodule, with steps explaining how to add them to your
local "src" tree?  Ok, now that you read and disliked that, how
about the obvious alternative; offering to add them later to the
binutils+gdb git?  Then I think it would be preferred to *not*
name the git repo "the gdb+binutils repo", but (ehhh...) the src
repo!

To summarize the actions I want:
- Git migrations and work-flow offered for remaining projects
(like newlib and cgen).
- (related:) Do not set in stone this git repo as "binutils+gdb"
in any documentation changes and names.
- Add remaining "partial checkout" methods as mentioned in
modules to src-release, for related migrating projects (well, at
least "sim").  Mention this as a (the only?) supported way to
generate the project-specific tree.

I'll add them to PR14768 and hope to make the src-release
changes unless beaten to it.

> I'd like to do the final switch around mid-September.  Not sooner,
> because I am going to be away for a little while near the end of
> August, and I want to be available to fix problems.

Very much appreciated.

brgds, H-P

Re: A Proposal to Move to Git

[Joseph S. Myers]

On Fri, 30 Aug 2013, Hans-Peter Nilsson wrote:

> Sorry, but I have to react on that.  I waited a bit, but I
> really must say it: you say "all of us" like a the majority, but
> I doubt that there are more of "all of you" than "all of us"
> working on trees checked out as separate projects.  Also, how is
> that a change worth mentioning?  *Your* work-flow isn't changed,
> as you already check out gdb+binutils.  I can accept the
> inconvenience of the changed default thing that happens for "all
> of us" remaining, and having to adjust my work-flow somewhat,
> but I refuse to smile and call it good.  For example, we'll now
> each by default (when just going "make" after configure, not
> "make all-foo all-bar all-baz" for the bits in our project) be
> exposed to breakages in "the other" project, breakages unrelated
> to "our own" project.  Ok, now that I've got that out, over to
> some more constructive comments.

To build just one project, use --disable-<directory> options (this is 
already generically supported at toplevel).  It would be useful for there 
to be documentation listing sets of such options to use for building 
particular subsets of projects.

> At first I thought there was a huge inconvenience to projects
> left in the cold with CVS, but it it's less huge, it seems they
> will still be able to continue as they were - *except* they
> won't be able to update the shared files like configure.ac.

Rather, changing shared files now needs three repositories rather than two 
to change.  If we can get agreement on GCC as the master, this could be 
automated as it is for libiberty (check into GCC and let the automatic 
process merge to the others).  (Actually, my analysis suggests a master 
toplevel.git repository from which merges to all the others happen.  But 
that's not required and can always be added later.)

> local "src" tree?  Ok, now that you read and disliked that, how
> about the obvious alternative; offering to add them later to the
> binutils+gdb git?  Then I think it would be preferred to *not*
> name the git repo "the gdb+binutils repo", but (ehhh...) the src
> repo!

No, I think it should be binutils+gdb and other projects should not be 
added to it.  Most of the problems with the combined repository have been 
because it combines logically unrelated projects.  A key goal as I see it 
is that checkouts, updates, branching, tagging etc. work in the ordinary 
way for the version control system being used, without any further special 
steps or oddities - the existing use of CVS modules fails that ("cvs 
update -d" checks out things that aren't wanted in your partial checkouts) 
and so do things such as git submodules - which indicates avoiding such 
things in git arrangements.  Given how BFD is an integral part of both 
binutils and GDB, a binutils+gdb repository seems to be the smallest 
convenient unit.

> To summarize the actions I want:
> - Git migrations and work-flow offered for remaining projects
> (like newlib and cgen).

By design, it's for each project to choose to migrate or not, to its own 
repository, independently, including choosing whether git is the version 
control system they want at all.

As I noted in <http://gcc.gnu.org/ml/gcc/2011-03/msg00486.html>, cgen 
doesn't appear to use the shared toplevel at all, so if it moves out of 
src, it would just be the cgen directory moving without a copy of anything 
at toplevel; with regard to binutils+gdb, it's just another build tool.

If all active projects move out, we really ought then to try to sort out 
the Unix groups on sourceware (replace the "src" group by putting the 
people writing to each project in a group for that project which controls 
access to just the relevant repository - so a binutils+gdb group, a newlib 
group, etc.).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: A Proposal to Move to Git

[Hans-Peter Nilsson]

On Fri, 30 Aug 2013, Joseph S. Myers wrote:
> To build just one project, use --disable-<directory> options (this is
> already generically supported at toplevel).  It would be useful for there
> to be documentation listing sets of such options to use for building
> particular subsets of projects.

Hah, if I've ever heard of that --disable-<subdir>, I've
certainly forgot about it! :)  It's not something I expect to be
maintained and stable until actually documented as a suggested
work-flow.  Anyway, I'd prefer to generate and autotest from
src-release-generated tarballs, as that's what's released and
what needs to work anyway, for projects actually released.  A
good thing: two tests for one.  (Maybe there is already a
separate autotester actually generating and testing
release-snapshots.)

> > At first I thought there was a huge inconvenience to projects
> > left in the cold with CVS, but it it's less huge, it seems they
> > will still be able to continue as they were - *except* they
> > won't be able to update the shared files like configure.ac.
>
> Rather, changing shared files now needs three repositories rather than two
> to change.  If we can get agreement on GCC as the master, this could be
> automated as it is for libiberty (check into GCC and let the automatic
> process merge to the others).

Or two repos rather than one and agreement whether src-CVS or
the new git is the master; some files are not in GCC.

> > To summarize the actions I want:
> > - Git migrations and work-flow offered for remaining projects
> > (like newlib and cgen).
>
> By design, it's for each project to choose to migrate or not, to its own
> repository, independently, including choosing whether git is the version
> control system they want at all.

For other projects not to be severely inconvenienced (i.e. to
actually leave them free to choose) we assume here that the src
CVS repo remains updated regarding shared files.  I did not take
that for granted; it seemed that shared directories would remain
read-only, but I guess that wasn't actually intended.

> As I noted in <http://gcc.gnu.org/ml/gcc/2011-03/msg00486.html>, cgen
> doesn't appear to use the shared toplevel at all, so if it moves out of
> src, it would just be the cgen directory moving without a copy of anything
> at toplevel; with regard to binutils+gdb, it's just another build tool.

Still, the work-flow of using it will change (for sure for
cgen-generated binutils and sim files) and the new work-flow of
cgen development (both of and with) will have to be tested and
documented.  For sim, that's --enable-cgen-maint which now'll
require an argument.  It seems it'd almost work already except
it'd have to live in a subdir called "lib" below the argument
dir, which seems unnecessary.

brgds, H-P

Re: A Proposal to Move to Git

[Joseph S. Myers]

On Fri, 30 Aug 2013, Hans-Peter Nilsson wrote:

> > Rather, changing shared files now needs three repositories rather than two
> > to change.  If we can get agreement on GCC as the master, this could be
> > automated as it is for libiberty (check into GCC and let the automatic
> > process merge to the others).
> 
> Or two repos rather than one and agreement whether src-CVS or
> the new git is the master; some files are not in GCC.

Well, such agreement for those files not in GCC (we already have GCC as 
master for libiberty and libdecnumber, for example) - unless and until we 
move the non-shared parts of include/ to a separate directory, which I 
think would be better.  (See 
<https://sourceware.org/ml/binutils/2013-08/msg00207.html> for my comments 
on the division of include/, with an unresolved question about whether 
anything not shared with GCC is used by anything outside of binutils+gdb.)

> For other projects not to be severely inconvenienced (i.e. to
> actually leave them free to choose) we assume here that the src
> CVS repo remains updated regarding shared files.  I did not take
> that for granted; it seemed that shared directories would remain
> read-only, but I guess that wasn't actually intended.

I consider it necessary that shared files get updated in all the 
repositories between which they are shared (with the possibility of 
removing files when no longer needed by anything in a particular 
repository).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: A Proposal to Move to Git

[Doug Evans]

On Fri, Aug 30, 2013 at 7:05 PM, Hans-Peter Nilsson <hp@bitrange.com> wrote:
> On Fri, 30 Aug 2013, Joseph S. Myers wrote:
> [...]
>> As I noted in <http://gcc.gnu.org/ml/gcc/2011-03/msg00486.html>, cgen
>> doesn't appear to use the shared toplevel at all, so if it moves out of
>> src, it would just be the cgen directory moving without a copy of anything
>> at toplevel; with regard to binutils+gdb, it's just another build tool.
>
> Still, the work-flow of using it will change (for sure for
> cgen-generated binutils and sim files) and the new work-flow of
> cgen development (both of and with) will have to be tested and
> documented.  For sim, that's --enable-cgen-maint which now'll
> require an argument.  It seems it'd almost work already except
> it'd have to live in a subdir called "lib" below the argument
> dir, which seems unnecessary.

Hi.  I didn't understand where "lib" came from so I checked.
The location of cgen is defined in src/opcodes/configure.in
and src/sim/common/acinclude.m4.
[Note to self: IWBN to combine them.]
I'm guessing you're referring to this line, in both files:
cgendir=${cgen_maint}/lib/cgen}.
We can probably change that if it's useful.
The current default is src/cgen, for reference sake.