Managing long patch series

Managing long patch series

[Jim Blandy]

Ulrich, I'm curious what techniques you use to manage these long
strings of patches.  Specifically, I was wondering:

- I'm usually working from a fully-patched tree, and then breaking it
  up into digestible pieces for submission.  If you are working this
  way as well, do you have a nice way to ensure the decomposed patch
  series remains equivalent to your fully-patched tree?

- Often I find I need to revise an earlier patch in the series, but
  that chance may affect later patches.  Do you have a nice way to
  handle this?

Or is it all just "blood, sweat, and tears"?  In the software world,
that approach usually results in "mistakes", but you and your fellow
IBM GDB hackers seem to do well.

I've tried using quilt, but if one doesn't keep very careful track of
what's going on things can get very tangled.  The Emacs mode helped
somewhat, but had other flaws, so I set it aside.

I've been tempted to try using Mercurial for this.

Re: Managing long patch series

[Daniel Jacobowitz]

On Fri, Oct 26, 2007 at 11:51:02AM -0700, Jim Blandy wrote:
> I've tried using quilt, but if one doesn't keep very careful track of
> what's going on things can get very tangled.  The Emacs mode helped
> somewhat, but had other flaws, so I set it aside.

I usually do it with quilt plus the emacs mode now.  What flaws?  The
only one I've noticed as a real inconvenience is trouble adding new
files, but a talented Lisp hacker could probably fix it in moments.

It's still mildly error prone, but popping everything and doing cvs
diff is good at turning up anything I've missed.

-- 
Daniel Jacobowitz
CodeSourcery

Re: Managing long patch series

[Jim Blandy]

Daniel Jacobowitz <drow at false.org> writes:
> On Fri, Oct 26, 2007 at 11:51:02AM -0700, Jim Blandy wrote:
>> I've tried using quilt, but if one doesn't keep very careful track of
>> what's going on things can get very tangled.  The Emacs mode helped
>> somewhat, but had other flaws, so I set it aside.
>
> I usually do it with quilt plus the emacs mode now.  What flaws?  The
> only one I've noticed as a real inconvenience is trouble adding new
> files, but a talented Lisp hacker could probably fix it in moments.

Quilt (without Emacs support) looked wonderful at first, but then I
found I would get confused by changes made to files outside the patch.

quilt.el fixed that specific problem, but there were also bugs that
got under my skin.  I did look at the lisp, and I was willing to hack
on it a bit, but it was clear the author wasn't very familiar with
Emacs Lisp, and I got worried about getting distracted from work that,
like, someone actually cared about me finishing.  :)

Perhaps I could have made it work, but it left me with the strong
impression that the problem hadn't been thought through carefully.
I'm happy to invest a solid idea that happens to have bugs, but if it
seems like the big idea isn't solid...

Re: Managing long patch series

[Vladimir Prus]

Jim Blandy wrote:

> 
> Ulrich, I'm curious what techniques you use to manage these long
> strings of patches.  Specifically, I was wondering:
> 
> - I'm usually working from a fully-patched tree, and then breaking it
>   up into digestible pieces for submission.  If you are working this
>   way as well, do you have a nice way to ensure the decomposed patch
>   series remains equivalent to your fully-patched tree?
> 
> - Often I find I need to revise an earlier patch in the series, but
>   that chance may affect later patches.  Do you have a nice way to
>   handle this?
> 
> Or is it all just "blood, sweat, and tears"?  In the software world,
> that approach usually results in "mistakes", but you and your fellow
> IBM GDB hackers seem to do well.
> 
> I've tried using quilt, but if one doesn't keep very careful track of
> what's going on things can get very tangled.  The Emacs mode helped
> somewhat, but had other flaws, so I set it aside.
> 
> I've been tempted to try using Mercurial for this.

I was using SVK for that. I have //patches/patch1/ ... //patches/patchN
then if I modify patchX I use svk smerge to update all later patches.
When mainline changes, I smerge from mainline mirrors to patch1 and then
smerge between (1->2, 2->3,  N-1->N).
The only gotcha I found is that you should always apply mainline changes
to the first patch and then propagate it via the chain. If you try
to apply mainline changes to a patch in the middle, SVK mergeinfo
representation may fall appart, and given you random conflicts in
fugure.

-  Volodya

Re: Managing long patch series

[Jim Blandy]

Vladimir Prus <ghost at cs.msu.su> writes:
> I was using SVK for that. I have //patches/patch1/ ... //patches/patchN
> then if I modify patchX I use svk smerge to update all later patches.
> When mainline changes, I smerge from mainline mirrors to patch1 and then
> smerge between (1->2, 2->3,  N-1->N).
> The only gotcha I found is that you should always apply mainline changes
> to the first patch and then propagate it via the chain. If you try
> to apply mainline changes to a patch in the middle, SVK mergeinfo
> representation may fall appart, and given you random conflicts in
> fugure.

Okay, that's interesting.  I think I can see how this would be done in
Mercurial, too.  I'd done something equivalent to merging into the
middle of the chain with the tracepoint patches; I guess I'll learn
soon enough whether its metadata representation disintegrates. :)

Tom Tromey referred me to Mercurial Queues, which looks really good:

          http://hgbook.red-bean.com/hgbookch12.html

The reason I'm so fond of Mercurial is partially that it's quite fast.
You can jump from one place to another easily.  I have a tree
which is simultaneously a CVS checkout and a Mercurial repository,
that I use as a gateway between the two; I say, essentially:

   cvs update
   hg addremove
   hg commit -m 'from public CVS'

from time to time, and then I can pull those changesets into whatever
other working copies I have.  My .hgignore file tells Mercurial to
ignore the 'CVS' subdirectories, and I just ignore the '?' markers
from CVS for '.hg' and '.hgignore'.  It has all worked nicely so far.

It sounds like you're doing something similar with SVK --- unless
you're pushing/pulling from CodeSourcery's internal SVK repository.

Re: Managing long patch series

[Ulrich Weigand]

Jim Blandy wrote:

> Ulrich, I'm curious what techniques you use to manage these long
> strings of patches.  Specifically, I was wondering:
> 
> - I'm usually working from a fully-patched tree, and then breaking it
>   up into digestible pieces for submission.  If you are working this
>   way as well, do you have a nice way to ensure the decomposed patch
>   series remains equivalent to your fully-patched tree?
> 
> - Often I find I need to revise an earlier patch in the series, but
>   that chance may affect later patches.  Do you have a nice way to
>   handle this?
> 
> Or is it all just "blood, sweat, and tears"?  In the software world,
> that approach usually results in "mistakes", but you and your fellow
> IBM GDB hackers seem to do well.

I'm handling this manually.  Usually, I have two trees: one vanilla
check-out at a particular stage, and one "work tree" that started
out as a copy of the vanilla tree with my changes added.

To re-base a patch set to mainline changes, or to some change in an
earlier patch in the series, I'll typically go back to making the
work tree a fresh copy of the vanilla tree, apply the first patch,
resolve any conflicts, re-diff against the vanilla tree to get a
new 'clean' version of the patch, apply the clean patch to the
vanilla tree, then apply the second patch to the work tree and so on.
(After the refresh is complete, I revert the whole series from the
"vanilla" tree again, and verify that "cvs diff" is back to empty.)

I try to break changes out into separate patches earlier rather than
later if possible, but of course on occasion I still break up a big
changeset into multiple smaller ones.  To make sure the broken-out
patches still combine back into the original on, I'd typically
compare the work tree after applying the small patch series against
the vanilla tree with the old big patch applied.

> I've tried using quilt, but if one doesn't keep very careful track of
> what's going on things can get very tangled.  The Emacs mode helped
> somewhat, but had other flaws, so I set it aside.

I understand that quilt employs a method that is similar to the
manual approach I'm currently using, so if I were to use some tool,
quilt would probably be it.  (However, I'm a vi person, so I have no
opinion either way on the Emacs quilt mode ...)

> I've been tempted to try using Mercurial for this.

I've never used that, sorry.

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand@de.ibm.com