First, install bzr and the bzr-svn plugin. I strongly recommend using the latest release of bzr-svn, 0.4.9, as Jelmer is fixing bugs and improving the speed at an impressive rate. 0.4.9 is in Ubuntu Hardy.

Then make a shared repository for your Twisted branches:

bzr init-repo --rich-root-pack Twisted

(Forgetting --rich-root-pack is a common gotcha: the default repository format in bzr 1.3 doesn't have a feature bzr-svn needs.)

Then checkout the SVN trunk into the repository. You could do this direct from SVN, but it's much faster to use an already converted copy:

cd Twisted
bzr checkout http://www.twistedmatrix.com/users/spiv/bzr/twisted/trunk/
cd trunk
bzr switch svn://svn.twistedmatrix.com/svn/Twisted/trunk

To work on a branch already in SVN, you can just checkout or branch (depending on which workflow you prefer) it directly into your local repository. Checkouts will be more familiar to SVN users than independent branches, but if you want to make commits you'll need write access to Twisted's SVN repository.

cd ..
bzr checkout svn://svn.twistedmatrix.com/svn/Twisted/branches/feature-branch-1
cd feature-branch-1
hack hack hack...
bzr commit

(Don't worry too much about branches vs. checkouts. You can always use bzr reconfigure to change a checkout into a branch and vice versa if you change your mind later.)

To make a new branch in the SVN repository it's best to just use svn cp as before, then make a checkout/branch of that with bzr. (See bug 203368)

svn cp svn://svn.twistedmatrix.com/svn/Twisted/trunk svn://svn.twistedmatrix.com/svn/Twisted/branches/new-branch
bzr checkout svn://svn.twistedmatrix.com/svn/Twisted/branches/new-branch

If you can't commit to Twisted SVN you can still use bzr. Just make independent branches rather than checkouts. To share your branches with other people you can simply bzr push them to anywhere you like, including Launchpad.

And that's really it. Everything else is just regular bzr, including the excellent merging and offline commits.

For example, here's how you'd merge a branch to trunk and commit it:

bzr checkout svn+ssh://svn.twistedmatrix.com/svn/Twisted/trunk
cd trunk
bzr merge ../my-branch   # or you could use an SVN url!
bzr commit

If you think this is right, "think about two different patches each adding a new keyword and also changing the line ``#define NUM_OF_KEYWORDS 17'' to ``#define NUM_OF_KEYWORDS 18''." (example taken from the darcs manual, because I'm not going to come up with a better example than that).

In the example he cites, there will most likely be a conflict, assuming the added keywords are different (and depending on what the source changes to add keywords look like, but the obvious schemes would cause a conflict). The conflict will be at the point where the different keywords are added, rather than at the “#define NUM_OF_KEYWORDS”, but the human resolving that conflict would, if your source is clear, know that they need to check the NUM_OF_KEYWORDS value too. (Although if your source code has the property that you need to update multiple places to register one new thing, it's violating “Don't Repeat Yourself” principle, but that's another discussion...)

But more significantly, there are plenty of ways that two patches that don't conflict textually can conflict semantically. One patch might change the signature of a function, and another may add a new caller of that function that assumes it has the old signature. And there's lots of more subtle ways that they could conflict.

If you want to ensure your code is correct in the face of changes, then the tool for the job isn't a Version Control System. It's a Test Suite, preferably an automated one. Version Control Systems help you collaborate and manage changes, but it's your Test Suite that tells you if the code still works.

“It's sort of like you're a combination architect / builder, but you're a really terrible one. Someone asks you to build a new bathroom for their house, so you do, but despite all your careful planning the finished product is just shite. For example, the shower wall is made out of soap, because you thought it would be convenient — now nobody needs to buy soap, because they can just rub up and down against the wall for a bit and then hose themselves off (you installed a hose instead of a shower head for greater flexibility).”

It's worth reading the full piece.

(I spoke about this a little at my OSDC talk last year, but I recently remembered that I ought to blog about it to get a wider audience.)

The benefits are pretty obvious: it saves people time. Bleeding edge users/testers can always be confident that a checkout of your project will build and work. Developers can start new feature branches confident that nothing is seriously broken before they've even started. Test coverage hopefully is improving over time, so you can be fairly sure that the quality of the trunk is also improving. And so on.

So how hard is it? Many projects have a policy that requires developers to run tests before checking in, and if it turns out they forgot and broke the test suite, their commit will be reverted. Well, it'll be reverted as soon as someone notices. And hopefully no-one checked in while you were running tests, which might easily happen if you have slow test suite.

Here's what the Launchpad and Bazaar developers do: use PQM (Patch Queue Manager) to guard commits to the trunk. No developer has direct write access to the trunk branch, only the PQM instance. To merge a branch to trunk, developers submit a merge request to PQM. We configure PQM to run “make check” before committing a merge, and only if the tests pass (and the merge had no conflicts) will the change be committed.

You could configure your revision control system to use a pre-commit hook, but that doesn't tend to work very well unless the checks are instantaneous. It's nice to finish your work, send the merge request, and then be able to disconnect your laptop from the network and hop on a train without interrupting the processing of your branch. PQM accepts requests via GPG-signed email, and also sends the results via email (although in principle it could just as easily use something like XML-RPC). PQM will simply queue requests if they arrive while it is busy with an existing request. It's an asynchronous workflow, which works well when the checks can take some time.

If you have a bunch of inter-dependent projects, you can even configure PQM to run a command to check all of them every time someone tries to commit to any one of them. We do this with Launchpad, so we know that upgrading a dependency of the main Launchpad project won't cause problems in Launchpad itself.

It's not a perfect process, and PQM isn't a perfect tool. PQM in some ways is still quite simple and has rough edges, but it's worked very well for us. Our tests always pass on trunk. Occasionally there's a crisis like tests that normally pass but intermittently fail, but these would still be a problem without PQM, assuming you really do care about making sure tests always pass on your trunk.

This process can be complementary with the post-commit testing that many projects already do with BuildBot or Tinderbox. You could have PQM run tests on one or two key platforms, and then after the commit is done let BuildBot run tests on every other platform the community has buildslaves for. For instance, you may have a buildslave that tries to run your latest code with the latest development version of Python: failures there are worth knowing about, but not necessarily a reason to stop a branch from landing.

There's already a Bazaar plugin for sending requests to PQM, but you could pretty easily write plugins or scripts for other systems too.

Jonathan Corbet's Kernel Report was fairly interesting. I think I would have found it much more interesting if I hadn't been regularly reading LWN's kernel page every week, but even though I'd read about most of it before it was an enjoyable overview.

Dave Airlie's Nouveau talk was interesting, it was a project I'd heard of but actually knew very little about. In brief: they're doing some very clever things to reverse engineer the driver, but don't expect it to be ready any time soon (alpha quality by the end of 2007 is the current plan).

I actually got a chance to ride around on Geoffrey Bennett's Open Source Segway™ yesterday, but was still enjoyed his talk about it today. I'm not really an electronics geek, but the fact that he could monitor the readings from the components with a cute GTK app (via bluetooth) was very nifty. Even more nifty was that he was casually tuning the settings of the controller on the fly from the same simple GTK app. His description of the development and testing process for it was quite similar to the way software that is built from scratch is developed and tested, although I've never had to worry about my software throwing me off and then running me over if I screw it up.

I'm glad the organisers programmed a break between each session, so that people have time to move to a different room for the next talk, and perhaps strecth their legs. OSDC would have benefitted from this. I just wish that the organisers were stricter about making sure speakers actually finish on time so that the breaks don't evaporate.