Monday, August 02, 2010

Syncing your Git repo with the Subversion repo

This post is part of a series on Git and Subversion. To see all the related posts, screencasts and other resources, please click here

This is a follow-up to the previous post on how to live with Git and Suversion in parallel. It's really trivial stuff for any experienced user, but was worth noting down somewhere for my own sake.

I've cloned a project called "fudge" from Subversion:

>git svn clone https://scm.mycompany.com/svn/fudge

So my local repo has the correct svn-remote configuration and all that to go with it inside fudge/.git/svn. This is done automatically when you svn-clone.

The bad thing is that all my Git-mates at work have to wait for me doing svn-rebase before they can pull the latest code from Subversion from my repo into theirs.

I want to get rid of this responsibility. I'll put something similar to my svn-rebasing repository on a server, have it run svn-rebase regularly, and push the changes to a centralized git repository, that everyone can pull from whenever they want. Here's how:

First, I'll init a repo on the build server called "builder". It has a designated place for the git repos in the directory /scm/git/

>cd /scm/git
>git init --bare fudge.git

The --bare means a git repo with no local checkout, i.e. just the compressed stuff inside .git.


Back on my own machine, I register the above repo as a remote for my own. I'll call it origin (this may sound confusing as my local repo is actually the origin here, but it's kind of a git convention to name the centralized repo like that):

>git remote add origin ssh://tfnico@builder/scm/git/fudge.git

The above should add the following to my fudge/.git/config:

[remote "origin"]
fetch = +refs/heads/*:refs/remotes/origin/*
url = ssh://tfnico@builder/git/java/ips.git

Now, I can push my local repository to the build server. I have to specify which branch in the remote repo I'll push to (master oughta work):

>git push origin master

So, centralized repo is ready. Now, we want a second repository on builder that can do the svn-rebase action. You could probably do this in the same repo as the first one, but I think it's cleaner to do it separately. Besides, in order for svn-rebase to work, you need to have a repo with a local checkout.

We'll create this repo just like any other developer would: By cloning from the central repo (still on builder). I'll pretend there's a user on builder called "worker" designated for this kind of action:

cd /home/worker/projects
git clone /scm/git/fudge

If we try just running svn-rebase right away, something like this might happen:

>cd fudge
>git svn rebase
Migrating from a git-svn v1 layout...
Data from a previous version of git-svn exists, but
.git/svn
(required for this version (1.7.1) of git-svn) does not exist.
Done migrating from a git-svn v1 layout
Unable to determine upstream SVN information from working tree history

You'll see that Git created the .git/svn/refs/remotes/git-svn/  folder on its own, but there's not much in there. We have to make the repo aware of the remote Subversion repo in two steps:

First, copy the svn-remote config from your original repo (the one you svn cloned) .git/config into the worker's config:

[svn-remote "svn"]
url = https://scm.mycompany.com/svn/fudge
fetch = :refs/remotes/git-svn


While imagination would allow us to think that git-svn now could figure out the rest by itself, we still need to add a pointer to the git-svn remote. Look inside the original repo's references, and you should find a file .git/refs/remotes/git-svn with a SHA in it. You have to copy this file into the worker's repo as well (or just paste the SHA into a new file with that name).

Now, try svn-rebase again:

>git svn rebase
Rebuilding .git/svn/refs/remotes/git-svn/.rev_map.901b3fc1-1df5-aa1e-233f-0cced8b7b346 ...
r123 = 38aada9c84fdb74f2271706c2a4be768334b0cc1
....
Done rebuilding .git/svn/refs/remotes/git-svn/.rev_map.901b3fc1-1df5-aa1e-233f-0cced8b7b346
Current branch master is up to date.

That's it! So, let's recap what we had to do:
  1. Create a central repo (bare)
  2. Clone the repo 
  3. Add the svn-remote to .git/config
  4. Add a pointer/SHA in .git/refs/remotes/git-svn
Now, we can set up a process that regularly does svn rebase and svn push to have it continuously update the central repo with the latest changes from Subversion. You can do it in your CI-server, as a cron-job, or maybe trigger it with a Subversion post-commit hook.

Note: When you go back to work and want to push commits back to the Subversion repo, you have to use git svn dcommit. Pushing your changes to a central git repo and then having some automated job dcommitting back is a bad idea, I think. Please let me know if you any thoughts on what workflow to use, especially when multiple people are working with Git/Subversion repos.