Skip to main content

Git-SVN Mirror without the annoying update-ref

This post is part of a series on Git and Subversion. To see all the related posts, screencasts and other resources, please click here

So no sooner than I had done my git-svn presentation at JavaZone, I got word of a slightly different Git-SVN mirror setup that makes it a bit easier to work with:

In short, my old recipe includes an annoying git update-ref step to keep the git-svn remote reference up to date with the central bare git repo. This new recipe avoids this, so we can simply use git svn dcommit  directly.

So, longer version, with the details. My original recipe is laid out in five steps:

  1. Clone a fresh Git repo from Subversion. This will be our fetching repo.
  2. Set up a bare repo.
  3. Configure pushing from the fetching repo to bare repo
  4. In the shoes of a developer, clone the repo
  5. Set up an SVN remote in the developer's repo
In the new approach, we redefine those last two steps:

(See the original post for how to do the first three.)

4. Clone the bare repo

Note the -o parameter that specifies that we want to call the origin remote "mirror":

tfnico@flint:~/sources/git/>git clone -o mirror ~/git-repos/websites.git/

We can now track the remote branches, and pull the latest changes as they come in to the bare repo.

tfnico@flint:~/sources/git/websites/>git checkout -t mirror/yksi
tfnico@flint:~/sources/git/websites/>git pull --rebase

(Remember to use --rebase to avoid merge commits in case you have local commits.)

When the time comes to push commits back to Subversion, we have to complete the last step.

5. Set up an SVN remote in the developer's repo

We'll do an svn-init with the same parameters as we cloned in the beginning, adding a prefix that matches the name of the remote "mirror":

tfnico@flint:~/sources/git/websites/>git svn init --prefix=mirror/ -s file:///svn-repos/company-repo/websites/

The point of the mirror/ prefix is that whenever we do a pull, the git-svn reference will be updated along with the normal remote reference. So now you can do git svn dcommit without any further ado.


  1. Great, I came to a similar conclusions earlier - but haven't seen it written down and forgot it in the meantime as well (so kept using update-ref to keep the two pointers refs/remotes/trunk refs/svn/trunk refs/remotes/mirror/svn/trunk in sync).

    Have you run ran into any negative side-effects/gotchas with this --prefix setup since?

  2. Hi Gin, thanks for your comment!

    Generally, no, I haven't seen any real problems with the prefix setup.

    There is one tiny detail: If you accidentally push directly to the bare bare repo (instead of doing a dcommit), and you need to revert that by doing a forced push to remote/svn, you don't have any local reference any more to which commit the SVN HEAD is at. However, this is not really a problem if you are (a) careful, or (b) take measures to make the bare repo non-writable for developers.

  3. Good point, thanks Thomas. I'd probably want to ensure that trouble doesn't strike (most our devs are new to git and might do mistakes). However, I'd prefer not making the whole mirror repo read-only (some of as are pushing pure-git branches there as well..), so will take a look at the gitolite setup: my impression is that it may be able to protect specific branches.

  4. Gin: Perhaps a more practical way to avoid accidents is to have each developer disable the push configuration in their local repositories:

  5. Here's a nice one-liner for the above setup:

    git config remote.mirror.pushurl WARNING_USE_SVN_DCOMMIT_INSTEAD_OF_PUSH

    This will cause developers to see an error message when trying to push to the bare repo (mirror).

  6. Good idea - thanks for that too!
    I'd still prefer to do it once (on the server), rather than setting it up on the growing numbers of developer boxes. I guess there is not easy way to share repo configuration (apart from nasty tricks to commit them too) - I'd be happy to be wrong one this.

    Also, do you think it's easy and fast to migrate to this -synced-prefix setup? (the repo is big)
    I guess I could do a 'git remote rename oldmirror svn' , where 'svn' is default; but I'd prefer that one to 'mirror' instead.. can't see anything obvious on man git-svn. Any ideas? thanks again.

  7. Hmm, I think that git-svn needs to be re-initialized (with the --prefix parameter), and to avoid any mess, I've done this in clean clones (so far).

    However, this all takes place on the developer's machines. There's no need to change anything with the fetching/bare repos. So the developers can clone their old repo locally (fastest), add the proper mirror remote, and do git svn init with the --prefix=mirror/.

    For me most of the work was rebuilding the IDE configuration files that were scattered around in the repository work tree.

  8. It would be great if a simple 'git svn init' worked. I was just concerned about the metadata (.git/svn/{.metadata,refs}) and stuff that might take a lot of time to rebuild. In contrast, the pure git-counterpart 'git remote rename' feels fast and safe.

    If a simple 'git-svn init --prefix..' followed by a 'git svn rebase' will rebuild the metadata quickly and safely, that's great. Why shall I use "clean clones"? What kind of mess are referring to: e.g old stuff in .git/svn dir, or anything else?

  9. @Gin: To be honest, I've never tried svn init twice in the same repository. It could be that it works fine, and the old configuration is properly overwritten. Maybe you have to remove the old svn-remote first, and/or the .git/svn folder. Try it out, and let me know. (but if you're worried about corrupting your working git-svn clone, make a copy of it first).

  10. I tried this method a few times today and kept getting snagged when cloning the bare repo. Kept getting "warning: remote HEAD refers to nonexistent ref, unable to checkout."

    Inside the bare repo I found that HEAD pointed to refs/heads/master but the master branch did not exist. So I edited HEAD to use refs/heads/trunk and was able to proceed. Everything is looking fine so far.

    This was on MacOSX 10.6.8 with git

  11. After a little more reading and playing around it looks like I could just as well done the following to fix up the bare repo, to create the master branch and use trunk to set the start-point.

    $git branch master trunk

  12. Hi James, thanks for commenting!

    Yes, after creating the bare repo, the "default branch" has to be set to be trunk like this:

    git symbolic-ref HEAD refs/heads/trunk

    I'm not really sure why I didn't include that step in this guide. I think I did do it in my JavaZone presentation, so I must have discovered that after I wrote the original guide. I'll see if I get a chance to work that in. Maybe write a completely overhauled guide some day :)

  13. Anyone following along here may find this snippet useful:

    test $(git svn find-rev $(git log --all --oneline -n 1 |awk '{print $1}')) -ne $(svn info $(git config --local svn-remote.svn.url) |awk '/^Revision:/ {print $2}')

    I am using it with the Jenkins ScriptTrigger plugin to poll for when the svn Revision no longer matches the latest in the fetching repo, in order to automate the mirroring.

    Of course, if you have admin access to the svn repo you should probably just set up a commit hook there.

    FYI, the output looks like this:
    +++ git log --all --oneline -n 1
    +++ awk '{print $1}'
    ++ git svn find-rev 90fcd3c
    ++ awk '/^Revision:/ {print $2}'
    +++ git config --local svn-remote.svn.url
    ++ svn info file:///Users/james/src/svn-test
    + test 77 -ne 77

  14. Tanks for the tip, James! If I ever get around to doing the big re-write of the big git-svn guide, I'll include this.

  15. James Stansell21/1/12 16:15

    I'm glad you like it! Feel free to suggest improvements. In particular on MS/Win it would need cygwin, whereas rewriting in groovy could make it easier to use there.

    Thanks again for the very helpful git-svn guide!

  16. James Stansell25/1/12 09:31

    Just an update about my jenkins job to fetch from svn and push to the bare repo. The snippet above was interesting but I think duplicated (poorly) what git svn fetch already does. So now I have moved the git svn fetch into the trigger script.

    cd /Users/james/src/git-svn/git-test-fetch
    fetchinfo="$(git svn fetch)"
    [[ $? -ne 0 || ${#fetchinfo} -eq 0 ]] && exit 1 #the fetch either failed or found no change
    read d <<< $(echo "$fetchinfo" |awk '/^r/ {print $1}') #description
    cat <<- _EOD
    < cause$z>< pre>
    $(git svn log --oneline --show-commit --remotes -${d// *r/:})

    < /pre>< /cause>
    < description$z>$d< /description>

  17. @tfnico, a few items you might want to consider for the big re-write of the big git-svn guide:

    1. Use a local svn repository if possible. Yes, svnsync is your friend. Splitting up the network transfer and the svn->git migration into two separate steps will greatly speed up the entire process, even for medium-size projects.

    2. Use an authors.txt file. Your git log will thank you!

    3. When you don't have admin access to the svn server, jenkins can be used to poll for changes, as I wrote above.

    Also, some related projects which are currently active:

    * reposurgeon:

    * svn-dump-fast-export:


Post a Comment

Popular posts from this blog

Open source CMS evaluations

I have now seen three more or less serious open source CMS reviews. First guy to hit the field was Matt Raible ( 1 2 3 4 ), ending up with Drupal , Joomla , Magnolia , OpenCms and MeshCMS being runner-ups. Then there is OpenAdvantage that tries out a handful ( Drupal , Exponent CMS , Lenya , Mambo , and Silva ), including Plone which they use for their own site (funny/annoying that the entire site has no RSS-feeds, nor is it possible to comment on the articles), following Matt's approach by exluding many CMS that seem not to fit the criteria. It is somewhat strange that OpenAdvantage cuts away Magnolia because it "Requires J2EE server; difficult to install and configure; more of a framework than CMS", and proceed to include Apache Lenya in the full evaluation. Magnolia does not require a J2EE server. It runs on Tomcat just like Lenya does (maybe it's an idea to bundle Magnolia with Jetty to make it seem more lightweight). I'm still sure that OpenAdvant

Managing dot-files with vcsh and myrepos

Say I want to get my dot-files out on a new computer. Here's what I do: # install vcsh & myrepos via apt/brew/etc vcsh clone mr mr update Done! All dot-files are ready to use and in place. No deploy command, no linking up symlinks to the files . No checking/out in my entire home directory as a Git repository. Yet, all my dot-files are neatly kept in fine-grained repositories, and any changes I make are immediately ready to be committed: config-atom.git     -> ~/.atom/* config-mr.git     -> ~/.mrconfig     -> ~/.config/mr/* config-tmuxinator.git       -> ~/.tmuxinator/* config-vim.git     -> ~/.vimrc     -> ~/.vim/* config-bin.git        -> ~/bin/* config-git.git               -> ~/.gitconfig config-tmux.git       -> ~/.tmux.conf     config-zsh.git     -> ~/.zshrc How can this be? The key here is to use vcsh to keep track of your dot-files, and its partner myrepos/mr for o

Leaving eyeo

Thirteen blog posts later, this one notes my departure from eyeo after 4 years and 3 months. I joined eyeo around the headcount of 80 employees, and now I think there's just over 250 people there. My role coming in was as operations manager, doing a mix of infrastructure engineering and technical project management. I later on took on organizational development to help the company deal with its growing pains . We introduced cross-functional teams, departments (kind of like guilds), new leadership structures, goal-setting frameworks, onboarding processes and career frameworks.  And all of this in a rapidly growing distributed company. I'm proud and happy that for a long time I knew every employee by name and got to meet every single new-hire through training them on company structure and processes.  At some point, we had enough experienced leaders and organizational developers that I could zoom back in on working in one team, consulting them on  Git and continuous integration

Using Voice-Chat for Gamers in Distributed Teams

This is a post going into the usefulness of live voice-chat tools in distributed teams. If you've ever seen the Leeeeeroooooyy Jeeeenkiiins video of World of Warcraft fame, you've heard this kind of tool in action. It's how the participants in the video are speaking with each other - this is not a feature built into the World of Warcraft game - it's a separate team-oriented VoIP software, and it's all about letting gamers communicate orally while gaming.  Since these tools are for gamers, they have to be fast (low latency) light (as not to steal CPU-cycles from heavy games graphics)  moderate in bandwidth usage (as not to affect the game server connection) There are several options around: TeamSpeak , Ventrilo , more recently the massively grown Discord , and finally Mumble , which is the open-source alternative of the gang. A few years ago, when I joined eyeo (a distributed company), several of the operations team were avid gamers, and had a TeamSp

Joining eyeo: A Year in Review

It's been well over a year since I  joined eyeo . And 'tis the season for yearly reviews, so... It's been pretty wild. So many times I thought "this stuff really deserves a bloggin", but then it was too inviting to grab onto the next thing and get that rolling. Instead of taking a deep dive into some topic already, I want to scan through that year in review and think for myself, what were the big things, the important things, the things I achieved, and the things I learned. And then later on, if I ever get around to it, grab one of these topics and elaborate in a dedicated blog-post. Like a bucket-list of the blog posts that I should have written. Here goes: How given no other structures, silos will grow by themselves This was my initial shock after joining the company. Only a few years after taking off as a startup, the hedges began growing, seemingly almost by themselves, and against the will of the founders. I've worked in silos, and in companies wit