Skip to main content

Syncing your Git repo with the Subversion repo

This post is part of a series on Git and Subversion. To see all the related posts, screencasts and other resources, please click here

This is a follow-up to the previous post on how to live with Git and Suversion in parallel. It's really trivial stuff for any experienced user, but was worth noting down somewhere for my own sake.

I've cloned a project called "fudge" from Subversion:

>git svn clone https://scm.mycompany.com/svn/fudge

So my local repo has the correct svn-remote configuration and all that to go with it inside fudge/.git/svn. This is done automatically when you svn-clone.

The bad thing is that all my Git-mates at work have to wait for me doing svn-rebase before they can pull the latest code from Subversion from my repo into theirs.

I want to get rid of this responsibility. I'll put something similar to my svn-rebasing repository on a server, have it run svn-rebase regularly, and push the changes to a centralized git repository, that everyone can pull from whenever they want. Here's how:

First, I'll init a repo on the build server called "builder". It has a designated place for the git repos in the directory /scm/git/

>cd /scm/git
>git init --bare fudge.git

The --bare means a git repo with no local checkout, i.e. just the compressed stuff inside .git.


Back on my own machine, I register the above repo as a remote for my own. I'll call it origin (this may sound confusing as my local repo is actually the origin here, but it's kind of a git convention to name the centralized repo like that):

>git remote add origin ssh://tfnico@builder/scm/git/fudge.git

The above should add the following to my fudge/.git/config:

[remote "origin"]
fetch = +refs/heads/*:refs/remotes/origin/*
url = ssh://tfnico@builder/git/java/ips.git

Now, I can push my local repository to the build server. I have to specify which branch in the remote repo I'll push to (master oughta work):

>git push origin master

So, centralized repo is ready. Now, we want a second repository on builder that can do the svn-rebase action. You could probably do this in the same repo as the first one, but I think it's cleaner to do it separately. Besides, in order for svn-rebase to work, you need to have a repo with a local checkout.

We'll create this repo just like any other developer would: By cloning from the central repo (still on builder). I'll pretend there's a user on builder called "worker" designated for this kind of action:

cd /home/worker/projects
git clone /scm/git/fudge

If we try just running svn-rebase right away, something like this might happen:

>cd fudge
>git svn rebase
Migrating from a git-svn v1 layout...
Data from a previous version of git-svn exists, but
.git/svn
(required for this version (1.7.1) of git-svn) does not exist.
Done migrating from a git-svn v1 layout
Unable to determine upstream SVN information from working tree history

You'll see that Git created the .git/svn/refs/remotes/git-svn/  folder on its own, but there's not much in there. We have to make the repo aware of the remote Subversion repo in two steps:

First, copy the svn-remote config from your original repo (the one you svn cloned) .git/config into the worker's config:

[svn-remote "svn"]
url = https://scm.mycompany.com/svn/fudge
fetch = :refs/remotes/git-svn


While imagination would allow us to think that git-svn now could figure out the rest by itself, we still need to add a pointer to the git-svn remote. Look inside the original repo's references, and you should find a file .git/refs/remotes/git-svn with a SHA in it. You have to copy this file into the worker's repo as well (or just paste the SHA into a new file with that name).

Now, try svn-rebase again:

>git svn rebase
Rebuilding .git/svn/refs/remotes/git-svn/.rev_map.901b3fc1-1df5-aa1e-233f-0cced8b7b346 ...
r123 = 38aada9c84fdb74f2271706c2a4be768334b0cc1
....
Done rebuilding .git/svn/refs/remotes/git-svn/.rev_map.901b3fc1-1df5-aa1e-233f-0cced8b7b346
Current branch master is up to date.

That's it! So, let's recap what we had to do:
  1. Create a central repo (bare)
  2. Clone the repo 
  3. Add the svn-remote to .git/config
  4. Add a pointer/SHA in .git/refs/remotes/git-svn
Now, we can set up a process that regularly does svn rebase and svn push to have it continuously update the central repo with the latest changes from Subversion. You can do it in your CI-server, as a cron-job, or maybe trigger it with a Subversion post-commit hook.

Note: When you go back to work and want to push commits back to the Subversion repo, you have to use git svn dcommit. Pushing your changes to a central git repo and then having some automated job dcommitting back is a bad idea, I think. Please let me know if you any thoughts on what workflow to use, especially when multiple people are working with Git/Subversion repos.

Comments

  1. Thanks Thomas Ferris Nicolaisen, this is what I'm looking for, but I got an error when adding a pointer/SHA in .git/refs/remotes/git-svn, bcause I couldn't find the file in my original repo, so that I always get error when rebasing. Am I miss something basic?

    ReplyDelete
  2. You have to create the pointer manually. Maybe it's a bit better explained in the next post after this one: http://blog.tfnico.com/2010/08/some-clarifications-on-living-with-git.html

    You can also send a mail to this mailing list:

    http://groups.google.com/group/git-users

    Include which command you are trying to do, and what error message you get.

    ReplyDelete
  3. After following the link above to this gitfaq url:
    https://git.wiki.kernel.org/index.php/GitFaq#How_do_I_mirror_a_SVN_repository_to_git.3F

    I think providing the --prefix=origin/ option to "git-svn init" seemed to work fine, and is easier than keeping a copy of the SHA around.

    ReplyDelete
  4. Hi! Your tutorial was really helpful for me, but I encountered a problem. When I try to git rebase, the process seems to do nothing. It seems like it's just hanging there. I have cloned the repo and instead of adding the SHA, I have done git svn init -s --prefix=origin/ svn_project. Could you help me?

    ReplyDelete
  5. Hi Andea, unfortunately it is so long since I wrote this or even used git-svn. I would recommend you look at my later writing on the subject, since this post was very early version of my setup. If you're still stuck, you better try asking on the Git user mailing list, or on StackOverflow, etc. Good luck!

    ReplyDelete

Post a Comment

Popular posts from this blog

Open source CMS evaluations

I have now seen three more or less serious open source CMS reviews. First guy to hit the field was Matt Raible ( 1 2 3 4 ), ending up with Drupal , Joomla , Magnolia , OpenCms and MeshCMS being runner-ups. Then there is OpenAdvantage that tries out a handful ( Drupal , Exponent CMS , Lenya , Mambo , and Silva ), including Plone which they use for their own site (funny/annoying that the entire site has no RSS-feeds, nor is it possible to comment on the articles), following Matt's approach by exluding many CMS that seem not to fit the criteria. It is somewhat strange that OpenAdvantage cuts away Magnolia because it "Requires J2EE server; difficult to install and configure; more of a framework than CMS", and proceed to include Apache Lenya in the full evaluation. Magnolia does not require a J2EE server. It runs on Tomcat just like Lenya does (maybe it's an idea to bundle Magnolia with Jetty to make it seem more lightweight). I'm still sure that OpenAdvant

Git Stash Blooper (Could not restore untracked files from stash)

The other day I accidentally did a git stash -a , which means it stashes *everything*, including ignored output files (target, build, classes, etc). Ooooops.. What I meant to do was git stash -u , meaning stash modifications plus untracked new files. Anyhows, I ended up with a big fat stash I couldn't get back out. Each time I tried, I got something like this: .../target/temp/dozer.jar already exists, no checkout .../target/temp/core.jar already exists, no checkout .../target/temp/joda-time.jar already exists, no checkout .../target/foo.war already exists, no checkout Could not restore untracked files from stash No matter how I tried checking out different revisions (like the one where I actually made the stash), or using --force, I got the same error. Now these were one of those "keep cool for a second, there's a git way to fix this"situation. I figured: A stash is basically a commit. If we look at my recent commits using   git log --graph --

Leaving eyeo

Thirteen blog posts later, this one notes my departure from eyeo after 4 years and 3 months. I joined eyeo around the headcount of 80 employees, and now I think there's just over 250 people there. My role coming in was as operations manager, doing a mix of infrastructure engineering and technical project management. I later on took on organizational development to help the company deal with its growing pains . We introduced cross-functional teams, departments (kind of like guilds), new leadership structures, goal-setting frameworks, onboarding processes and career frameworks.  And all of this in a rapidly growing distributed company. I'm proud and happy that for a long time I knew every employee by name and got to meet every single new-hire through training them on company structure and processes.  At some point, we had enough experienced leaders and organizational developers that I could zoom back in on working in one team, consulting them on  Git and continuous integration

Managing dot-files with vcsh and myrepos

Say I want to get my dot-files out on a new computer. Here's what I do: # install vcsh & myrepos via apt/brew/etc vcsh clone https://github.com/tfnico/config-mr.git mr mr update Done! All dot-files are ready to use and in place. No deploy command, no linking up symlinks to the files . No checking/out in my entire home directory as a Git repository. Yet, all my dot-files are neatly kept in fine-grained repositories, and any changes I make are immediately ready to be committed: config-atom.git     -> ~/.atom/* config-mr.git     -> ~/.mrconfig     -> ~/.config/mr/* config-tmuxinator.git       -> ~/.tmuxinator/* config-vim.git     -> ~/.vimrc     -> ~/.vim/* config-bin.git        -> ~/bin/* config-git.git               -> ~/.gitconfig config-tmux.git       -> ~/.tmux.conf     config-zsh.git     -> ~/.zshrc How can this be? The key here is to use vcsh to keep track of your dot-files, and its partner myrepos/mr for o

Git-SVN Mirror without the annoying update-ref

This post is part of  a series on Git and Subversion . To see all the related posts, screencasts and other resources, please  click here .  So no sooner than I had done my git-svn presentation at JavaZone , I got word of a slightly different Git-SVN mirror setup that makes it a bit easier to work with: In short, my old recipe includes an annoying git update-ref step to keep the git-svn remote reference up to date with the central bare git repo. This new recipe avoids this, so we can simply use git svn dcommit   directly. So, longer version, with the details. My original recipe is laid out in five steps: Clone a fresh Git repo from Subversion. This will be our  fetching repo. Set up a  bare repo. Configure pushing from the fetching repo to bare repo In the shoes of a developer, clone the repo Set up an SVN remote in the developer's repo In the new approach, we redefine those last two steps: (See the original post for how to do the first three.) 4. Clone t