Skip to main content

Syncing your Git repo with the Subversion repo

This post is part of a series on Git and Subversion. To see all the related posts, screencasts and other resources, please click here

This is a follow-up to the previous post on how to live with Git and Suversion in parallel. It's really trivial stuff for any experienced user, but was worth noting down somewhere for my own sake.

I've cloned a project called "fudge" from Subversion:

>git svn clone https://scm.mycompany.com/svn/fudge

So my local repo has the correct svn-remote configuration and all that to go with it inside fudge/.git/svn. This is done automatically when you svn-clone.

The bad thing is that all my Git-mates at work have to wait for me doing svn-rebase before they can pull the latest code from Subversion from my repo into theirs.

I want to get rid of this responsibility. I'll put something similar to my svn-rebasing repository on a server, have it run svn-rebase regularly, and push the changes to a centralized git repository, that everyone can pull from whenever they want. Here's how:

First, I'll init a repo on the build server called "builder". It has a designated place for the git repos in the directory /scm/git/

>cd /scm/git
>git init --bare fudge.git

The --bare means a git repo with no local checkout, i.e. just the compressed stuff inside .git.


Back on my own machine, I register the above repo as a remote for my own. I'll call it origin (this may sound confusing as my local repo is actually the origin here, but it's kind of a git convention to name the centralized repo like that):

>git remote add origin ssh://tfnico@builder/scm/git/fudge.git

The above should add the following to my fudge/.git/config:

[remote "origin"]
fetch = +refs/heads/*:refs/remotes/origin/*
url = ssh://tfnico@builder/git/java/ips.git

Now, I can push my local repository to the build server. I have to specify which branch in the remote repo I'll push to (master oughta work):

>git push origin master

So, centralized repo is ready. Now, we want a second repository on builder that can do the svn-rebase action. You could probably do this in the same repo as the first one, but I think it's cleaner to do it separately. Besides, in order for svn-rebase to work, you need to have a repo with a local checkout.

We'll create this repo just like any other developer would: By cloning from the central repo (still on builder). I'll pretend there's a user on builder called "worker" designated for this kind of action:

cd /home/worker/projects
git clone /scm/git/fudge

If we try just running svn-rebase right away, something like this might happen:

>cd fudge
>git svn rebase
Migrating from a git-svn v1 layout...
Data from a previous version of git-svn exists, but
.git/svn
(required for this version (1.7.1) of git-svn) does not exist.
Done migrating from a git-svn v1 layout
Unable to determine upstream SVN information from working tree history

You'll see that Git created the .git/svn/refs/remotes/git-svn/  folder on its own, but there's not much in there. We have to make the repo aware of the remote Subversion repo in two steps:

First, copy the svn-remote config from your original repo (the one you svn cloned) .git/config into the worker's config:

[svn-remote "svn"]
url = https://scm.mycompany.com/svn/fudge
fetch = :refs/remotes/git-svn


While imagination would allow us to think that git-svn now could figure out the rest by itself, we still need to add a pointer to the git-svn remote. Look inside the original repo's references, and you should find a file .git/refs/remotes/git-svn with a SHA in it. You have to copy this file into the worker's repo as well (or just paste the SHA into a new file with that name).

Now, try svn-rebase again:

>git svn rebase
Rebuilding .git/svn/refs/remotes/git-svn/.rev_map.901b3fc1-1df5-aa1e-233f-0cced8b7b346 ...
r123 = 38aada9c84fdb74f2271706c2a4be768334b0cc1
....
Done rebuilding .git/svn/refs/remotes/git-svn/.rev_map.901b3fc1-1df5-aa1e-233f-0cced8b7b346
Current branch master is up to date.

That's it! So, let's recap what we had to do:
  1. Create a central repo (bare)
  2. Clone the repo 
  3. Add the svn-remote to .git/config
  4. Add a pointer/SHA in .git/refs/remotes/git-svn
Now, we can set up a process that regularly does svn rebase and svn push to have it continuously update the central repo with the latest changes from Subversion. You can do it in your CI-server, as a cron-job, or maybe trigger it with a Subversion post-commit hook.

Note: When you go back to work and want to push commits back to the Subversion repo, you have to use git svn dcommit. Pushing your changes to a central git repo and then having some automated job dcommitting back is a bad idea, I think. Please let me know if you any thoughts on what workflow to use, especially when multiple people are working with Git/Subversion repos.

Comments

  1. Thanks Thomas Ferris Nicolaisen, this is what I'm looking for, but I got an error when adding a pointer/SHA in .git/refs/remotes/git-svn, bcause I couldn't find the file in my original repo, so that I always get error when rebasing. Am I miss something basic?

    ReplyDelete
  2. You have to create the pointer manually. Maybe it's a bit better explained in the next post after this one: http://blog.tfnico.com/2010/08/some-clarifications-on-living-with-git.html

    You can also send a mail to this mailing list:

    http://groups.google.com/group/git-users

    Include which command you are trying to do, and what error message you get.

    ReplyDelete
  3. After following the link above to this gitfaq url:
    https://git.wiki.kernel.org/index.php/GitFaq#How_do_I_mirror_a_SVN_repository_to_git.3F

    I think providing the --prefix=origin/ option to "git-svn init" seemed to work fine, and is easier than keeping a copy of the SHA around.

    ReplyDelete

Post a Comment

Popular posts from this blog

Managing dot-files with vcsh and myrepos

Say I want to get my dot-files out on a new computer. Here's what I do:

# install vcsh & myrepos via apt/brew/etc
vcsh clone https://github.com/tfnico/config-mr.git mr
mr update

Done! All dot-files are ready to use and in place. No deploy command, no linking up symlinks to the files. No checking/out in my entire home directory as a Git repository. Yet, all my dot-files are neatly kept in fine-grained repositories, and any changes I make are immediately ready to be committed:

config-atom.git
    -> ~/.atom/*

config-mr.git
    -> ~/.mrconfig
    -> ~/.config/mr/*

config-tmuxinator.git  
    -> ~/.tmuxinator/*

config-vim.git
    -> ~/.vimrc
    -> ~/.vim/*

config-bin.git   
    -> ~/bin/*

config-git.git          
    -> ~/.gitconfig

config-tmux.git  
    -> ~/.tmux.conf    

config-zsh.git
    -> ~/.zshrc

How can this be? The key here is to use vcsh to keep track of your dot-files, and its partner myrepos/mr for operating on many repositories at the same time.

I discovere…

Encrypting and Decrypting with Spring

I was recently working with protecting some sensitive data in a typical Java application with a database underneath. We convert the data on its way out of the application using Spring Security Crypto Utilities. It "was decided" that we'd be doing AES with a key-length of 256, and this just happens to be the kind of encryption Spring crypto does out of the box. Sweet!

The big aber is that whatever JRE is running the application has to be patched with Oracle's JCE in order to do 256 bits. It's a fascinating story, the short version being that U.S. companies are restricted from exporting various encryption algorithms to certain countries, and some countries are restricted from importing them.

Once I had patched my JRE with the JCE, I found it fascinating how straight forward it was to encrypt and decrypt using the Spring Encryptors. So just for fun at the weekend, I threw together a little desktop app that will encrypt and decrypt stuff for the given password and sa…

The End of GitMinutes (my podcast)

I'm just about ship GitMinutes episode 46, which is going to be the final episode. I'll just paste the outro script here, as it sums up the sentimental thoughts pretty well:

I’m happy to have finally finished [publishing the last episodes from Git-Merge 2017], just in time before Git-Merge 2018 takes place in March. I won’t be going there myself, so I’m counting on someone else to pick up the mic there.

It’s sad to be shipping this one as it is probably the last GitMinutes episode ever. To go a bit down memory lane, 6 years ago, my daughter was born, and as I used a little of that paternity leave to set up my podcasting infrastructure and produce the first few episodes. Initially it was just going to be 10 episodes and call the experiment finished. Instead, I got to 46 episodes, the last dozen or so lazily tailing the last few Git-Merge conferences.

To every one of my guests, thank you so much again for coming on to share your passion in this little niche of computer science a…

Joining eyeo: A Year in Review

It's been well over a year since I joined eyeo. And 'tis the season for yearly reviews, so...

It's been pretty wild. So many times I thought "this stuff really deserves a bloggin", but then it was too inviting to grab onto the next thing and get that rolling.

Instead of taking a deep dive into some topic already, I want to scan through that year in review and think for myself, what were the big things, the important things, the things I achieved, and the things I learned. And then later on, if I ever get around to it, grab one of these topics and elaborate in a dedicated blog-post. Like a bucket-list of the blog posts that I should have written. Here goes:
How given no other structures, silos will grow by themselves This was my initial shock after joining the company. Only a few years after taking off as a startup, the hedges began growing, seemingly almost by themselves, and against the will of the founders. I've worked in silos, and in companies without the…

Automating Computer Setup with Boxen

I just finished setting up a new laptop at work, and in doing so I revamped my personal computer automation quite a bit. I set up Boxen for installing software, and I improved my handling of dot-files using vcsh, which I'll cover in the next blog-post after this one.

Since it's a Mac, it doesn't come with any reasonable package manager built in. A lot of people get along with a combination of homebrew or MacPorts plus manual installs, but this time I took it a step further and decided to install all the "desktop" tools like VLC and Spotify using GitHub's Boxen:

  include vlc
  include cyberduck
  include pgadmin3
  include spotify
  include jumpcut
  include googledrive
  include virtualbox

If the above excerpt looks like Puppet to you, it's because it is. The nice thing about this is that I can apply the same puppet scripts on my Ubuntu machines as well. Boxen is Mac-specific, Puppet is not.

It was a little weird to get started with Boxen, as you're offered…