This post is part of a series on Git and Subversion. To see all the related posts, screencasts and other resources, please click here.
I just got an email asking me how one can set up a bi-directional Git-SVN mirror. It ended up being quite a long answer, so I'll post it here for the benefit of other Git-SVN readers with the same idea.
As you may know, I'm a proponent of my own Git-SVN setup.
I remember trying to go down the path of a bi-directional repository, but always ran into problems. Here is how it could work:
However nice this would be to have, it can be very hard to achieve in practice:
Git-svn requires working in a non-bare repository, so pushing to it is by default refused. You can work around this by doing this in the target sync repo:
git config receive.denyCurrentBranch ignore
You also have to automatically perform a git reset --hard in the syncing repo after each push (by some git hook?), because the work-dir is considered dirty when it is out of sync with the repository (a consequence of the previous workaround).
And now comes the really tricky part. Consider the following example:
You see what has happened now? Bob's history has diverged from that of the sync repo:
Bob's history: X <- Y <- Z <- A
Central history: X <- Y <- Z <- A´
So the next time Bob does a pull, Git will see that history has diverged, and automatically create a merge commit so Bob's history looks like this:
A
Bob's history: X <- Y <- Z <- / \ <- merge commit
\ /
A´
You don't want this to happen, because next time Bob pushes, the merge commit will be dcommited, and so an evil circle is created.
You can avoid this by getting Bob to do a git pull --rebase, as Git (at least now when I tried it out) automagically figures out that A and A´ are actually the same commit, and discards the former of them appropriately.
You still will have to deal with some other problems:
* Concurrency between commits - basically if somebody commits to SVN before the git repository has a time to sync a push from a git user, your syncing-repo gets messed up and needs to be manually reset (and probably the git pusher will have to repeat his push). This can get ugly, and it would happen relatively often in a busy repository.
* In case you have set up a physically split bare-repo and fetching-repo (like I've recommended on numerous occasions), the concurrency issues get slightly worsened, as you need to implement some locking mechanism across both repositories. Perhaps the best thing to do here is to go back to one single git/svn syncing-repo instead.
In summary:
So, I hope you have enough information to properly consider whether this is worth going for. If you do, I would love to hear how it works out.
For the record: I do not recommend the bi-directional syncing repo approach. I still prefer my original setup where each developer is responsible for dcommitting back to Subversion. It's not so much more complicated for the Git users, and it is much safer, concurrency-wise.
I just got an email asking me how one can set up a bi-directional Git-SVN mirror. It ended up being quite a long answer, so I'll post it here for the benefit of other Git-SVN readers with the same idea.
As you may know, I'm a proponent of my own Git-SVN setup.
I remember trying to go down the path of a bi-directional repository, but always ran into problems. Here is how it could work:
However nice this would be to have, it can be very hard to achieve in practice:
Git-svn requires working in a non-bare repository, so pushing to it is by default refused. You can work around this by doing this in the target sync repo:
git config receive.denyCurrentBranch ignore
You also have to automatically perform a git reset --hard in the syncing repo after each push (by some git hook?), because the work-dir is considered dirty when it is out of sync with the repository (a consequence of the previous workaround).
And now comes the really tricky part. Consider the following example:
- Bob commits change A in his local git repository, and pushes this to the sync repo.
- The sync repo does a git svn fetch (automatically to retain linear history).
- It then performs a dcommit, but in doing so, Bob's commit is re-written (with SVN meta data), therefore the commit changes into being A´.
- The central git repo dcommits the change A´ to the SVN repo.
You see what has happened now? Bob's history has diverged from that of the sync repo:
Bob's history: X <- Y <- Z <- A
Central history: X <- Y <- Z <- A´
So the next time Bob does a pull, Git will see that history has diverged, and automatically create a merge commit so Bob's history looks like this:
A
Bob's history: X <- Y <- Z <- / \ <- merge commit
\ /
A´
You don't want this to happen, because next time Bob pushes, the merge commit will be dcommited, and so an evil circle is created.
You can avoid this by getting Bob to do a git pull --rebase, as Git (at least now when I tried it out) automagically figures out that A and A´ are actually the same commit, and discards the former of them appropriately.
You still will have to deal with some other problems:
* Concurrency between commits - basically if somebody commits to SVN before the git repository has a time to sync a push from a git user, your syncing-repo gets messed up and needs to be manually reset (and probably the git pusher will have to repeat his push). This can get ugly, and it would happen relatively often in a busy repository.
* In case you have set up a physically split bare-repo and fetching-repo (like I've recommended on numerous occasions), the concurrency issues get slightly worsened, as you need to implement some locking mechanism across both repositories. Perhaps the best thing to do here is to go back to one single git/svn syncing-repo instead.
In summary:
- Your developers will have to always do pull --rebase before and after pushing.
- It has to be a low-traffic subversion repository, preferably all committers sitting co-located.
- You have to be ready to dive in and fix any problems quickly (or have several people who can).
So, I hope you have enough information to properly consider whether this is worth going for. If you do, I would love to hear how it works out.
For the record: I do not recommend the bi-directional syncing repo approach. I still prefer my original setup where each developer is responsible for dcommitting back to Subversion. It's not so much more complicated for the Git users, and it is much safer, concurrency-wise.
in your diagram above, does the "Sync repo" represent the "fetching repo" + "bare repo" setup that you mentioned in your other post?
ReplyDeleteHi Joe,
ReplyDeleteThe diagram/setup is theoretical, but yes, I reckon that it would be a combined fetching - and central git repo. There would have to be a lot of hooks and locks to keep the repository in a correct state. I haven't given it too much thought.
In the meantime, someone has actually come up with a product that has this capability:
http://subgit.com/
have you tried subgit yet? i'm debating on whether or not i should follow your fetching-repo + bare-repo setup or give subgit a shot. wanted to get your opinion on subgit before i go down that path.
ReplyDelete@joe: I gave Subgit a quick test on a tiny local svn-repo, and it worked great. As far as I can tell it's a working two-way mirror. (there is a list of known issues on their homepage)
ReplyDeleteStill, Subgit hasn't been released yet, and I wouldn't latch the EAP version onto my main subversion repo before there's some guaranteed stability.
Also the pricing is an important factor, of course. I have no idea how they're gonna price it.
That's how we do it: https://github.com/mrts/git-svn-bridge. But be sure to check out the caveats.
ReplyDelete@mrts: Wow, that's really impressive! Well done :)
ReplyDeleteThanks :)!
Delete