Skip to main content

Broken Snapshots in Java Builds

Recently I've done a lot of thinking about build tools, especially in regards to Maven, Grails and Gradle, and how they play into release management and versioning with Git. This is just a post to get some of those thoughts off my chest. I'll come back to Gradle in future posts, as I build some more experience with it at work.

A few months ago, I wrote an article on our company blog about Grails' broken snapshot dependency mechanism.
Even though Grails (up onto, and including Grails 2) support snapshot dependencies, the feature is flawed in a way that makes it unusable for us. This will be fixed in Grails 3, but we couldn't wait that long, so we ended up hacking together a workaround. This article describes why and how we did it. (cont)
Now I've done a lot of modularization of huge builds over the years, and I've come to really like Maven's snapshot dependencies as an enabler for balancing between externalizing a library, and keeping it as part of the build.

So when a build tool comes along and claims that it supports snapshots, I expect it to fully support it, all the way, with local and remote repositories, time-stamping, update-policies, the lot.

So which build tools properly grok snapshots?


Which do not?

  • Ivy
  • Grails, and Griffon, because they're based on Ivy, but they will switch to Gradle next year
  • SBT uses Ivy, and thereby the Play 2.0 framework is affected as well.
I'd just like to emphasize how terrifying I find it that relatively fresh projects like SBT and Play base themselves on the muddy foundations of Ivy, instead of building something on top of Aether.

I haven't properly tested Lein, but I'll give it a run soon and update this post with the results. If anyone knows already, please comment and I'll sort it in.

What was the point of this blog post again?

The reason I'm bringing this up again is actually that there were some responses to my workaround on Twitter. Graeme Rocher, Grails project lead, responded with an alternative solution I figured it'd be nice to post in full, as his solution might work for some.

My tweet announcing the post.

Graeme:
With Grails 2.0 why didn't you just re-order the repositories as per the docs?
We're in fact still stuck on the old Grails 1.3.6, but anyhoo, I replied:
Because we want the freshest snapshot, whether it is from remote or local. See https://github.com/alkemist/grails-snapshot-dependencies-fix/issues/1
Graeme:
Ok, but surely a command line / system property to switch the repo order would have solved that for you?
Me:
TBH, that didn't occur to me. It would be a bit annoying for local dev though: Need to invoke grails twice to update deps.
Graeme:
More annoying than maintaining the hack? It would be "grails -Duselocal=true run-app". You could alias it to another cmd even :-)
In retrospect, my own hack has withstood the test of time pretty well. We haven't made any further Grails upgrades though, and I'm not sure if we will before Grails 3.0.

Some reflections on the approach Graeme suggests:

  • CI builds could always run with useLocal=false, but we would have to always deploy up-stream dependencies through the central maven repo. We pretty much always do this though, so this would work fine. Your build might be taking some shortcuts on this involving the local maven repo.
  • Developers would have to make a conscious choice when they would want to use locally built snapshots, and then run with the switch. This would work fine for us, as this is happening less as our snapshot deps are currently very stable, but I can imagine a build where you want the newest of both remote and local very often: You would then first have to run a build to get the remote ones, and then run a second to overwrite with your locally built snapshots.
  • His workaround (and mine) could be ported to other build tools (like SBT) as well.

In the end..

I suppose most Grails project developers don't care, and avoid using snapshots. This implies that
the modules that they do externalize must be released in a new version for every change that they want  to include in the downstream Grails application.

This is a hassle, but OK for slow-moving modules. If you have modules with a lot of development in them, chances are you'll just keep them part of the Grails application, and your build grow bigger and bigger, as well as maybe duplicating libraries you'd rather want to re-use in other applications.

Well, enough rambling. I hope this post might help out some Grails users, and make people more aware of problems with build tools based on Ivy..

Popular posts from this blog

Encrypting and Decrypting with Spring

I was recently working with protecting some sensitive data in a typical Java application with a database underneath. We convert the data on its way out of the application using Spring Security Crypto Utilities. It "was decided" that we'd be doing AES with a key-length of 256, and this just happens to be the kind of encryption Spring crypto does out of the box. Sweet!

The big aber is that whatever JRE is running the application has to be patched with Oracle's JCE in order to do 256 bits. It's a fascinating story, the short version being that U.S. companies are restricted from exporting various encryption algorithms to certain countries, and some countries are restricted from importing them.

Once I had patched my JRE with the JCE, I found it fascinating how straight forward it was to encrypt and decrypt using the Spring Encryptors. So just for fun at the weekend, I threw together a little desktop app that will encrypt and decrypt stuff for the given password and sa…

Managing dot-files with vcsh and myrepos

Say I want to get my dot-files out on a new computer. Here's what I do:

# install vcsh & myrepos via apt/brew/etc
vcsh clone https://github.com/tfnico/config-mr.git mr
mr update

Done! All dot-files are ready to use and in place. No deploy command, no linking up symlinks to the files. No checking/out in my entire home directory as a Git repository. Yet, all my dot-files are neatly kept in fine-grained repositories, and any changes I make are immediately ready to be committed:

config-atom.git
    -> ~/.atom/*

config-mr.git
    -> ~/.mrconfig
    -> ~/.config/mr/*

config-tmuxinator.git  
    -> ~/.tmuxinator/*

config-vim.git
    -> ~/.vimrc
    -> ~/.vim/*

config-bin.git   
    -> ~/bin/*

config-git.git          
    -> ~/.gitconfig

config-tmux.git  
    -> ~/.tmux.conf    

config-zsh.git
    -> ~/.zshrc

How can this be? The key here is to use vcsh to keep track of your dot-files, and its partner myrepos/mr for operating on many repositories at the same time.

I discovere…

Always use git-svn with --prefix

TLDR: I've recently been forced back into using git-svn, and while I was at it, I noticed that git-svn generally behaves a lot better when it is initialized using the --prefix option.

Frankly, I can't see any reason why you would ever want to use git-svn without --prefix. It even added some major simplifications to my old git-svn mirror setup.

Update: Some of the advantages of this solution will disappear in newer versions of Git.

For example, make a standard-layout svn clone:

$ git svn clone -s https://svn.company.com/repos/project-foo/

You'll get this .git/config:

[svn-remote "svn"]
        url = https://svn.company.com/repos/
        fetch = project-foo/trunk:refs/remotes/trunk
        branches = project-foo/branches/*:refs/remotes/*
        tags = project-foo/tags/*:refs/remotes/tags/*

And the remote branches looks like this (git branch -a):
    remotes/trunk
    remotes/feat-bar

(Compared to regular remote branches, they look very odd because there is no remote name i…

The Best Log Viewer Ever

This is what it looks like when I want to have a look through the logfile, to see what a user did on one of our machines one day:


Read the whole story about how it works on the Viaboxx Systems blog (and upvote on DZone!).

Microsoft ups their Git efforts another notch

This week Microsoft announced first class Git support embedded in the coming version of Visual Studio.

Now, it's not completely shocking. We could have seen it coming since Microsoft started offering Git repos on CodePlex, and more recently offering a Git client for TFS. In any case, these are some big news. Scott Hanselman weighs on some features and some more background here.

For those who are a bit unaware of what the Git situation on Windows looks like these days, I've dotted down these notes:
Some explanation on these:

msysGit has long been The Way to use Git on Windows. It's basically a port of Git itself, so it's a command-line tool.GitExtensions (includes Visual Studio integration), TortoiseGit, Git Shell, posh-git and most other tools are powered by msysGit.libgit2 is a native library for doing Git stuff. It is developed completely separate from Git itself. The above tools could (and should) probably use libgit2 instead of hooking onto and around msysGit.Github…