Broken Snapshots in Java Builds

Recently I've done a lot of thinking about build tools, especially in regards to Maven, Grails and Gradle, and how they play into release management and versioning with Git. This is just a post to get some of those thoughts off my chest. I'll come back to Gradle in future posts, as I build some more experience with it at work.

A few months ago, I wrote an article on our company blog about Grails' broken snapshot dependency mechanism.

Even though Grails (up onto, and including Grails 2) support snapshot dependencies, the feature is flawed in a way that makes it unusable for us. This will be fixed in Grails 3, but we couldn't wait that long, so we ended up hacking together a workaround. This article describes why and how we did it. (cont)

Now I've done a lot of modularization of huge builds over the years, and I've come to really like Maven's snapshot dependencies as an enabler for balancing between externalizing a library, and keeping it as part of the build.

So when a build tool comes along and claims that it supports snapshots, I expect it to fully support it, all the way, with local and remote repositories, time-stamping, update-policies, the lot.

So which build tools properly grok snapshots?

• Maven, of course.
• Maven Ant tasks (although it's a lot of Ant scripts to get them working)
• The Aether libraries
• Gradle (finally, since 1.0, I think they make some use of the Maven Ant tasks, and in the future they'll do more Aether stuff probably)

Which do not?

• Ivy
• Grails, and Griffon, because they're based on Ivy, but they will switch to Gradle next year
• SBT uses Ivy, and thereby the Play 2.0 framework is affected as well.

I'd just like to emphasize how terrifying I find it that relatively fresh projects like SBT and Play base themselves on the muddy foundations of Ivy, instead of building something on top of Aether.

I haven't properly tested Lein, but I'll give it a run soon and update this post with the results. If anyone knows already, please comment and I'll sort it in.

What was the point of this blog post again?

The reason I'm bringing this up again is actually that there were some responses to my workaround on Twitter. Graeme Rocher, Grails project lead, responded with an alternative solution I figured it'd be nice to post in full, as his solution might work for some.

My tweet announcing the post.

Graeme:

With Grails 2.0 why didn't you just re-order the repositories as per the docs?

We're in fact still stuck on the old Grails 1.3.6, but anyhoo, I replied:

Because we want the freshest snapshot, whether it is from remote or local. See https://github.com/alkemist/grails-snapshot-dependencies-fix/issues/1

Graeme:

Ok, but surely a command line / system property to switch the repo order would have solved that for you?

Me:

TBH, that didn't occur to me. It would be a bit annoying for local dev though: Need to invoke grails twice to update deps.

Graeme:

More annoying than maintaining the hack? It would be "grails -Duselocal=true run-app". You could alias it to another cmd even :-)

In retrospect, my own hack has withstood the test of time pretty well. We haven't made any further Grails upgrades though, and I'm not sure if we will before Grails 3.0.

Some reflections on the approach Graeme suggests:

• CI builds could always run with useLocal=false, but we would have to always deploy up-stream dependencies through the central maven repo. We pretty much always do this though, so this would work fine. Your build might be taking some shortcuts on this involving the local maven repo.
• Developers would have to make a conscious choice when they would want to use locally built snapshots, and then run with the switch. This would work fine for us, as this is happening less as our snapshot deps are currently very stable, but I can imagine a build where you want the newest of both remote and local very often: You would then first have to run a build to get the remote ones, and then run a second to overwrite with your locally built snapshots.
• His workaround (and mine) could be ported to other build tools (like SBT) as well.

In the end..

I suppose most Grails project developers don't care, and avoid using snapshots. This implies that

the modules that they do externalize must be released in a new version for every change that they want  to include in the downstream Grails application.

This is a hassle, but OK for slow-moving modules. If you have modules with a lot of development in them, chances are you'll just keep them part of the Grails application, and your build grow bigger and bigger, as well as maybe duplicating libraries you'd rather want to re-use in other applications.

Well, enough rambling. I hope this post might help out some Grails users, and make people more aware of problems with build tools based on Ivy..