Skip to main content

Broken Snapshots in Java Builds

Recently I've done a lot of thinking about build tools, especially in regards to Maven, Grails and Gradle, and how they play into release management and versioning with Git. This is just a post to get some of those thoughts off my chest. I'll come back to Gradle in future posts, as I build some more experience with it at work.

A few months ago, I wrote an article on our company blog about Grails' broken snapshot dependency mechanism.
Even though Grails (up onto, and including Grails 2) support snapshot dependencies, the feature is flawed in a way that makes it unusable for us. This will be fixed in Grails 3, but we couldn't wait that long, so we ended up hacking together a workaround. This article describes why and how we did it. (cont)
Now I've done a lot of modularization of huge builds over the years, and I've come to really like Maven's snapshot dependencies as an enabler for balancing between externalizing a library, and keeping it as part of the build.

So when a build tool comes along and claims that it supports snapshots, I expect it to fully support it, all the way, with local and remote repositories, time-stamping, update-policies, the lot.

So which build tools properly grok snapshots?


Which do not?

  • Ivy
  • Grails, and Griffon, because they're based on Ivy, but they will switch to Gradle next year
  • SBT uses Ivy, and thereby the Play 2.0 framework is affected as well.
I'd just like to emphasize how terrifying I find it that relatively fresh projects like SBT and Play base themselves on the muddy foundations of Ivy, instead of building something on top of Aether.

I haven't properly tested Lein, but I'll give it a run soon and update this post with the results. If anyone knows already, please comment and I'll sort it in.

What was the point of this blog post again?

The reason I'm bringing this up again is actually that there were some responses to my workaround on Twitter. Graeme Rocher, Grails project lead, responded with an alternative solution I figured it'd be nice to post in full, as his solution might work for some.

My tweet announcing the post.

Graeme:
With Grails 2.0 why didn't you just re-order the repositories as per the docs?
We're in fact still stuck on the old Grails 1.3.6, but anyhoo, I replied:
Because we want the freshest snapshot, whether it is from remote or local. See https://github.com/alkemist/grails-snapshot-dependencies-fix/issues/1
Graeme:
Ok, but surely a command line / system property to switch the repo order would have solved that for you?
Me:
TBH, that didn't occur to me. It would be a bit annoying for local dev though: Need to invoke grails twice to update deps.
Graeme:
More annoying than maintaining the hack? It would be "grails -Duselocal=true run-app". You could alias it to another cmd even :-)
In retrospect, my own hack has withstood the test of time pretty well. We haven't made any further Grails upgrades though, and I'm not sure if we will before Grails 3.0.

Some reflections on the approach Graeme suggests:

  • CI builds could always run with useLocal=false, but we would have to always deploy up-stream dependencies through the central maven repo. We pretty much always do this though, so this would work fine. Your build might be taking some shortcuts on this involving the local maven repo.
  • Developers would have to make a conscious choice when they would want to use locally built snapshots, and then run with the switch. This would work fine for us, as this is happening less as our snapshot deps are currently very stable, but I can imagine a build where you want the newest of both remote and local very often: You would then first have to run a build to get the remote ones, and then run a second to overwrite with your locally built snapshots.
  • His workaround (and mine) could be ported to other build tools (like SBT) as well.

In the end..

I suppose most Grails project developers don't care, and avoid using snapshots. This implies that
the modules that they do externalize must be released in a new version for every change that they want  to include in the downstream Grails application.

This is a hassle, but OK for slow-moving modules. If you have modules with a lot of development in them, chances are you'll just keep them part of the Grails application, and your build grow bigger and bigger, as well as maybe duplicating libraries you'd rather want to re-use in other applications.

Well, enough rambling. I hope this post might help out some Grails users, and make people more aware of problems with build tools based on Ivy..

Popular posts from this blog

Managing dot-files with vcsh and myrepos

Say I want to get my dot-files out on a new computer. Here's what I do:

# install vcsh & myrepos via apt/brew/etc
vcsh clone https://github.com/tfnico/config-mr.git mr
mr update

Done! All dot-files are ready to use and in place. No deploy command, no linking up symlinks to the files. No checking/out in my entire home directory as a Git repository. Yet, all my dot-files are neatly kept in fine-grained repositories, and any changes I make are immediately ready to be committed:

config-atom.git
    -> ~/.atom/*

config-mr.git
    -> ~/.mrconfig
    -> ~/.config/mr/*

config-tmuxinator.git  
    -> ~/.tmuxinator/*

config-vim.git
    -> ~/.vimrc
    -> ~/.vim/*

config-bin.git   
    -> ~/bin/*

config-git.git          
    -> ~/.gitconfig

config-tmux.git  
    -> ~/.tmux.conf    

config-zsh.git
    -> ~/.zshrc

How can this be? The key here is to use vcsh to keep track of your dot-files, and its partner myrepos/mr for operating on many repositories at the same time.

I discovere…

Always use git-svn with --prefix

TLDR: I've recently been forced back into using git-svn, and while I was at it, I noticed that git-svn generally behaves a lot better when it is initialized using the --prefix option.

Frankly, I can't see any reason why you would ever want to use git-svn without --prefix. It even added some major simplifications to my old git-svn mirror setup.

Update: Some of the advantages of this solution will disappear in newer versions of Git.

For example, make a standard-layout svn clone:

$ git svn clone -s https://svn.company.com/repos/project-foo/

You'll get this .git/config:

[svn-remote "svn"]
        url = https://svn.company.com/repos/
        fetch = project-foo/trunk:refs/remotes/trunk
        branches = project-foo/branches/*:refs/remotes/*
        tags = project-foo/tags/*:refs/remotes/tags/*

And the remote branches looks like this (git branch -a):
    remotes/trunk
    remotes/feat-bar

(Compared to regular remote branches, they look very odd because there is no remote name i…

Encrypting and Decrypting with Spring

I was recently working with protecting some sensitive data in a typical Java application with a database underneath. We convert the data on its way out of the application using Spring Security Crypto Utilities. It "was decided" that we'd be doing AES with a key-length of 256, and this just happens to be the kind of encryption Spring crypto does out of the box. Sweet!

The big aber is that whatever JRE is running the application has to be patched with Oracle's JCE in order to do 256 bits. It's a fascinating story, the short version being that U.S. companies are restricted from exporting various encryption algorithms to certain countries, and some countries are restricted from importing them.

Once I had patched my JRE with the JCE, I found it fascinating how straight forward it was to encrypt and decrypt using the Spring Encryptors. So just for fun at the weekend, I threw together a little desktop app that will encrypt and decrypt stuff for the given password and sa…

Android Voice Commands for Cyclists Listening to Podcasts or Music

Disclaimer: I do not recommend using earphones while on your bike, but there are times or roads where I think it's OK. Pull out your earphones when nearing potentially dangerous situations (like intersections). At least pause the audio.
These tips also apply to anyone unable to look at and touch their device, leaving voice commands their only option (useful for visually impaired people, people wearing thick gloves, etc). First of all, you need an Android with a fairly new version of Google Now installed, like Lollipop. You'll need a headset with a microphone button. I’ve got an iphone headset that works great with my old Moto G, excluding the volume control. You need to make sure that a connected headset can bypass the device’s lock mechanism. It’s in:         Settings -> Language & input -> Google voice typing -> Hands-free Your audio playback software has to work with the Google Now commands. I’ve tried Google Music and BeyondPod successfully. So, off we go! You’…

Considerations for JavaScript in Modern (2013) Java/Maven Projects

Disclaimer: I'm a Java developer, not a JavaScript developer. This is just what I've picked up the last years plus a little research the last days. It's just a snapshot of my current knowledge and opinions on the day of writing, apt to change over the next weeks/months.

We've gone all modern in our web applications, doing MVC on the client side with AngularJS or Ember, building single-page webapps with REST backends. But how are we managing the growing amount of JavaScript in our application?
You ain't in Kansas anymore So far we've just been doing half-random stuff. We download some version of a library and throw it into our src/main/webapp/js/lib, or we use it from a CDN, which may be down or unreachable when we want to use the application..

Some times the JS is minified, other times it's not. Some times we name the file with version number, other times without. Some times we get the latest library of master branch and name it with the commit-id in the fi…