Skip to main content

Reason 1: Get your libs-folder out of SVN

This post is a part of a tiny series I'm doing on why we use Maven, and you should too.

Previous posts:

The libs folder

We have a libs-module in Subversion. When you check out the source code for our main product, this is one of the modules you get. It contains a hundred-and-some JAR-files. These are the dependencies for the sum of our modules:

fizz-project
\
 fizz-core
 fizz-web
 fizz-libs
   \
     junit.jar
     spring.jar
     common.jar
Downloading all these MB's of JAR files is something you'll have to do no matter which build system you use, so bandwith cost isn't an argument. Subversion also uses binary diffs, so copying or changing a JAR file doesn't increase repository size significantly.

The problem arises when you want more than one product. As soon as you want to split your product into two applications (or services), each project needs its own libs-folder:

fizz-project
\
 fizz-core
 fizz-web
 fizz-libs
   \
     junit.jar
     spring.jar
     common.jar
woop-project
\
 woop-core
 woop-web
 woop-libs
   \
     junit.jar
     spring.jar
     common.jar
By doing this, we almost efficiently duplicate our dependencies. While each project can manage its own dependencies without fearing for breaking functionality in the other project, efforts to maintain the libs is duplicated. All upgrading, figuring out bugs, transitive dependencies will have to be done in two places.

The alternative is to introduce a global libs folder:
fizz-project
\
 fizz-core
 fizz-web
woop-project
\
 woop-core
 woop-web
global-libs
\
 junit.jar
 spring.jar
 common.jar
... but unfortunately this would over time become a giant heap of libs of different JAR files in different versions, and for even the tiniest project you would need to check out the entire global-libs folder. Because you have no hard linking to which project a lib belongs, you can not actively do cleanups and get rid of unused JARs. Over time, it would grow to be hundreds of MB, perhaps even GB, containing every JAR file of very version all the products of the company ever used.

So neither of these approaches are particularly tasty for an organization with a large codebase and number of projects. The first approach enforces all-source-code-in-one-build with one gigantic sluggish workspace in your Eclipse, while the second enforces one gigantic global-libs JAR folder that everyone hates to download and deal with.

Maven's solution to this: There is one global JAR-repository like in the second approach, but download is done lazily. Each project has meta-data for which JAR-files it needs, and only upon building a particular project are the needed JAR-files downloaded from the company JAR-repository. Also, JAR-files are cached and referenced in a local Maven repository on each machine, so if you are working on three projects that use junit3.jar, you only have one junit3.jar in your ~/.m2/repository directory.

Additionally, Maven support inheriting dependency management. You can define parent projects that define dependencies, and have different projects inherit these.

This post is already long enough for publishing, so I'll continue next week with Reason 2: Clean up your JAR-files.
Labels:

Comments

Post a Comment

Popular posts from this blog

Open source CMS evaluations

I have now seen three more or less serious open source CMS reviews. First guy to hit the field was Matt Raible ( 1 2 3 4 ), ending up with Drupal , Joomla , Magnolia , OpenCms and MeshCMS being runner-ups. Then there is OpenAdvantage that tries out a handful ( Drupal , Exponent CMS , Lenya , Mambo , and Silva ), including Plone which they use for their own site (funny/annoying that the entire site has no RSS-feeds, nor is it possible to comment on the articles), following Matt's approach by exluding many CMS that seem not to fit the criteria. It is somewhat strange that OpenAdvantage cuts away Magnolia because it "Requires J2EE server; difficult to install and configure; more of a framework than CMS", and proceed to include Apache Lenya in the full evaluation. Magnolia does not require a J2EE server. It runs on Tomcat just like Lenya does (maybe it's an idea to bundle Magnolia with Jetty to make it seem more lightweight). I'm still sure that OpenAdvant...
6 comments
Read more

Git tools for keeping patches on top of moving upstreams

At work, we maintain patches for some pretty large open source repositories that regularly release new versions, forcing us to update our patches to match. So far, we've been using basic Git operations to transplant our modifications from one major version of the upstream to the next. Every time we make such a transplant, we simply squash together the modifications we made in the previous version, and land it as one big commit into the next version. Those who are used to very stringent keeping of Git history may wrinkle their nose at this, but it is a pragmatic choice. Maintaining modifications on top of the rapidly changing upstream is a lot of work, and so far we haven't had the opportunity to figure out a more clever way to do it. Nor have we really suffered any consequences of not having an easy to read history of our modifications - it's a relatively small amount of patches, after all. With a recent boost in team size, we may have that opportunity. Also the need for be...
1 comment
Read more

The Git Users Mailing List

A year ago or so, I came across the Git-user mailing list (aka. "Git for human beings"). Over the year, I grew a little addicted to helping people out with their Git problems. When the new git-scm.com webpage launched , and the link to the mailing list had disappeared, I was quick to ask them to add it again . I think this mailing list fills an important hole in the Git community between: The Git developer mailing list git@vger.kernel.org  - which I find to be a bit too hard-core and scary for Git newbies. Besides, the Majordomo mailing list system is pretty archaic, and I personally can't stand browsing or searching in the Gmane archives. The IRC channel #git on Freenode, which is a bit out-of-reach for people who never experienced the glory days of IRC. Furthermore, when the channel is busy, it's a big pain to follow any discussion. StackOverflow questions tagged git , these come pretty close, but it's a bit hard to keep an overview of what questio...
Post a Comment
Read more

Git-SVN Mirror without the annoying update-ref

This post is part of  a series on Git and Subversion . To see all the related posts, screencasts and other resources, please  click here .  So no sooner than I had done my git-svn presentation at JavaZone , I got word of a slightly different Git-SVN mirror setup that makes it a bit easier to work with: In short, my old recipe includes an annoying git update-ref step to keep the git-svn remote reference up to date with the central bare git repo. This new recipe avoids this, so we can simply use git svn dcommit   directly. So, longer version, with the details. My original recipe is laid out in five steps: Clone a fresh Git repo from Subversion. This will be our  fetching repo. Set up a  bare repo. Configure pushing from the fetching repo to bare repo In the shoes of a developer, clone the repo Set up an SVN remote in the developer's repo In the new approach, we redefine those last two steps: (See the original post for how to do the fir...
17 comments
Read more

Git Stash Blooper (Could not restore untracked files from stash)

The other day I accidentally did a git stash -a , which means it stashes *everything*, including ignored output files (target, build, classes, etc). Ooooops.. What I meant to do was git stash -u , meaning stash modifications plus untracked new files. Anyhows, I ended up with a big fat stash I couldn't get back out. Each time I tried, I got something like this: .../target/temp/dozer.jar already exists, no checkout .../target/temp/core.jar already exists, no checkout .../target/temp/joda-time.jar already exists, no checkout .../target/foo.war already exists, no checkout Could not restore untracked files from stash No matter how I tried checking out different revisions (like the one where I actually made the stash), or using --force, I got the same error. Now these were one of those "keep cool for a second, there's a git way to fix this"situation. I figured: A stash is basically a commit. If we look at my recent commits using   git log --graph --...
12 comments
Read more