Skip to main content

Expose your test reports

I started off this post with the title "Our Quest for Agility" but it kind of trailed off into being about our latest customer-dashboard tool. I'll do another post about our agile quest later.

The Greenback project

Before the summer, I was issued with the task of setting up a customer-management portfolio tool, which I named Greenback (as in, money, because that's what we want from our customers). The first goal was to provide our customer-managers with an overview of which customers exists, where they are installed, which applications they are running and other configuration details. Greenback is basically a web application with a big table with our customers and some filters to see the customers for a single manager, or with a particular configuration. Our customer-managers can also edit, add and comment all the entries, naturally.

We have a rather large portfolio of customers (200+), and each of these are hosted with their own application of our software. Through development, we strive to keep all these customers in a working state for testing, but it's not always easy. Developing a feature for one customer will some times break the functionality in many others. Maintaining the test-data for every customer is also a challenge.

Automatic deployment and web testing


At the same time, I started pondering about how we could continuously test all these customers' applications. First thing, we got automatic deployment set up. I then made a simple web-test that basically tried loading the front page of a customer's application on the nightly deployment. I then looped this test over every customer, every morning (in our CI-environment/Hudson).

At the same time, we made a big overhaul of our test data to get every customer's test-data into working condition. It didn't take long before the first customer's application broke again on the nightly build. I started wondering: How can we keep every customer in a working state? We need some kind of proper reporting that people will respond to, because scrolling through a console output to see which applications are broken simply does not work over time.

Synergy

Then it occurred to me. Why not report the results of the web-tests back to the Greenback overview? I put together a little call-back at the end of our customer-test that wrote the result into Greenback, merely as a traffic light: Red meant that the setup scripts for the customer were not working at all, yellow meant that the application was working, but the front page wasn't working properly, and green meant everything is OK. I then made a new column in our customer table with the color of the test-result, and a link to the nightly deployment for those who want to investigate any problems, or test a particular application.

Later on, I also added a little test-history so customer-managers and developers alike could track when a certain application stopped working (we can't always drop what we're doing and fix test data every time we see a red light). Next in line is to produce a feed that people can subscribe to, to see any changes in the state of applications.

The great advantage of this is that the customer-managers themselves can take ownership in making sure that their applications are working. It was all in all pretty easy to set up (a few weeks of work, all in all), and I hope others with similar needs will find some inspiration in this.

Popular posts from this blog

Encrypting and Decrypting with Spring

I was recently working with protecting some sensitive data in a typical Java application with a database underneath. We convert the data on its way out of the application using Spring Security Crypto Utilities. It "was decided" that we'd be doing AES with a key-length of 256, and this just happens to be the kind of encryption Spring crypto does out of the box. Sweet!

The big aber is that whatever JRE is running the application has to be patched with Oracle's JCE in order to do 256 bits. It's a fascinating story, the short version being that U.S. companies are restricted from exporting various encryption algorithms to certain countries, and some countries are restricted from importing them.

Once I had patched my JRE with the JCE, I found it fascinating how straight forward it was to encrypt and decrypt using the Spring Encryptors. So just for fun at the weekend, I threw together a little desktop app that will encrypt and decrypt stuff for the given password and sa…

Always use git-svn with --prefix

TLDR: I've recently been forced back into using git-svn, and while I was at it, I noticed that git-svn generally behaves a lot better when it is initialized using the --prefix option.

Frankly, I can't see any reason why you would ever want to use git-svn without --prefix. It even added some major simplifications to my old git-svn mirror setup.

Update: Some of the advantages of this solution will disappear in newer versions of Git.

For example, make a standard-layout svn clone:

$ git svn clone -s https://svn.company.com/repos/project-foo/

You'll get this .git/config:

[svn-remote "svn"]
        url = https://svn.company.com/repos/
        fetch = project-foo/trunk:refs/remotes/trunk
        branches = project-foo/branches/*:refs/remotes/*
        tags = project-foo/tags/*:refs/remotes/tags/*

And the remote branches looks like this (git branch -a):
    remotes/trunk
    remotes/feat-bar

(Compared to regular remote branches, they look very odd because there is no remote name i…

Managing dot-files with vcsh and myrepos

Say I want to get my dot-files out on a new computer. Here's what I do:

# install vcsh & myrepos via apt/brew/etc
vcsh clone https://github.com/tfnico/config-mr.git mr
mr update

Done! All dot-files are ready to use and in place. No deploy command, no linking up symlinks to the files. No checking/out in my entire home directory as a Git repository. Yet, all my dot-files are neatly kept in fine-grained repositories, and any changes I make are immediately ready to be committed:

config-atom.git
    -> ~/.atom/*

config-mr.git
    -> ~/.mrconfig
    -> ~/.config/mr/*

config-tmuxinator.git  
    -> ~/.tmuxinator/*

config-vim.git
    -> ~/.vimrc
    -> ~/.vim/*

config-bin.git   
    -> ~/bin/*

config-git.git          
    -> ~/.gitconfig

config-tmux.git  
    -> ~/.tmux.conf    

config-zsh.git
    -> ~/.zshrc

How can this be? The key here is to use vcsh to keep track of your dot-files, and its partner myrepos/mr for operating on many repositories at the same time.

I discovere…

Joining eyeo: A Year in Review

It's been well over a year since I joined eyeo. And 'tis the season for yearly reviews, so...

It's been pretty wild. So many times I thought "this stuff really deserves a bloggin", but then it was too inviting to grab onto the next thing and get that rolling.

Instead of taking a deep dive into some topic already, I want to scan through that year in review and think for myself, what were the big things, the important things, the things I achieved, and the things I learned. And then later on, if I ever get around to it, grab one of these topics and elaborate in a dedicated blog-post. Like a bucket-list of the blog posts that I should have written. Here goes:
How given no other structures, silos will grow by themselves This was my initial shock after joining the company. Only a few years after taking off as a startup, the hedges began growing, seemingly almost by themselves, and against the will of the founders. I've worked in silos, and in companies without the…

Automating Computer Setup with Boxen

I just finished setting up a new laptop at work, and in doing so I revamped my personal computer automation quite a bit. I set up Boxen for installing software, and I improved my handling of dot-files using vcsh, which I'll cover in the next blog-post after this one.

Since it's a Mac, it doesn't come with any reasonable package manager built in. A lot of people get along with a combination of homebrew or MacPorts plus manual installs, but this time I took it a step further and decided to install all the "desktop" tools like VLC and Spotify using GitHub's Boxen:

  include vlc
  include cyberduck
  include pgadmin3
  include spotify
  include jumpcut
  include googledrive
  include virtualbox

If the above excerpt looks like Puppet to you, it's because it is. The nice thing about this is that I can apply the same puppet scripts on my Ubuntu machines as well. Boxen is Mac-specific, Puppet is not.

It was a little weird to get started with Boxen, as you're offered…