The best grade I ever achieved in college (apart from English) was of all things in the subject of statistics. It wasn't particularly fun or fascinating for me. It was just easy, the rules were straight and the formulas were logical (well, I kinda dropped off when we reached quantum statistics, but luckily that wasn't within the curriculum).
The reason I mention it it that I just came back from this 3-hour presentation about Evo Project Management, signed Tom Gilb and Junior (Kai Gilb). In short, Evo is an agile method focused on measurements. They have a lot of other sound agile practices in there, but that's the main thing ringing in my head after the presentation: They use measurements to aid them in prioritizing the backlog.
Now Tom's been around the Oslo agile scene for quite a while, he did a presentation in XP-meetup a year ago. I also saw him around on the Smidig 2007 conference, doing an open-space on agile estimation or something.. Didn't attend it myself unfortunately (well, I even managed to miss my own open-space for that matter). Anyhow, this stuff isn't new to me, but it's good to get some freshup on it, cause each time I hear about it I think "this is good stuff that makes sense!".
And sense it makes. For instance, Kai started off by presenting the problems with requirement specs today: Requirements are expressed through describing the solution. By example: "Requirement 12a: The solution must be password protected.".
What's the problem with requirement 12a above? It describes the solution instead of stating the real requirement: Security. First state your requirements as qualities, then state the level of quality: The solution should be so secure that it takes a professional security-team more than 4 hours to break in.
Maybe your solution is better off using a fingerprint-scanner or smart-card solution for security.
State your requirements, make measurable goals, then let the developers decide on what solutions best serve the requirements.
This makes sense because it gives you a better way of saying when you're done with a task (big problem in agile methods).
In the long run, the method becomes a self-fulfilling prophecy. If you measure the implementations by their rate of measured of success, it means your discovering which developer efforts are making you money. Supposedly, the method has a track-record of zero failures recorded by the Gilbs. If one follows the method strictly, I'm not surprised. You will only do stuff that proves to reward yourself or your customers.
There is plenty of free documentation around on the subject, so I suggest you take a read, starting off with the Gilbs' blog.
Now onwards to the problems of the methodology. One of the last things that were said in the course was that it is a top-down method only. Errmmright. That will be a problem, being the floor-mopping, monkey-coder low level consultant with no political power that I am (I have to sneak stuff in) :)
So here we are: It's not a how-to, but more of a 10 ideas to get you started measuring success bottom-up for average agilists:
1. Learn statistics! Or Zed will kill you.
2. Managers love numbers, statistics and proof. Show them concrete figures and they will love you for it.
3. Try to avoid absolute requirements (meaning political requirements, or solution constraints). If you do get them, try to connect to a quality requirement - "Why do you want XHTML-compliance? Oh, usability, well there are plenty of other things we can do for usability that will be much cheaper and more effective than trying to be XHTML-compliant.."
4. Put some numbers on your backlog. If you've already got an agile method in use in your project, you've probably got a backlog shoved full of requirements (we've got more than we could ever hope to achieve in 2 years). Apply measurable goals to those tasks that you can do so for. "Fixing that bug will probably increase up-time to 5 nines."
5. Other things you could perhaps measure: Traffic in your webapp, number of error-lines in your log-files, number of sales. You're probably already measuring velocity on the function points you are delivering each sprint, but be sure to measure the tasks in something else than hours (measure them in difficulty/complexity and/or business value points).
6. Measure time-to-fix period. This could come in handy when convincing management to buy into larger rewrites/refactoring efforts. Right now, our mean time to get a larger functionality fix into production is 2-3 sprints. Given refactoring module X, the time will be reduced to 1 sprint.
7. Don't kill yourself measuring. Like in any other practice, be reasonable and use common sense. If it's gonna take you 2 weeks to get a web-traffic analyzing Apache instance up and running in front of your website, drop it (and put Google Analytics in or something). Use subsets, subjective input (you don't have to use empirical research data to measure), quick'n'dirty measurements.
Well, I'll try applying the list above when I get into work tomorrow. If I achieve anything remarkable with it I'll post back on the subject later. My biggest worry is that the backlog is choked with political requirements that are not based on measurable factors. Priorities are partly based on which stakeholder has got the biggest mouth or the biggest money-bag.