Proving Agile actually works (hint: you need to measure)


As I wrote before, Agile needs to prove its worth just like any other business improvement effort. However, that is easily said and harder to actually implement. What does it mean that it “works”, that there are “actual benefits”? This is a hard question to answer, however, I will give it a try.

Traditionally there are two ways of dealing with Agile improvements. The first is to just ignore measuring at all. Agile is great in and of itself, and is a goal by itself. If Agile works than you have success. Needless to say this works for some companies but not for most others. The second approach is to start measuring in detail, and there are dozens of possible metrics to track. The issue then becomes how you can actually judge based on these metrics whether things got better, and how you can possibly keep track of them. This approach works in principle, but it is hard for busy managers to deal with. And ultimately, who cares whether your CPI, focus factor, or any of the other metrics changed?

As a starting Agile consultant, I had the pleasure to work with one of the geniuses of our business, Ian Spence. He thought me a great way of dealing with this problem that I still use at each and every company work with.

The solution begins with remember the old saying “You can get better, faster, or cheaper, you can even have two out of these but not three”. Well, actually that is not true. It is true if you never improve. But in an Agile implementation we are trying to improve. In this case, you can do even better. You can get Better, Faster, Cheaper and in addition get Happier.

We then use these categories to judge how successful the Agile implementation is. To summarise, the Better, Faster, Cheaper, Happier categorisation is as shown here.

What I do is the following:
1. Together with the client decide on what the current problem is
2. Define what measurements (metrics) could be used to track improvements
3. Assign each metric to one of the 4 categories
4. Make sure you have at least one, but preferably a few, metrics per category
5. Define how you can get a single number out of the metrics for each category
6. Create a dashboard that shows the values of each category over time

Needless to say, step 5 is where the complication sets in, but it is manageable.
What you end up with is something like this:

The goal is to give an easily judged overview of where things stand at any given time. The management that you are going to be dealing with will not care about all of the underlying metrics, they just want to see whether or not things are working.

In this way, we create transparency and we can show that the real investment made in an Agile transformation actually leads to improvements over time. And if it shows we are not getting improvements, then maybe we should change course, or even stop completely. That is a perfectly valid outcome, although if you start really measuring and acting on the results that will rarely, if ever, happen.