Finding an alternative to metrics in assessing success

Dear Bob …

I am working with my leadership team to implement a strategy of collaboration between IT and our business partners. I’ve read your article “Run IT as a business — why that’s a train wreck waiting to happen,” Advice Line, 1/18/2010) and completely subscribe to the theory of governance within IT, and I’m trying to put together quantitatively a proposal within my company to begin down this road. I’m looking for some solid numbers though to help support my proposal from an industry standpoint.

[ Want to cash in on your IT experiences? InfoWorld is looking for stories of an amazing or amusing IT adventure, lesson learned, or war tale from the trenches. Send your story to offtherecord@infoworld.com. If we publish it, we’ll keep you anonymous and send you a $50 American Express gift cheque. ]

Do you have specific industry numbers to support this model?  If so, can you share them with me?

– Change Agent

Dear Mr. Agent …

Not only don’t I have any numbers, I don’t even know what numbers to have. We’re talking about a strategic internal shift. Strategic shifts don’t generally improve metrics — they change which metrics matter.

An example: With the standard model, “IT projects” are considered successful when they have provided all planned deliverables according to the signed-off specifications.

What often happens is that IT considers the project successful, even when nobody in the business ever makes use of the software. For example, when business managers complain that the spec was wrong, that they had no idea what the spec meant when they signed off on it, and that they only did so because if they didn’t, their project couldn’t move forward.

So according to the metric, the project was successful, even though no business benefit ever appeared and the most important result was an argument.

Compare that to the model I described in the article (and explain in detail in my most recent book). There are no IT projects — everything is about business change or improvement. Here’s what will happen to the metrics:

• Project cost estimates will be higher, because every project will have an expanded scope that includes process change, skills redefinition, often culture change as well, and business change management tasks.

• Project scope “creep” will increase, because instead of just saying no (IT doesn’t have any reason to say yes and plenty of incentive to refuse all changes), changes that enhance the likelihood of successful business change will be accepted, increasing project budget or schedule adjustments.

• The metric will change, invalidating the baseline. In the standard model, the metric is completion within the original timeframe and budget with all deliverables provided. In the new model the metric is some combination of improvements in the business: Overhead cost, unit cost, cycle time, throughput, quality, or excellence, with the business sponsor responsible for translating these to bottom-line benefits.

Here’s my suggestion: Don’t start with the numbers. It’s the wrong place to start the discussion anyway. Start with the business goal, which I think should be an improved ability to implement designed and planned change.

If that’s the goal, next figure out a metric that assesses your business’s level of success at achieving it (metrics should always be the mathematical expression of verbally defined goals) and compute a baseline that covers the past two years.

Next, choose a pilot project to try out the new way of working, and see if it works better through practical experience.

Assuming it does, broaden your use of the new approach to additional projects until it becomes a way of life.

Good luck. Sorry I don’t have the sort of numbers you’re looking for.

– Bob

This story, “Finding an alternative to metrics in assessing success,” was originally published at InfoWorld.com.