Bob Lewis
Columnist

The ROI of metrics

analysis
Mar 19, 20082 mins

Dear Bob ...I have long been a skeptic of metrics in IT. So many problems -- measuring the right thing, the reliability and sensitivity of the measures, creating perverse incentives, and gaming the measurement systems for starters. As a CIO, the only thing that really matters to me is how well the services I provide help everyone else in the organization perform well (output, cost reduction, quality, creativity,

Dear Bob …

I have long been a skeptic of metrics in IT. So many problems — measuring the right thing, the reliability and sensitivity of the measures, creating perverse incentives, and gaming the measurement systems for starters. As a CIO, the only thing that really matters to me is how well the services I provide help everyone else in the organization perform well (output, cost reduction, quality, creativity, goodwill, widgets — whatever they are measuring success on) relative to what IT services cost.

As I was crafting a message to a colleague today, I had an ah-ha moment fueled by and reinforcing my skepticism. I wondered:

Can we measure the value of performance metrics?

Has there been some objective measure of the performance of organizations that use SMART metrics versus those that don’t?

In business it might be some measure of profitability trends, P/E ratio, and/or market capitalization. In government… well, that’s tougher? Some measure of the effectiveness and value of the government services?

I have doubts as to whether the link between performance metrics and organizational performance has been proven. And if we can’t measure it (the performance impact of metrics), how can we manage it?

Thought you might appreciate this angle on the topic. Most people I know would think I’m crazy with this line of reasoning. I may be crazy, but I don’t think this line of reasoning is proof.

– Need a diagnosis

Dear Diagnosticated …

Ah, you remind me of me! Way back when, I asked the budget director of my then employer what the Return on Investment was on the budgeting process. His answer … I’m not making this up … was, “We have to have budgets!”

Spoken in a thoroughly shocked tone of voice, too.

I don’t know of any certain answer to your question regarding the existence of research on the subject. For questions of this kind — do metrics/SMART goals/outsourcing/offshoring/whatever the heck — result in business success, I go back to the two big studies I know about that looked at the sources for long-term business success: Jim Collins Good to Great study and William Joyce, Nitin Nohria, and Bruce Roberson’s Evergreen Study.

Both isolated a list of factors required for companies to outperform their competitors consistently. Neither included metrics, SMART goals, outsourcing or offshoring as factors common to highly successful companies.

Draw your own conclusions.

– Bob