A cocky CIO’s cloud plan comes crashing down

Lo, the CFO put out the message to cut costs. Behold, the CIO took up the call to trim expenses. Forsooth, IT was left to clean up after a poorly planned and executed cloud migration.

In the CIO’s defense (we’ll call him “Jack”), some of his ideas weren’t bad. The bigger problem was that he covered his inexperience with the temper of a Tasmanian Devil and the tact of a drill sergeant. It was a rare moment when he’d listen to what anyone else had to say — well, he listened a little when something broke.

[ Take a walk on tech’s lighter side with America’s funniest IT stories. | Get your weekly dose of workplace shenanigans by following Off the Record on Twitter and subscribing to the Off the Record newsletter. ]

The organization provided products, services, and resources for a niche market and had built up a vast amount of critical data. Security was, of course, a high priority, and the current system had worked well. IT had been hosting all of the company’s apps on internal servers and had maintained a remarkable record of uptime for the mission-critical apps by having at least two servers onsite with load-balancing capabilities and a third server at a hot offsite backup facility 500 miles away.

The IT department was operating under budget and had been for some years. In fact, the whole company was in the black. But a new CFO came onboard. After twiddling his thumbs for some weeks, he decided to take on an important CFO task: Talk incessantly about cutting costs.

Fun fact: Both Jack and the CFO owed their positions to family ties with the company bigwigs. Coincidence? In any case, the CFO’s directive buzzed in Jack’s ears, and it was only a matter of time until he came up with an idea to save the company loads of money and make him look good to the bigwigs. Jack’s command came down on a snowy winter day.

Jack called us into his office and said, “I want you to explain something to me. I’ve been reading up about this whole cloud thing, and it looks like it would be way cheaper than what we are paying right now. I mean, you all are killing me with the cost of keeping up the data center, and the folks in finance are really putting some pressure on me to get costs down. If we could put all of this on the cloud, we could save hundreds of thousands of dollars a year.”

Then he asked, “Can you give me any reason why we shouldn’t put the whole kit and caboodle on the cloud immediately?”

My boss spoke up first, explaining the company’s current system and why it worked, as well as the reasons why a move to the cloud didn’t make sense at the time for the apps we were running.

Jack ignored all this, pressing for more details about how to move to the cloud — immediately. We explained the different types of cloud offerings, the upgrades needed for the Internet bandwidth (among other items), and the costs and changes to be considered carefully. Again, we stated it was best to stay with the current system for the time being.

Obviously upset that his great idea had met with resistance, Jack waved away our advice. With a cynical chuckle, he said, “You are just afraid that if we put everything on the cloud, we won’t need you anymore to run our servers and whatever else it is you do in there.” With that, we were abruptly dismissed — end of discussion.

Over the next few weeks, Jack kept talking about the cloud. But everything we said went in one ear and out the other. Jack was hell-bent on getting every single app we used out on a cloud service and gutting the data center — the sooner, the better.

Jack took it upon himself to find a cloud service provider, and after a few conference calls with miscellaneous suitors — which we could hear clearly through the thin drywall of his corner office — a couple of salespeople showed up for a meeting. Seeing the logo on their shirts, we Googled them. Sure enough, Jack had gone with the bottom-of-the-barrel, lowest-bid provider.

Initially we groaned in dread, then realized this could work out in the end. As the old saying goes: Give them enough rope and they’ll hang themselves.

Jack tasked us to work with the provider to begin moving apps to their servers. Altogether, there were 15 different apps that would need to be transitioned in phases, and Jack wanted to start with one of the most critical programs: the app that our sales, billing, and fulfillment center used to retrieve customer orders.

After a two-week process of migration and testing, the app had been transitioned to the cloud service company, and end-user PCs had been updated to point to the new server location. Jack was raring to go on migrating the next apps. Then the problems started.

Since all our data had previously been moving around internally and we had strict Internet usage policies, a T1 line was all we’d ever needed. But Jack had refused to upgrade when we switched to the cloud provider, and now all of the customer order data was flowing through a tiny T1 Internet connection on an encrypted virtual tunnel to the cloud hosting facility two time zones away. On top of that, the app used to have its own dedicated server, but now was running on a shared virtual server with dozens of other programs.

Before, customer orders could be retrieved instantly. It now took as long as 20 seconds to bring up a single order; on busy days, it took up to 2 minutes for an order to creak out. This made it nearly impossible for the salespeople to enter orders, created a huge backlog for the fulfillment department, and wreaked havoc on the billing department.

After only a week, the anger from the user community boiled over. Jack called us into his office to explain why this was happening. Without being too smug, we gently explained it to him. From the deer-in-headlights look on his face, we wondered if he even faintly understood what had happened, much less believed us.

We then sat in on a contentious meeting between Jack and upper management to explain what changed and why. Thankfully, they listened to our explanations. And while they appreciated Jack’s efforts to cut costs, they wanted everything changed back ASAP.

Thus, the contract with the cloud provider was canceled, the app was migrated back, and Jack retreated to his corner office to bide his time until he came up with the next great idea — and the IT department enjoyed the calm before the next storm that was sure to come.

Send your own crazy-but-true tale of managing IT, personal bloopers, supporting users, or dealing with bureaucratic nonsense to offtherecord@infoworld.com. If we publish it, you’ll receive a $50 American Express gift cheque.

This story, “A cocky CIO’s cloud plan comes crashing down,” was originally published at InfoWorld.com. Read more crazy-but-true stories in the anonymous Off the Record blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.