Trust me, I’m a consultant

To customers looking to upgrade their technology, a piece of advice: It’s OK to be cheap. But if you pay a consultant good money for their expertise, you’ll save yourself time and aggravation in the long run by heeding their recommendations.

About 10 years ago, I did consulting work at a company that was in the awkward stage between a small business and a midsize enterprise. The company needed to upgrade its data center and add servers to accommodate new clients’ data — in the cheapest way possible.

[ For more IT stories, read “Stupid tech support tricks: IT calls of shame.” | Follow InfoWorld’s Off the Record on Twitter for tech’s war stories, career takes, and off-the-wall news. | Subscribe to the Off the Record newsletter for your weekly dose of workplace shenanigans. ]

The “data center” was a 20-by-30-foot space in the middle of the building that used to be a conference room. There was no raised floor or dedicated HVAC unit. It had a metal shelf in the middle with 12 desktop PCs running Windows NT 4.0 and a snake’s den of extension cords running every which direction from the two power outlets in the room. One 20-amp circuit ran the whole show.

My manager put together a proposal that would help the company jump to the next level, prioritizing so that the owner, “Jim,” could get an idea of changes that would meet current needs and help reach long-term goals.

Jim liked our ideas, but strongly disagreed with my manager about some of the recommended immediate changes, such as environmental upgrades: “The room has worked just fine until now. I just need more computing power, not HVAC, an electrical upgrade, or a raised floor.”

Because my manager disagreed, Jim felt like we were trying to squeeze more money out of him. My manager backed down from his recommendations and went with Jim’s pared-down plan. We would put in a rack, three fast new servers, a SCSI-attached storage array, and a UPS. Over the next few months, we would migrate everything over to the new servers and decommission the old ones.

Two weeks later, the equipment was delivered and we arrived onsite to set it all up. Six hours, three paper cuts, and a pile of cardboard boxes later, everything was racked and ready to roll.

Fleeting success

We started hitting power buttons, and for 10 glorious seconds the new equipment roared to life. Then the lights dimmed with an angry buzz and the whole data center went dark — the circuit was overloaded. We unplugged the new servers, popped the breaker back on, and powered the old servers back up.

When we told Jim what had happened, he finally agreed to take care of the electrical problem, though he would not admit he’d been wrong. The next day, an electrician installed an additional 20-amp circuit, and we came back to install the new servers again. This time everything worked without a hitch.

A month later the equipment was working great, but it was the middle of August and the building HVAC couldn’t keep the room cool enough. The ambient temperature was around 95 degrees.

To make matters worse, Jim decided to keep the old servers running instead of decommissioning them as we had discussed. “Those computers have worked fine for years, and I don’t see why we have to get rid of them. Besides, a little heat won’t hurt anything!” he claimed.

After one old server became unstable and crashed, Jim changed his mind. The immediate solution he agreed to was propping the door open and keeping a fan blowing air in. But even he soon agreed that something more permanent had to be done.

Jim had installed a wall-mounted AC unit — with its own dedicated electrical circuit, I might add — that knocked the temperature down to the high 70s. He complained to us about the electric bill that month, but he figured that once fall hit and the temperatures cooled it wouldn’t be an issue anymore.

Fast-forward to late October. On that first cool, crisp night when the temperature dipped into the middle 30s and we had our first frost of the year, my BlackBerry exploded with warning emails from the server monitoring software — the whole data center was overheating.

Hot and cold

I arrived at the site at 4 a.m., opened the data center door, and was hit with a wave of blasting heat. The wall-mounted AC was still working, pumping out cold air as fast as it could. Then I felt a stream of hot air blowing in from the ceiling vent above. The building’s HVAC had switched to heater mode and hot air was pouring into the data center.

I propped the door open until the company handyman could remove the vent from the room later that day. The temperature then came back down to a pleasant 70 degrees and all was well again — for the time being.

As inconvenient as they were, these problems did show Jim that we were looking out for his best interests and not just trying to get more money out of him. The upside was that he began to trust us and continued using our services until his company folded in the financial chaos of 2008.

Do you have a tech story to share? Send it to offtherecord@infoworld.com. If we publish it, you’ll receive a $50 American Express gift cheque.

This story, “Trust me, I’m a consultant,” was originally published at InfoWorld.com. Read more crazy-but-true stories in the anonymous Off the Record blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.