The Windows server migration that wouldn’t go away

At the time of this story, I was involved with a huge datacenter consolidation project. With only a handful of people, we were condensing datacenters, combining three major networks on four continents, and integrating five different carriers. As the multiyear project was winding down, one lone server at a closing datacenter remained an enigma — and became my headache.

It was a Windows server, yet it was not part of the Windows server migration. It had fallen through the cracks. Despite the fact that my role in the project was coordinating network integration and that our group had a fully staffed Windows/Intel team, this migration was tossed in my court.

I was buried in my own migratory trials and tribulations with no time for one more inherited fiasco in the making. After putting it off for several weeks, my boss became insistent that I work on it. When I asked what he wanted me to put on the back burner, I got a rather disbelieving look, complete with rolled eyes.

In between other migration duties, I tried to learn what exactly this thing did. I found that the server’s software would soon be out of support and the hardware support would run out the following year. On top of that, most of the function could be migrated to existing servers with minimal fuss on current hardware and software. I reported my finding up the food chain and hoped that was the end of it.

After another week, I was asked where I was on the server’s migration. So, again, I started looking into it. The contractor in charge of the server’s datacenter provided some resources, but they proved to be unhelpful runarounds. I finally spoke to The Guy (we’ll call him Jim) at the closing datacenter who knew about the server inside and out — or so I was told.

Jim said it was a simple thing to move the server — a small matter to change the IP addresses and be done with it. I relayed this information as a quick way to settle the issue and got back to my primary tasks.

None of my superiors were satisfied with that simple plan. I was instructed to create a migration document. The Windows/Intel team were not writing migration plans for their servers, so I put a few key steps down on paper and submitted it. No go, insufficient detail.

At that, I set out to create a micromanager’s dream come true. I documented every single thing — creating the plan, powering off the unit, unplugging the cords, crating it all the way, plugging in a network cable, and pinging the gateways. In total it was seven pages of minutiae.

Then it backfired. The detailed migration plan was well received — too well, as a matter of fact. It was heralded as a model that all other plans would be judged against. Great, I had met the enemy and it was me!

Now I had to implement the monster. The first page and a half went well. One item was receiving the administrator-level IDs and passwords from Jim and testing them. This device had a secure portion that can only be accessed via local keyboard, video, and mouse (KVM). Since I was not on-site, Jim assured me that he had tested them and they worked. I documented it, and we moved on.

After more delays getting the expired maintenance and support reinstated, the unit was finally de-racked and shipped. When it arrived at our location, I proceeded with the next two pages of the plan, which included re-racking the server and connecting the KVM. Once powered on, the administrator-level ID and password did not work. I called Jim, and he was baffled. He gave me another password to try. No luck. By the time I got off the phone with him, I had seven passwords, none of which worked. I’d also discovered that Jim had never actually logged on to the secure portion of the server.

Now I jumped to the back of the seven-page plan for the contingencies. Based on the circumstances, the server was shipped back to the original site to let Jim sort it out. Again I caught up on more pressing matters while this mess simmered out of sight.

A meeting was eventually held with my superiors and those from Jim’s datacenter to determine the next step. I was adamant that I would not accept the unit back without first logging in to the secure area myself. I would not take anyone’s word for it.

Jim and his bosses assured me that would be possible as soon as they determined how to recover the lost password. It came out in this meeting that the server was installed by another person, no longer on our account, and Jim had been drafted to deal with it. This had been a common thread throughout the consolidation: The talent had moved on and we were left with what was left over.

Days passed and I heard they finally found a guy that knew a guy that knew The Man that built the server originally. The password was supposedly tested locally by someone Jim knew and trusted.

Right. My request for KVM access floated around for a day or two before it was discovered they did not have any KVM access to the server. One more day passed before I shipped them an IP-enabled KVM preconfigured for their network so that I could check off the box on my plan.

After some confusion, mostly on the remote end, I was logged in to the secure server, and the migration continued. There were other issues before it was finally completed, but nothing insurmountable.

The moral of this story is that the saying holds true: Trust but verify.

Oh, and those seven passwords Jim gave me were very helpful for the Windows team. When they were working with the same datacenter, they needed a few of them for their own migrations with similar circumstances.

Related articles

• Stupid user tricks 4: IT horror never ends: Nine more real-world disasters courtesy of your network’s weakest link.
• True IT confessions: Supergeeks fess up to some of the dumbest things they’ve ever done — and the lessons they learned as a result
• Even dirtier IT jobs: The muck stops here: More dirty tech deeds, done dirt cheap.
• Dirty vendor tricks: From magical demos to deceptive pricing and fictional charges, here are the six most devious tricks vendors use to get their hands in your pocket.
• Tech’s all-time top 25 flops: These pivotal moments are the history you don’t want to repeat.
• Stupid hacker tricks, part two: The folly of youth: Tech-savvy delinquents set the Net aflame with boneheaded exploits that earn them the wrong kind of fame.
• Crazy tech support stories: An IT support specialist remembers the calls that made him push the mute button while he pulled himself together
• IT snake oil: Six tech cure-alls that went bunk: Legendary promises, little delivery — IT history is fraught with “transformational” sales pitches that fell far short
• The 7 deadly sins of IT management: Beware these common IT transgressions before you inadvertently sabotage your company’s tech agenda.