The surprise price of the integrated, virtualized data center

Not all that long ago, maintaining firmware and software compatibility among different pieces of data center tech was a fairly easy affair. Upgrading your OS? No problem, a quick check of a compatibility guide would tell you whether your server’s BIOS or RAID controller needed to be upgraded. Installing a SAN? You might need to check to see that your existing FibreChannel host-based adapters and switches were at a compatible revision first.

Even in cases where multiple components were cross-dependent (such as in a FibreChannel SAN), sussing out the compatibility issues wasn’t terribly complex — usually a single compatibility matrix provided by the hardware manufacturer would get you what you needed.

Since then, the industry has seen both virtualization become more ubiquitous and a steady push toward compute, network, and storage convergence. Although I don’t dispute the potentially enormous benefits of both virtualization and convergence, the increasing degree of integration between different layers of data center tech has substantially complicated the task of keeping everything current and compatible.

The never-ending upgrade parade

Case in point: Late last year, a client started the process of upgrading its virtualization infrastructure from VMware’s vSphere 4.1 to the recently released vSphere 5.0. This client operated a pair of geographically independent virtualization clusters — each with its own FibreChannel-attached SAN and storage replication hardware that, together with VMware’s Site Recovery Manager, let the clusters provide redundancy for each other.

In general, the hypervisor upgrade process begins with an upgrade to the centralized management component (in this case, VMware vCenter) and is followed by upgrades of each virtualization host. However, in this case, the hosts were running a version of vSphere that wasn’t compatible with the newest version of vCenter, so the hosts had to be upgraded to a newer revision of 4.1 before vCenter could go to 5.0. Only then could the hosts get the final upgrade to 5.0.

As the hosts were being upgraded to 5.0, an incompatibility was discovered between their FibreChannel HBAs and vSphere — requiring new firmware for the HBAs before storage would work reliably. Along the way, the rest of the hosts’ firmware was also updated — including BIOS, local RAID, various environmental/power modules, and remote management controller to ensure that no other incompatibilities would jump out of the woodwork.

Next, it was time to get VMware’s SRM (Site Recovery Manager) software upgraded and reintegrated with the SAN and storage replication gear. By this time, a new version of SRM had been released that corrected problems in the original 5.0 release and added a few features. Unfortunately, using this new version also required a newer version of vCenter — necessitating a second upgrade of that system.

It then became clear that the new version of SRM required the firmware on the storage replication gear to be upgraded for the integration between the two to work. That firmware upgrade, in turn, required that the SANs they were attached to also be upgraded. As one of the two SANs wasn’t client-upgradable, we had to call the storage vendor to do those upgrades — introducing a substantial delay as engineering resources and downtime windows were lined up.

After all of this was completed, it looked like the original project might only be a few steps from being finished. However, only days later, new bug-fix and feature-laden versions of vSphere, vCenter, and SRM were released — triggering yet another round of upgrades. (Applying them was optional, but nobody wanted to finish an upgrade project with software that was already out of date.)

Nearly six months after the process began, nearly every component of the virtualization, storage, and compute infrastructure had been upgraded at least once — and several components had been upgraded two or three times.

The lesson: You need to research and plan more than ever

I wouldn’t fault you for deriding VMware for having so many cross-dependencies with subcomponents and third-party hardware. However, this phenomenon isn’t isolated to VMware by any means. It is regrettably something we’re all just going to have to get used to. As customers clamor for automation and integration between the compute and storage layers to add new capabilities, increase performance, and make systems easier to manage, these kinds of dependencies are an inevitable result.

In this instance, some extra research up front would have shown that what appeared to be an upgrade of a hypervisor would in fact become a software refresh for the majority of the data center. Knowing that ahead of time might have allowed it to be planned in a way that took less time and had fewer dependency surprises.

Allowing an upgrade of existing data center tech or implementation of new gear go smoothly requires a lot of upfront research to ensure there won’t be any surprises as the work progresses. As your data center gets more integrated, virtualized, and cross-dependent, that means you need to do more research and planning than you’ve done before.

This article, “The surprise price of the integrated, virtualized data center,” originally appeared at InfoWorld.com. Read more of Matt Prigge’s Information Overload blog and follow the latest developments in storage at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.