InfoWorld review: Virtualization for development and test

As virtualization continues its fast run at transforming IT, many organizations are starting to employ the technology to create and manage transiently configured systems. These systems are typically assembled for a one-off project and torn down at project end. Virtualization is an almost perfect match for this need. IT organizations that employ virtualization for temporary systems rely on software packages called virtual lab managers, or just lab managers for short.

The term “lab managers” doesn’t quite describe all of the purposes these solutions are good for. The use cases for temporary virtualized systems cover a wide spectrum, including development, testing software, reviewing new products, running demos, doing in-house instruction, and so on. Lab managers simplify buildup and teardown, while providing many other services whose needs are not easily anticipated until you deploy virtual machines this way on a regular basis.

For this review, I looked at VMware’s Lab Manager (which I reviewed in 2006, when it was still sold by the soon-to-be-acquired Akimbi); Surgient’s Virtual Automation Platform (which I also reviewed in 2006); LabManager from VMLogix, a newcomer to lab management tools but a pioneer vendor in virtualization technologies; and Skytap, whose product is entirely cloud-based. I found that the products were excellent solutions that greatly simplified management of nonproduction virtualized systems.

How lab managers work
Lab managers are built around several basic features, all of which are implemented in the reviewed products. The software generally runs on its own dedicated server and interacts with a pool of virtualization resources (servers and storage), as well as with a dedicated storage server that holds artifacts I’ll describe shortly. In sum, the minimum standard configuration consists of at least three systems: the lab manager, the storage server, and the virtualization host or hosts.

When virtual machines are created on the host, they are implementations of templates housed on the storage server. (A template might be a Red Hat server configured with three NICs and running Tomcat.) When the need arises, an admin will select a group of templates to instantiate into virtual machines. If this group of virtual machines — for example, a database server, Web server, and a client machine — is to be handled as a single entity, the virtual machines are bundled into a management unit called a configuration. This configuration can be saved to the storage server and, later, run as a single unit.

When lab manager products create a configuration, they optionally bundle a virtual router into it. This router provides network address translation (NAT), which is necessary in the event that two instances of the same configuration are run simultaneously. The NAT in the virtual router maintains the IP addresses of the individual virtual machines inside the configuration, while exposing them outside the configuration as translated addresses. This translation prevents the conflicts that would normally arise when two virtual machines with the same IP addresses run on the same network segment. Additional software built into the configuration similarly handles the problem of multiple instances of MAC addresses.

A key feature of lab managers is the ability to take snapshots of running configurations and thereby create clones. On-the-fly cloning is invaluable in testing and QA, where a test engineer can take a snapshot of a bug when it appears and forward a link to the cloned configuration to the developer for remediation. Like VM templates, these snapshots are saved on the storage server. (They could be kept anywhere, actually, but a storage server under the care of a lab manager can enforce access control, inventory management, lifetime duration, and so on.)

Lab managers can perform many other small tasks, such as security, auditing, and the like, but their primary function is managing VM templates, configurations, and individual virtual machines in a convenient fashion.

Real labs virtually
While I’ve strived to highlight the differences between the products, the real feel as you sit at the console is that the products are much alike. They all accomplish the same tasks — construction, deployment, and teardown of VMs in groups — very well. As you’ll see in the accompanying figures, they look a lot alike too. Given that their principal functions are identical, this is not surprising.

The products shared other aspects. The native versions (i.e., all the products except Skytap) were remarkably difficult to install, and in all cases the documentation was poor. The vendors clearly expect to send out technicians on any sale, but it behooves the IT organization to have managers and admins well versed in virtualization if they expect to keep operations going without spending a lot of time on the vendor support line. In this regard, Skytap is an IT site’s dream. It is totally turnkey. You install nothing, and within an hour or two of taking the demo, users can be actively provisioning and productive.

All four packages were easy to use once the basic concepts were understood. Thus, the real determinants in the selection process are the secondary features. The most IT-friendly packages are Surgient and VMLogix. They have the greatest interoperability, both have license trackers, and Surgient has an extensive scheduling mechanism.

VMware’s product stood out comparatively in two areas: scalability and performance. While tested on different platforms, VMware was definitely faster — a tribute to skillful use of its linked clone design. It also has the reputation for running huge labs and being installed at many sites. However, it requires a complete commitment to VMware, as it does not manage other virtual machines. It is in many ways the barest but fastest offering in this group. It is also backed by the largest vendor in this review, if this factors enter into your selection equation.

For sites that have never used lab management software, I highly recommend trying Skytap. This will provide a hands-on experience with a small investment and no disruption to their existing infrastructure. Then, once addicted to lab management’s capabilities, they might consider one of the other solutions if they don’t want to be on the cloud or care to deal with Skytap’s twice-monthly windows of downtime. Skytap is also my first choice for organizations that need only occasional lab management facilities, such as testing surges caused by imminent product releases.

Overall, any solution you pick will work well, and the vendors offer evaluation plans that make it inexpensive to compare and contrast the solutions in pilot projects.

Virtual lab managers at a glance

InfoWorld review: Surgient Virtual Automation Platform 7
Surgient is in many ways the prototype of lab management, being the first vendor in this market, via its acquisition of ProTier in 2003. Surgient Virtual Automation Platform 7 is the latest release of the product that garnered the top rating in our reviews in 2006. The platform provides lab management for VMware ESX and Microsoft Hyper-V hypervisors. In fact, it can manage virtual machines on both platforms from the same pane of glass. The company expects within the next 12 months to add support for Red Hat KVM (Kernel-based Virtual Machine), the new hypervisor bundled with Red Hat Enterprise Linux, which marks that OEM’s departure from its longtime association with Xen. Surgient has no timetable for Xen support.

From the Surgient console, administrators first outfit servers with a hypervisor. These servers are then added to the inventory of usable systems. According to its capacity, a new server is given an estimate of its capabilities, including what Surgient calls EPU, or effective processing units, which are the smallest divisible unit of computing power. The EPUs, the RAM, and the storage are then assigned to resource pools, which are management entities for hosting virtual machines. As with the other products, configurations of virtual machines can be hand-tooled by the administrator or copied from a library of templates. The virtual router discussed earlier is an optional feature for the configurations. Cloning a running configuration is a simple menu click. The Surgient console below shows a configuration and its clone running simultaneously. (Click image for closer view.)

Configurations can be set up to use a physical network address and attached as a WAN segment, so that users can access them transparently, as a wide area network, without awareness that they are using a virtual machine. Surgient uses this capability in house for its own administrative needs.

As I used Surgient, I became intensely aware of how well it has integrated itself into the fabric of enterprise IT. Not only does it support both leading hypervisors, but accessing the consoles of running VMs can be done through a wide choice of options: RDP (Remote Desktop Protocol), VNC, VMware’s vSphere console access, or Citrix ICA. Deployment of VMs can be automated within Surgient or via HP Server Automation (HP’s provisioning tool for enterprises) or Altiris from Symantec. Like the other packages reviewed here, Surgient integrates with Microsoft Active Directory and LDAP. It also can relay usage and performance data to most BI tools.

While the other packages have some of these IT features, none has the rich scheduling environment that comes with Surgient. The scheduler enables users to schedule a given number of machines at a specific time and be confident the VMs will be available. This option is critical in several use cases, such as education (when a class might need 30 preconfigured VMs at a set time) or doing demos from remote locations.

The only detail that gave me pause was how Surgient handles cloning. This requires some explanation. When a clone is made on VMware vCenter Lab Manager, the system saves only the delta (that is, the changes) from the base template virtual machine. This is called a “linked” clone. The reason for this design is that deltas tend to be small files, and they’re easy to store economically and set up quickly.

Performance is the key, but it has a price: complexity. The complexity comes from situations in which the clone is later cloned. In the VMware product, cloning clones creates two or more generations of descendants from the original virtual machine. This works fine. When you boot the VM, the series of deltas are combined transparently with the original virtual machine, and you’re good to go — but what happens if a linked clone is deleted?

On VMware’s implementation, you cannot delete the original virtual machine if it has linked clones. (The virtual machine will show up as deleted on the console and be unusable, but in the background it will continue to exist as long as it has dependent clones.) However, you can delete one of the intermediate generations of delta files, thereby marooning all subsequent generations of clones on the system. They can no longer boot.

Surgient gets around this problem by allowing only one generation of linked clones. If you clone a virtual machine, it creates a delta file. If you clone the clone, it consolidates the delta file into the virtual machine, then clones the entire VM. This neatly solves the problem, but it incurs a distinct performance penalty: Second-generation clones take a long time to create.

How much this delay is a benefit or limitation depends on how often you need to clone machines. The most common use case for frequent cloning is development and test environments. In the example of the tester cloning a virtual machine for a developer to witness a bug, the developer might then run the clone and in the debugging process take a clone of that virtual machine.

At this point, the developer crosses the threshold where VMware and Surgient take divergent paths. As a user, I prefer the VMware solution because creating a clone at these points is something you want to do quickly. As a manager, I recognize the complications of multiple generations can be costly and represent an occasional burden to IT administrators.

If you tend toward the IT perspective, Surgient has an excellent solution; otherwise, you might prefer the approach taken by VMware or VMLogix.

That concern aside, I find Surgient to be an excellent product. From an IT perspective, it is the most complete package reviewed here — running on the major platforms and offering a broad range of IT hooks.

InfoWorld review: VMLogix LabManager 3.8.1
VMLogix has been providing management tools for virtualization since roughly 2004. It recently moved into the lab management market with two versions of its LabManager: One runs on native systems, while the other supports pure cloud architecture.

While the LabManager name emulates VMware’s product moniker, the product is decidedly different. VMLogix supports virtual machines running on VMware platforms, Microsoft Hyper-V, and Citrix Xen. The cloud product works with instances of Amazon EC2 machines. It supports these platforms by installing an agent on each server and passing commands back and forth from VMLogix to the server via this agent. This design is what enables VMLogix to offer a resource pool that includes hypervisors from multiple vendors — as Surgient does.

As with the other products discussed here, VMLogix acquits itself well in the creation, provisioning, deployment, and teardown of configurations of multiple virtual machines. On VMware platforms, it uses the linked clone pattern discussed earlier; with Hyper-V and Xen, it uses similar technology. However, VMLogix does not allow any links in the clone chain to be deleted if it has children. In this way, it enjoys both performance and safety.

VMLogix templates have a feature that neatly echoes one found in VMware vCenter Lab Manager — namely, advanced configuration of networking. It’s easy to create multihomed virtual machines, in which virtual network adapters can be attached to different networks and selectively bridge to outside nets. VMLogix also supports multiple virtual NICs on the same network segment to increase network bandwidth.

Moreover, the virtual switch built into configurations is uniquely enhanced in VMLogix to include a full firewall. This firewall has all the standard features you’d expect, including opening and closing ports, rerouting various protocols to specific ports, and so on.

Another unique and useful feature is the ability to push changes to a virtual machine the next time it boots up. VMLogix offers several prebuilt possibilities, such as installing software, upgrading existing packages, patching the operating system, and running internally designed scripts. (You can use any scripting language supported by the operating system, shell or batch languages being typical.) This feature is particularly helpful for admins who want to perform updates across a group of virtual machines, but don’t have the time or the inclination to start up each one to push out the change. With VMLogix, they can simply push the change, confident that it will be applied the next time the machine starts up.

One last feature that distinguishes VMLogix is the flexibility of the main console (click image below for closer view), which shows not only the usual thumbnails and resource usage levels in a real-time chart, but also user-defined fields. These are not user-selected, but user-defined, so specific attributes of VMs can be listed on the management pane. These can include reminders to the admin, ways of classifying the virtual machines, and so forth.

Overall, I was favorably impressed by VMLogix. The company has announced that an upcoming release will be able to perform lab management for both on-site and cloud setups. This will come about by integrating support for Amazon’s EC2 cloud into the product. This hybrid will enable IT sites to quickly scale resources when demand spikes, such as when performing large-scale testing of products or in other development and QA use cases.

InfoWorld review: VMware vCenter Lab Manager 4.0
VMware’s vCenter Lab Manager 4.0 is the defining product in this category, not only because it’s from VMware, the 800-pound gorilla in virtualization, but because it’s an excellent product with obvious enterprise credentials. The product shows up in some of the biggest deployments of lab virtualization, such as a large ISV that uses it on a configuration consisting of 3,000 virtual machines running on 100 hosts supported by 27TB of storage.

Installation of Lab Manager is a complex task that has the feel of heavy enterprise. VMLogix and Surgient were also difficult to install and configure, but VMware requires a three-step process: first ESX 4, then VMware vCenter, and finally Lab Manager. vCenter already contains many management features; however, Lab Manager adds capabilities that greatly facilitate the configuration and deployment of transient systems.

Prior to the advent of vCenter, Lab Manager managed the hosts directly. Today, it goes primarily through vCenter. This step enables admins to see a correct image of their systems from the vCenter console. Lab Manager does install an agent on the ESX hosts that routes the VM consoles to Lab Manager, where they are accessible via RDP and SSH. The screen image below shows thumbnails of these VMs as they appear in Lab Manager. (Click image for closer view.)

Lab Manager has a SOAP interface, which enables sites to provision virtual machines via Web services. (This is in addition to the SOAP interface in VMware ESX, which provides the ability to perform a variety of administrative tasks.) Lab Manager’s SOAP access enables QA and development staff to create and run VMs on the fly during testing runs — a unique and valuable benefit.

Another salient feature of Lab Manager is that, like VMLogix, it can enable complex, multihomed VMs that attach to multiple networks (including a bridge to the outside net) via multiple virtual NICs.

Lab Manager also provides a unique configuration switch that forces a CPU to emulate 32-bit architecture, even if it is a 64-bit processor. This enables backward compatibility with some applications and early versions of some Linux distributions — reinforcing VMware’s reputation for being able to run an enormous swath of legacy systems.

VMware’s approach to license monitoring and user management is different from the other vendors, and in my opinion, vCenter Lab Manager 4.0 trails the other products reviewed here. VMware’s entire management layer for users and licenses consists of limiting permissions of users and user roles. While VMLogix and Surgient enable you to enforce a maximum number of machines that use a given software license (VMLogix takes this one step further by having hard and soft limits), VMware attempts to reach the same ends by limiting who can create virtual machines. At that point, it’s up to those individuals to monitor their own license deployments. I find this to be a distinct shortcoming.

Lab Manager also lacks the scheduling layer found in Surgient (as does VMLogix). In this sense, Lab Manager gives the impression of being more of a lab tool than an IT-oriented product — that is, it hews closely to its origins as a development and test solution.

VMware is the least expensive product in this review. To its cost must be added the price of vCenter. However, for sites that have already committed to VMware’s products and invested in vCenter, the good pricing and the natural product fit make it a highly attractive option.

InfoWorld review: Skytap Cloud
The Skytap product, whose name changed during this review to Skytap Cloud, is an entirely cloud-based solution. For a monthly fee based on projected number of resources, you’re given a login that enables you to build and run virtual machines and configurations on Skytap’s cloud. The cloud itself is hosted by Savvis, a $1 billion market cap provider of corporate infrastructure services. It is tempting to compare Skytap with Amazon’s EC2 cloud service, but in reality there is no comparison.

For starters, Skytap runs on VMware hosts; unlike EC2, which only recently announced limited support for Windows, it lets you run all forms of Windows machines, from old versions to 64-bit heavy lifters with eight cores and 8GB of RAM. It also runs just about every other major operating system. Skytap facilitates construction of your own template library by having templates of all major machines available for retrieval from its central vault. If you grab a Windows machine, you’ll need to enter the 25-character license key the first time it starts up; then you’re good to go.

If you already have your own virtual machines, you can upload them to your library. Running Skytap was trivially easy and intuitive. Weeks after the half-hour demo, I was still able to log in and successfully run around the system, tending to all the standard tasks. Features work as expected, the interface is both intuitive and attractive, and users are immediately productive.

The cloud environment is nicely leveraged by Skytap. It’s possible to configure the lab manager’s IP configuration to be an extension of an in-house segment so that you can make the virtual machines available as normal machines to users. This is safe, as IP addresses are not exposed to the outside unless you explicitly configure them for public access and assign them one of the IP addresses in your public pool.

The linked clone issue is handled differently by Skytap. Rather than use VMware’s approach, clones are not linked. Rather Skytap drops to the OS level and simply copies the disk image of the VM in its entirety. The cloud storage is optimized for disk I/O, and the company claims that this approach can be as fast as VMware’s cloning. Given the impossibility of comparing like to like, this could not be verified — but I remain skeptical.

The Skytap console below shows a configuration of three machines running on the Skytap Cloud. (Click image for closer view.)

Pricing is favorable when compared with standard cloud options. The base enterprise pricing comprises 50 virtual machines, 5TB of storage, 1TB transfer, 10 IP addresses, 50 users, 1,000 VM-hours, IPsec, VPN, and training for $2,400 per month. VM time is measured by the minute, with no rounding for fractions of an hour. Sites can have their cloud custom branded so that there is no evidence the systems are running on an outside platform. An express plan for small businesses or tire kickers offers the full set of features for five users, five virtual machines, and 500 machine hours for a mere $250 per month.

Overall, I was very impressed with Skytap. It’s easy to use, works well, and is favorably priced. My principal complaint is that the company reserves a four-hour block of downtime twice a month (on Saturdays). It sends email reminders of these time slots well in advance. As the hour approaches, a notice appears across the admin console, but when the moment finally arrives, the virtual machines are shut down without further notice. I don’t like this downtime window, as it precludes longer-running projects, and I would prefer more warning rather than an abrupt shutdown.

Otherwise, however, Skytap is an excellent solution. I expect the company’s offering points the way to the future for lab managers.

Read more InfoWorld reviews of software development tools:

• InfoWorld review: Tools for rapid Web development
• InfoWorld review: Eight PHP power tools
• InfoWorld review: Visual Studio 2010 delivers
• InfoWorld review: Flash Builder 4 lights up rich Internet app development
• Inside the iPad SDK: Bigger screens, continued frustrations
• Java in the cloud: Google, Aptana, and Stax
• InfoWorld review: Databases primed for social networks
• Open source and SaaS offerings rethink the database