XenSource Versus VMware in Performance Comparison

If you remember listening to a recent Virtualization Report Podcast earlier this month, you may well remember the back and forth “Performance Comparison” news coming out of both camps – VMware and XenSource.

VMware initially offered a white paper comparing the performance between VMware’s ESX Server 3.0.1 product and Xen 3.0.3. In that report, VMware’s product clearly outperformed the open-source Xen solution in almost every category it tested. In doing so, the VMware product was deemed the superior hypervisor platform.

It wasn’t long after the release of VMware’s comparison white paper that XenSource CTO Simon Crosby came out with a response of his own. It was clear that Crosby found the white paper flawed. First, he claimed, “Xen 3.0.3 is not a commercial product, it’s a code base. Second, Xen 3.0.3 only had partial support for hardware virtualization, and third, the VMware results are off – it seems ESX can get more than 1Gb/s out of a Gig-E NIC!”

To respond, XenSource ran exactly the same benchmarks on the same kind of machine to compare their commercial hypervisor product, XenEnterprise, against VMware’s ESX Server 3.0.1.

Until now, we have been unable to see those results because of VMware’s EULA policy which prevents anyone from publishing benchmarks of their products without permission from the company. So XenSource submitted its test plan and findings to VMware for publishing approval, and after review, VMware has agreed to let XenSource publish the comparative numbers between the two products.

XenSource published the following conclusion in their white paper:

VMware appears to have failed to appreciate the difference between our Xen open source code base and our commercial XenServer products. For example, had they read the release notes for Xen 3.0.3, they would quickly have established that Intel VT was only partially supported in that release. Moreover, XenSource’s Xen Tools for Windows, which optimize the I/O path, were not installed. The VMware benchmarks should thus be disregarded in their entirety.

Our performance results show convincingly that XenEnterprise 3.2 performs equally well or better than VMware ESX Server 3.0.1 in all but a couple of tests.

Given that we have not spent much time on optimizing our product for traditional benchmarks we are pleased to see that there is essentially no difference between the two products. Our tests highlighted a couple of areas in which ESX marginally outperforms XenEnterprise, namely on compile time, and for Netperf TX.

XenEnterprise outperforms ESX on the Passmark memory operations. XenEnterprise scales well to utilize multiple CPUs to allow a wide variety of workloads to be run. Additionally the scalability testing found that XenEnterprise provides similar scalability to ESX when additional virtual machines are added to the platform.

VMware also appears to have under-appreciated the performance advantages of the latest generation of Intel and AMD processors with built-in support for virtualization. Although VMware’s legacy technology cannot exploit these features, XenEnterprise’s high performance is proof that these technologies deliver tremendous price/performance advantages to end users. What’s more, the performance of these CPU features will increase further over time, with Moore’s Law.

In conclusion, XenSource’s commercial products are reliable, affordable and offer the Windows or Linux IT Professional a high performance platform for server consolidation for production workloads. They are very easy to use, and offer open APIs that enable our ISV ecosystem partners to complement our products to deliver a range of solutions that in every way eclipses the expensive, single-vendor, closed “solutions” marketed by VMware.

Some time today, XenSource is expected to add a link to the entire comparison white paper. Once that happens, I will update the article to point directly to it.

** UPDATE **

Click here for the link to the report!