Ixia IxLoad’s multithreaded testing

The reason you can plug in a modern Ethernet switch and expect it to connect to another Ethernet switch of a different brand is a result of the standards created by the IEEE and the IETF. However, it is due to companies like Ixia that network equipment manufacturers are able to test their new products against those standards in a consistent manner. Ixia and its competition (companies like Spirent and Agilent) have spent an amazing amount of time and money developing test systems that can consistently generate repeatable patterns and varying amounts of traffic to test standards compliance. Through these tests, network device vendors can assure their customers that there will be interoperability between their new purchases and legacy equipment.

But it is the ways those tests are configured and run that separate one piece of test equipment from another. Although a network equipment manufacturer may be satisfied with testing a single function at a time, we’re not, and neither should buyers of that network equipment. That’s why, in the scenario-based testing of network devices conducted for InfoWorld at the Advanced Network Computing Laboratory (ANCL), we run lots of different traffic patterns all at once to more closely simulate the traffic found on a modern enterprise network.

[ When is a UTM not a UTM? Read the overall results of the InfoWorld Test Center’s great UTM challenge. Read the reviews: Astaro Security Gateway 425 | SonicWall NSA E7500 | WatchGuard Firebox Peak X5500e | ZyXel ZyWall USG1000 | Compare the UTMs feature by feature. ]

The concept behind these real-world tests is to base traffic patterns on network statistics harvested from ANCL partners around the globe; these are then used as the foundation for traffic rates and patterns in our tests, as in our evaluation of the Astaro, SonicWall, WatchGuard, and ZyXel UTMs. Traditionally, comparative reviews have tested single features in isolation, one at a time. Standard RFC throughput tests were used for LAN to WAN, WAN to DMZ, and LAN to DMZ. Then a second tool was used to measure raw throughput over the VPN tunnels, and finally malware tools were pointed at the firewall. This unrealistic style of testing allowed the firewall to dedicate 100 percent of its resources at each single task. Not surprisingly, the results would overestimate both throughput and the firewall’s ability to fend off malware.

Ixia proposed a new tool to us that matches our overall concept of testing many major pieces all at once, allowing us to correlate changes in traffic patterns based upon how the UTM, in our case, manages its resources.

The Ixia IxLoad system is designed to provide for multithreaded testing of application-aware systems in what Ixia calls a “multiplay” environment. We’ve chosen a small subset of its capabilities to allow us to run various synthetic traffic patterns (HTTP, FTP, POP, SMTP) and force the devices to examine the traffic with their rules, as well as to exercise the UTM functions as the data passes through those interfaces. The ability to run simultaneously the same traffic through VPN tunnels created by IxLoad and to see how the volume of VPN traffic affects the overall throughput of the device is new and valuable. And thanks to the dedicated embedded computers behind every interface port of the Ixia test hardware, we could run extremely fast test patterns without bumping up against the limitations normally found on traditional computers. If we were to try to duplicate these functions with banks of stand-alone PCs, we would not be able to generate anywhere near the traffic rates achievable on the Ixia system.

[ Read more about InfoWorld’s UTM acid test and the Mu test tool: “How to stress a UTM” | “Mu’s Internet attacks in a can.” ]

Furthermore, the IxLoad system is not merely a “traffic replay” tool. It doesn’t just blast away, but keeps track of state. As such, it is a dramatically better model of the real world than other testing tools we’ve used. Like a real machine, the IxLoad system pays attention to TCP throttling, allowing us to actually see HTTP traffic get out of the way when FTP traffic starts ramping up. But the IxLoad’s most useful trick is integrating so many pieces into a single console. Previously, we had to manually correlate performance metrics from a VPN traffic tool, a Web traffic tool, and yet another tool for miscellaneous traffic. If the clocks were even slightly off, we could only estimate the correlation between the different traffic loads. IxLoad allows us to correlate all the various types of traffic in a single report, eliminating the need to set a ballpark figure as we had in previous tests.

We would be remiss if we didn’t mention that the Ixia system is extremely modular; you can pick up new testing features simply by licensing more software features. A direct result of having additional intelligence at each port, the Ixia chassis can run specialized tests (voice, video, data) just by changing the software downloaded to each port. You can easily imagine that, in the near future, multiple test chassis from different vendors could be integrated into a single test to come even closer to providing a complete systems simulation.

With the typical enterprise no longer willing to run a homogenous, single-vendor networking environment, IT shops are discovering the need to confirm how their networks will handle major changes involving heterogeneous gear. We could easily see running a combination of “canned” test scripts from the Ixia library to confirm performance expectations before implementing that next big upgrade, not to mention as a way to choose the truly best of breed instead of just hoping that a single vendor can satisfy all your networking needs. Note that the Ixia iSimCity facility rents access to a huge amount of Ixia gear and can run extremely complex simulations you might not normally be able to afford.