Sizing a Web server farm with NeoLoad

In my new day job at Alpha Software, one of my responsibilities is to explore ways to expand the reach of the company’s products. One of the first projects to fall into my lap was load testing a prototype Web application built by Scott Lines, a programmer at a 600-store specialty retailer and a knowledgeable user of Alpha development products. What Scott needed was, first, proof that his application could scale on Alpha’s newest Web application server, and second, an estimate of the size of the server farm that would handle the load from all 600 stores at once.

When I was at PC Pitstop 10 years ago, our load testing was done for free, sort of. What happened was that Leo Laporte liked our site, and every once in a while he would demonstrate it on his TechTV show. Hundreds and sometimes thousands of PC users would follow along on their computers at home. At first, this would slow our site to an unusable crawl; later, as we refined our database indexes and our program logic and added another Web server behind a load balancer, this massive load would only increase our response times by some small percentage. In other words, we learned to make our application scale.

[ Cut straight to the key news for technology development and IT management with our once-a-day summary of the top tech news. Subscribe to the InfoWorld Daily newsletter. ]

The current Web load-testing software landscape ranges from a bunch of free open source projects, such as OpenSTA and Apache JMeter, with varying quirks and deficiencies, to a few very expensive products with all the bells and whistles you would ever need, such as HP LoadRunner. As it happened, while I was looking at free load-testing products, the PR firm for French load-testing vendor Neotys contacted me at my InfoWorld address about reviewing or discussing their product, NeoLoad. I explained about my day job and my fortuitous need to do a proof-of-concept load test; the company eventually decided to support the idea.

Birth of a Web farm

The next step was for us to find enough hardware to perform a meaningful test. Through the good offices of Clive Swanepoel, president of ZebraHost, we got a week’s access to a decent-sized server, installed Windows Server 2008 R2 on it, enabled Hyper-V, and created 10 VMs: eight for Web servers, one for MySQL, and one for NeoLoad. After installing and configuring the software on each VM, we set up a load-balancing switch with session affinity to balance the eight Web servers into a Web farm.

Rebecca Clinard of Neotys, USA, gave me a quick orientation on NeoLoad and took charge of setting up and running the tests. Becky, Scott, and I dialed in for a conference call and got the initial scripting and setup done for one Web server in a couple of hours using a trial NeoLoad license.

When a temporary 1,000-simulated-user license came through, we set up a test with all eight Web servers, but quickly found that all of the load was going to the first server. Becky recognized that as a common problem with “sticky” switches and asked Clive to set up seven more virtual IP addresses for the NeoLoad VM so that the load would be distributed. Her part of the setup change only took a few minutes when we reconvened the next day.

Proof of concept

We ran an initial load test and discovered a problem in the scalability, which both Becky and I thought was probably a thread-starvation condition. Even though I was new in my job, I was familiar enough with the application server to be able to change the maximum thread setting. I doubled it and crossed my fingers, as neither Becky, the server, nor the 1,000-user license would be available to do a full variational study.

When we re-ran the load test, the scalability was much improved, certainly enough for proof of concept. Scott’s employers were then able to move ahead with the project, and Scott has since been setting up real servers in the company’s data center and finishing up his application.

As I’m not intimately familiar with the whole gamut of load-testing software, I’m not really in a position to give NeoLoad a set of numerical scores for a formal review. I can say that NeoLoad had no trouble telling us what we needed to know, even though the company doesn’t have an “agent” for our application server. Just seeing the CPU and memory loads for the VMs running the Web application server told us quite a bit, and combined with the response time measurements and the MySQL diagnostics gave us a fairly good picture of what kind of hardware the retailer would need. It also provided some clues about where we might find additional performance and scalability improvements in our software.

What follows is a visual, step-by-step tour of our process. Click each screen for a closer view.

1. Defining user populations. In this case we only needed one kind of user.

5. Graphing multiple parameters for a 10-user load test on one server.

7. Maximum event duration for a 10-user load test on one server.

8. An overview graph for a second 10-user load test on one server.

10. Editing the script for the full test.

11. Setting up the ramped load for a 1,000-user test.

16. A capsule comparison graph for the two server-threading settings. The base case (dashed lines) appeared to exhibit thread starvation; the 32-thread case scaled much better.