Security appliances wrestle with blanket coverage

To someone responsible for the network security of an SMB (small to midsize business), a one-box solution that handles every enterprise security function is a hot commodity. Naturally, the all-in-one security appliance aims to provide the required level of effectiveness without the complexity and expense of layered security products and dedicated staff. And that’s a hugely attractive prospect in today’s Wild Wild Web, where worm infections, Trojan horse invasions, and exploits of security holes are constant threats.

To assess their success, we arrayed six all-in-one security appliances in our Advanced Network Computing Laboratory testing facility at the University of Hawaii. All six of the products reviewed here go by the moniker “appliance,” but consider yourself warned: These are not foolproof, plug-and-play devices.

We invited eight firewall manufacturers to send their newest appliance offerings. Going to the mat were six brave contenders: the Check Point Safe@Office 225, the Juniper Netscreen-5GT Enhanced, the NetGear VPN Firewall FVS328, the ServGate EdgeForce Plus, the SonicWall Pro 2040, and the WatchGuard Firebox X1000. As for the two invitees who sat out the contest, Symantec couldn’t produce a production-level unit in time, and iPolicy’s product didn’t qualify as an appliance.

Our tests taught us that vendors not only have different definitions of all-encompassing security, but they also have widely varying ideas about what constitutes an appliance.

Check Point Safe@Office 225

Descending from a long line of big-iron security products with high prices and complex OSes, Check Point’s appliance turned out to be plagued only by an awkward name. The Safe@Office is an excellent contender for the SOHO firewall space.

The fact that it was the easiest of the six to configure is even more impressive when you find out that its guts are based on Check Point’s industrial-strength, best-selling Firewall 1 platform. The power of Firewall 1 is in the Safe@Office, but not the complexity, thanks to a highly intuitive front-end interface. The Services tab allows easy administration of the Safe@Office, providing a one-stop shop to configure almost every device feature, including dynamic DNS, dynamic VPN, and e-mail AV. Each is handled via an Internet connection to Check Point’s corporate servers for automatic updates and configuration help.

Although the device probably has enough muscle to protect a larger business, Safe@Office truly is a SOHO product. Check Point limits the unit to 10 simultaneous firewall connections. It’s also limited to 10 simultaneous VPN connections; however, these can be of any combination, including client-to-LAN or LAN-to-LAN. But in small, SOHO environments, the Safe@Office makes managing network security an easy and accessible task.

Typical of Check Point, the engine behind these few connections is surprisingly robust. Client VPN connections can be IPSec — a free download from Check Point — or PPTP (Point to Point Tunneling Protocol) to support the Microsoft VPN client. Additionally, clients can be authenticated against an internal database or a back-end Radius server. And the box allows for static routes, meaning you can have additional subnets behind the firewall also doing routing.

The Safe@Office proved to be a very effective firewall in our tests. The box thwarted our simulated attack sequence and rebuffed all attempts to sneak pings from the WAN to either the DMZ (demilitarized zone) or LAN.

The Safe@Office has a unique way of handling AV, so we adjusted our test accordingly. The appliance redirects e-mail traffic to a server at Check Point, which scans then forwards it to its destination. This explains why such a small CPU can handle so many services at once, but it also means that, unlike some anti-virus solutions, Safe@Office can’t check for viruses downloaded before installation.

The bottom line? Check Point stripped its much-vaunted Firewall 1 down to its bare bones to build this appliance. Fronted by a friendly user interface and using Check Point-hosted services to handle advanced functions such as AV, Web filtering, and dynamic DNS, the Safe@Office offers complex features at a low cost, but locks customers into the vendor’s services.

Juniper NetScreen-5GT Enhanced

The NetScreen-5GT Enhanced packs a load of good impressions into a tiny box, combining everything you could ask for from an all-in-one security box, including firewall, VPN, intrusion detection, and AV. The NetScreen-5GT Enhanced sports an attractive $495 price tag for 10 VPN clients.

The NetScreen-5GT presents a logical home screen on its Web interface. It doesn’t require you to dig any deeper if you’re just looking for a general status of firewall health, which is handy in an emergency — and this box can react to emergencies. The NetScreen goes beyond the basic functionality of similar boxes with specific defenses for a host of popular attacks, including WinNuke, ICMP/UDP and SYN floods, malware based on Java or ActiveX, and much more.

Displaying responses to specific attacks in a menu-like fashion, this box allows you to configure its defenses to simply sound an alarm or to start dropping malicious packets. Configured in the latter manner, it stopped everything we threw at it, allowing us to go above and beyond our normal attack suite.

We were similarly impressed with its AV capabilities. As does Check Point’s Safe@Office, the NetScreen handles AV via a subscription-based service for which Juniper partnered with Trend Micro. The device differentiates AV settings between Webmail and POP3/SMTP mail services, so we were surprised to see no AV features supported for IMAP users.

Similar to the best this roundup offered, NetScreen’s VPN testing went almost without a hitch. The box handled all 20 VPN tunnels with aplomb.

The NetScreen can fully protect a SOHO or an SMB. Its performance indicated it could handle firewall duties for networks upward of 50 clients without breaking a sweat. It not only can support multiple ISPs for fail-over, but it also can drop a dial-up connection in case of WAN failure. Web content filtering is another example, allowing the NetScreen to access the WebSense subscription service to create corporate white- or blacklists (this costs extra).

One unique feature is the NetScreen’s source routing. Every box we tested can handle static route additions, but only the NetScreen can add source routing entries that report where the routes originated and whether the source arrived via OSPF, RIP (Routing Information Protocol), BGP (Boundary Gateway Protocol), or a static entry. This is truly advanced firewall functionality that we didn’t expect in a smaller box, and we definitely didn’t expect it to be so accessible.

NetGear VPN Firewall FVS328

NetGear has made a good name for itself in the SOHO-product arena. Although the FVS328 does no harm to that reputation, we stumbled enough in our testing to be cautious about recommending this device.

This is the only box we tested that does not support anti-virus. On the upside, the FVS328 was the second-easiest machine to get into a basic configuration and, at $195, it was also the least expensive. Unfortunately, its ease of use comes at the expense of interface functionality, so if you want to do anything even slightly customized, you must spend some time with NetGear’s tech support personnel.

We ran into most of our trouble while testing VPN functionality. Our test device, the Spirent SmartBits 600 running TeraVPN, wanted to use certain encryption schemes for performance reasons, and NetGear’s interpretation of the way to set up a VPN didn’t match Spirent’s. The FVS328 makes it tricky to configure advanced VPN handshaking features such as the Diffie-Hellman Key Agreement Standard. And when we finally got there, we discovered that the box can only get as far as 3DES and Diffie-Hellman group 2. This isn’t necessarily a critical flaw in a SOHO-oriented device, but the Check Point and ServGate products offered more flexibility.

Even after we got our VPN tunnels configured, the FVS328 displayed several idiosyncrasies. We couldn’t activate a VPN tunnel unless we first sent a ping through it, which was not mentioned in the NetGear manuals. And although the docs indicated IPSec should be defined as a service, they failed to point out that doing so is critical to configuring any VPN tunnel.

NetGear reported that this tunneling problem was a known issue and sent a firmware upgrade to solve the problem, but the upgrade only made things worse. In the end, the FVS328 was so unstable that it never managed to keep enough simultaneous tunnels running to complete our VPN performance tests.

Fortunately, our firewall tests went better. Basic configuration is simple and wizard-based. The box carries a full-state inspection firewall that passed our ad hoc ping-hack attempts and stopped our static DDoS (distributed DoS) attack sequence. The wizard even offers a few niceties such as locking out certain types of traffic functions based on the hour of the day. We also liked being able to insert addresses for certificate authorities.

Although the NetGear sells at an attractive price and features a friendly wizard interface, it boils down to a no-frills firewall with VPN capabilities shoehorned in — and it shows. With its CPU lacking help for crypto processing, its performance slows noticeably when more than 10 tunnels run at once. Knowing NetGear, a better firmware version exists already, but we’d recommend testing it thoroughly before spending even the little money this box costs. Your company will be more secure for the effort.

ServGate EdgeForce Plus

The EdgeForce Plus differentiated itself from every other appliance with an impressive feature set and an excellent approach to modular configuration.

It’s essentially a small Linux server. That’s not a new approach, but ServGate was careful to leave open all critical avenues for upgrades. This allows you to not only easily upgrade the EdgeForce with free security updates, but to do so when you need it to handle additional users or a new feature set. It can expand with new software plus two additional CPUs, which would handle offloading CPU-intensive tasks such as encryption and deep packet inspection.

We also liked its approach to configuration. As with most of the firewalls, the setup for VPN Phase I was on one screen and VPN Phase II was on another, making for quick and logical VPN setups.

Once configured, the box passed all VPN tests with no trouble. ServGate even leverages some strengths of Linux to provide services, such as SSH (Secure Shell), for encrypted terminal sessions, module swapping, and user authentication for VPN connections, which can be handled either through the internal database or via a back-end LDAP or Radius server. Our test box came equipped with ServGate’s VPN Performance module, extending its rated VPN throughput to 90Mbps and making it the fastest VPN performer in our tests.

The EdgeForce relies on McAfee for AV capabilities, and it allows you to add it later as your needs dictate. This AV feature supports scanning all inbound and outbound e-mail messages, covering Webmail, POP3/SMTP, and IMAP clients. The McAfee engine performed without problems, catching all of our test viruses.

The outcome of our firewall tests was similar to those of the NetScreen and the Safe@Office. The EdgeForce carries an arsenal of specific defenses against all of the most popular attack methods, including configurable thresholds for SYN flood, ICMP (Internet Control Message Protocol) floods, and UDP (User Datagram Protocol) floods as well as support for IP source routing, Fraggle and Smurf attacks, address sweeps, port scans, and many more.

Configuration of advanced features is entirely modular, allowing you to set up the box to serve all of your current needs, leaving the door open for new configurations as needs change. The Professional Module, for example, adds a 20GB hard disk, which can be used to enable features such as Web caching, URL filtering, and local logging.

The box also has good support for QoS settings. Really more like CoS (class of service), they allow you to configure priorities and maximum allowable bandwidth for different types of services or application traffic. They can then be attached to a policy that will forward the packets between physical or virtual interfaces. It may not be traffic shaping, but it can keep certain activities, such as file downloads or streaming video, from eating the rest of your WAN.

At $7,995, the EdgeForce is by far the most expensive appliance we tested, but it is certainly one of the best performers and definitely the most upgradeable over time. Even though its advanced features require some specific network security knowledge to configure, the EdgeForce is an excellent choice for the SMB with such resources at hand.

SonicWall Pro 2040

This mid-level firewall is a jack-of-all-trades, which is clear the instant you unwrap it. You can mount it in 1U of rack space for a midsize business or place it on a desk or shelf for a SOHO-style enterprise. The Pro 2040 combines SonicWall’s next-generation SonicOS Enhanced operating system with a hardware architecture that can handle some load, as long as it’s configured properly — a task not as simple as the vendor’s reputation might lead you to believe.

Our test unit came fully equipped, but normal customers must specify the Enhanced OS to get a host of advanced features, including ISP fail-over, load balancing among multiple 2040s, policy-based NAT settings, and WAN redundancy.

Although you can operate the Pro 2040 without SonicOS Enhanced, the box wants it even on a hardware level: By installing Enhanced, you activate the box’s fourth 10/100 Ethernet interface, which can become a second WAN, LAN, or DMZ connection or a hardware fail-over connection to another Pro 2040. On the multi-function side, SonicWall is no slouch, integrating the Pro 2040 into its full suite of managed security offerings, including AV and content filtering.

The Pro 2040 performed quite adequately. For example, because SonicWall equipped the Pro 2040 with a dedicated encryption CPU, our performance numbers ran identically whether we used AES-256 or 3DES crypto modes. The Pro also handled all of our static firewall attacks and passed our AV tests. Nevertheless, considering the Pro 2040’s $1,995 price tag, we had expected more than the similar $495 NetScreen-5GT.

We discovered our single caveat when we had to change the system’s internal LAN address. Doing so caused the box to lock us out entirely, forcing a complete re-installation of the firmware configuration and deleting our original configuration and licenses. We had to re-register the box with SonicWall before we could hook up more than two VPN connections. The moral? The Pro 2040’s Web-based interface is not fool- or Hawaiian-tester-proof.

WatchGuard Firebox X1000

WatchGuard has some work yet to do. Our initial appreciation of the Firebox’s attractive red color was the first and last time we gazed at it with fondness. Soon after we began configuring it, we nicknamed it “The Microsoft Box” because it requested a hard restart even more often than a Windows server — and this from a device running embedded Linux!

Considering that each restart took at least a full 60 seconds, we got tired of it pretty quickly. System administrators will too, because restarting the box basically means restarting at least the Internet gateway. Each configuration step often involved multiple restarts, so your users won’t be thrilled either.

Even without the reboot hell, setting up the Firebox is no picnic. It uses a thick configuration client, so you must install software on a designated management workstation just to get the device to function — no other appliance we tested required this. Then, you must connect to the firewall with both an Ethernet and serial connection. The serial port is not a console at this stage; it’s just a method for the configuration software to tell the box the basic configuration information so that it can be put into a known state. The reason for this eludes us.

Even more annoying is that you won’t get VPN functionality out of the box, even though this device with a $2,500 street price is billed as a firewall and VPN appliance. If you want VPN functionality, you must download a different version of the operating software from WatchGuard’s Web site. WatchGuard argues that because it configures its products for export, strong encryption must be eliminated from the base feature set. Furthermore, although the Firebox ships with five Ethernet ports, you must pay for additional licenses to get them all active; out of the box, only the single untrusted WAN port and two of the LAN ports are active.

The single feature that sets the Firebox apart is an intense concentration of proxy technology. A good example is the HTTP proxy, which — instead of just passing through TCP port 80 — allows for much deeper examination of all Web traffic. Although implementing this technology can mean an excellent level of penetration into incoming and outgoing Web traffic, WatchGuard has not made configuration easy enough to justify its presence in a market designed for non-IT security professionals.

For those appliances that earned their appliance label, our tests showed that their feature sets are robust and mature enough to provide a well-rounded security shield for smaller companies without requiring dedicated network security staff. Choosing the box that’s right for you is simply a function of comfort with the user interface, price point, and some idea of how much growth your network will experience in the years to come.