Network detectives sniff for snoops

Just a few short years ago, an IDS was a luxury. Before the rise of the Web application and the worm, most networks were adequately defended by a firewall at the perimeter and a virus scanner at the mail server. Today, the firewall remains effective against clumsy DoS attacks and run-of-the-mill exploits, but it’s hard-pressed to thwart application-layer attacks that piggyback on welcome protocols and worms that wind their way inside the network through any overlooked port or a mobile user’s laptop.

Not only are perimeter defenses less adequate than they used to be, but internal network resources — including business-critical applications exposed to the Web — are more valuable to their companies than ever. Naturally, the double whammy of a hole-ridden perimeter and an invaluable core has network managers looking for an edge. The IDS is becoming part of the standard toolkit.

We tested four network IDS products in May, June, and July at the Naval Postgraduate School in Monterey, Calif., pitting Internet Security Systems (ISS) Proventia G200, Lancope StealthWatch 4.0, Snort 2.10, and StillSecure Border Guard 4.3 against both live Internet traffic and a variety of attacks we launched from penetration testing tool Core Impact 4.0.

Our manual attacks included OS fingerprinting, privilege escalation, DoS, banner grabbing, traversal attacks, and Microsoft IIS and Apache Web server exploits, among others. More significantly, on the live network, the products were exposed to nearly a thousand unique “attackers” targeting more than 50 ports, detecting thousands of “events” coming in from the Internet or from several thousand hosts inside the network. Among the live threats our IDS products confronted were the Sasser worm and Gator spyware.

As we expected, all four products did a good job detecting threats. With only one exception, in which one IDS initially failed to identify the Sasser worm, the products successfully alerted us to the presence of all the manual attacks and live threats they confronted. Although the four proved roughly equal in terms of recognizing attacks, important differences — ranging from ease of setup and management to depth of packet analysis and reporting, but especially the fundamental approach taken in detecting threats — may help dictate which solution best suits your network.

Snort with ACID

Snort is the famous free and open source IDS. It’s supported by an active community of users and developers who regularly and promptly update Snort’s signatures in response to newly discovered threats. Snort is a great choice if you have more time than money. When regularly maintained, Snort can be very effective. The downside is that maintenance doesn’t come easy. Snort requires care from a dedicated expert, and you’ll need to roll up your sleeves and wrestle with a difficult installation and setup.

You can pull all the files you need off the Snort project, where you’ll also find many tutorials, FAQs, and Snort manuals to help you out. The standard installation of Snort — ACID (Analysis Console for Intrusion Databases); PHP, which is required by ACID; and MySQL on Red Hat Linux — is the best-documented. A Windows XP installation is also well-documented. Deviations such as Windows 2000 and Microsoft SQL Server 2000 aren’t supported as thoroughly.

There are three run modes for Snort: Sniffer, Packet Logger, or NIDS (Network IDS). It’s easy to operate in any mode. We installed Snort on both Windows XP and Red Hat Linux 9.0, running both instances in NIDS mode. The Windows XP installation requires installing WinPcap 3.0, an architecture for packet capture and network analysis, before installing Snort. We also installed Barnyard, a free plug-in that offloads Snort logging, helping to accelerate Snort’s packet processing and thereby alleviate packet loss.

Snort’s strength is its high degree of configurability. Its main weakness is its dependence on (sometimes poor) signatures. As with all signature-based IDSes, Snort can be defenseless against unknown or “zero-day” attacks until a signature becomes available. Another problem with Snort is that some of the signatures — no doubt designed to identify older attacks — look for benign words (such as “TOP”) in the payload to determine whether a packet is malicious. As a result, an initial ruleset from the Snort project gave us several hundred false positives.

Snort developers have addressed this drawback by allowing you to comment out rules that you do not want to use on your network. The problem with this is, anytime you update your rules with the newest set from Snort.org, you’ll have to comment them out again. Oinkmaster, an open source Perl script, automates the process of enabling and disabling specified rules after each update. It was designed to run easily on Unix or Linux, but using it in a 32-bit Windows environment requires that ActivePerl, GNU, and GNUwget be installed.

We liked the fact that we could use the detection rules that came with Snort or roll out our own. Snort logs packets that are flagged by Snort rules. The rules themselves are configured in a hierarchical structure and do a good job of capturing suspicious traffic. When Snort logs in binary mode, it logs the packets in tcpdump format to a single file in a designated directory. This is especially useful in large installations that will include additional analysis with the Ethereal protocol analyzer, for example.

ACID is a graphical front end for Snort. Using it isn’t strictly necessary, and it was painful to install on Windows XP and IIS 5.0 because it also required the installation and configuration of PHP and the JpGraph graph library for PHP. But ACID is a powerful tool for handling Snort alerts, and it makes a good alternative to analyzing raw Snort data from the command line. ACID can query Snort’s binary log files or a MySQL, PostgreSQL, Oracle, or Microsoft SQL Server database.

The reporting offered by Snort and ACID was better than we expected. This was especially true when it came to ACID’s graphical reporting, which can chart information based on date, signature, protocol, IP address, port, and so on. We liked how, at the end of each user session, ACID presented an informative graph of traffic statistics.

A free IDS offers a lot of flexibility. We didn’t have to think twice about creating IDS redundancy on our test network by having distributed Snort boxes monitoring different subnets. We also liked that we could specify a particular machine on our network for log storage. The downside is that there’s no way to centrally control multiple Snort consoles.

Snort doesn’t use the NMAP (Network Mapper) port scanner to map the network but instead relies on packet sniffing, so there’s no risk of locking up or crashing a host. But packet sniffing also doesn’t provide as much detail as active fingerprinting.

Snort will require hours of configuration to tune out false positives, and its rules must be managed carefully. But it has a loyal following for good reason. Every large network should be running some kind of rules-based IDS, and Snort gets the job done.  

StillSecure Border Guard

StillSecure’s Border Guard is a commercial product built on Snort. It offers an enhanced form of signature-based protection without the painful, time-consuming installation process, endless front-end configuration, and arduous rules upkeep. Unlike Snort, Border Guard can also serve as an intrusion prevention gateway, using the rudimentary Linux iptables firewall to provide several layers of traffic blocking. The downside, of course, is that it’s not free.

Sporting the best user interface of the four primary IDS products we tested, Border Guard has strong detection and reporting capabilities, including one interesting twist that it shares with Snort: the capability of sniffing out and reporting porn usage. This feature, which boils down to inspecting traffic for illicit keywords, can be especially helpful for identifying network utilization problems and enforcing company policy.

A StillSecure site engineer was present during our installation. We walked through the entire installation and configuration process and had the 1U appliance fully operational in less than 30 minutes. We also installed the Border Guard software on a PC at our satellite facility, turning a spare machine into a hardened appliance in 15 minutes.

We were immediately im-pressed by Border Guard’s intuitive, easy-to-navigate tabbed interface. The main dashboard is tidy and understated. A stoplight in the upper left of the screen provides an at-a-glance view of overall security status. The Make Decisions tab lists the current attack or rule violation and offers options based on the severity level of the attack, including blocking the source host, clearing the alert, or deciding later. The Attack Activity tab shows a graph or table of total attacks and actions pending or taken.

Border Guard’s reporting functionality and interface are excellent. Although exports are limited to only HTML, text, or CSV (comma-separated values) formats, we were impressed with the type and scope of reporting. Powerful filters make it easy to mine data in order to investigate specific attacks or offenders.

To ease initial setup, Border Guard provides a quick-tune option that is equivalent to a whitelist for instructing Border Guard to ignore threats to specific operating systems and hosts, such as Web or Microsoft Exchange servers, that are not on your network. Through quick-tuning, you can also configure Border Guard to ignore common traffic types, such as ICMP (Internet Control Message Protocol) and SNMP, to reduce potential false positives.

Border Guard goes beyond Snort in other ways. It uses NMAP to actively identify nodes on the network, providing more accurate and detailed information. (A passive method of identifying hosts isn’t provided.) It provides several layers of event notification, including e-mail alerts to identified recipients based on the severity of a detected attack or summary e-mails based on specified thresholds or attack limits. It stores backup settings in a Linux tar (tape archive) file, making configurations easy to recall and restore.

Border Guard also supports central, Web-based management of multiple nodes, where one node in the group becomes the master console. Using the multinode manager, a single ruleset can be configured and pushed out to all nodes or groups of nodes, a nice touch in a large environment.

Updating signatures is both flexible and granular. Options range from updating entire rulesets by automatically running a command script, to inserting a firewall policy, to logging or ignoring events. Border Guard allows rule updates to be installed automatically from the StillSecure database on an hourly basis, but we found every 12 hours to be more sensible.

Our Border Guard appliance crashed three times during testing. The first two crashes were caused by having filled up the appliance’s hard drive, which was due to our setting the period of application payload capture to a lengthy five weeks. To fix the problem we had to call tech support to reindex the hard drive. Unless you have a large hard drive (200GB or bigger), we recommend using application payload capture sparingly or limiting the number of retention days to a week. Obvious factors, such as alarm settings and the makeup of your network traffic, will determine the appropriate capture settings for your enterprise.

According to StillSecure’s tech support, the third crash was due to an incompatibility between the appliance’s Dell hardware and the Border Guard software. The bug inadvertently causes the hard drive to become read-only, which prevents Border Guard from logging data and thus crashes the system. This only happened once during a month of testing but could be a significant problem. StillSecure acknowledged the bug and claims it will have a fix in the next version.

Thanks to an excellent interface, simple setup, and easy rules maintenance, Border Guard is well-suited to either the novice or the seasoned administrator. It offers all the benefits of Snort and more, without all the headaches. 

ISS Proventia G200

The Proventia G200 appliance from ISS can be deployed passively as an IDS or in-line as an IPS. Although the Proventia does a decent job of detection, we discovered that it seems better suited as a network analysis or auditing tool.

We found installation cumbersome due to Proventia’s dependency on an external database for logging. We configured the Proventia as a passive network device, using a span port on our network to monitor all traffic flowing into and out of our test environment. In addition to IDS and IPS modes, the Proventia also offers an intermediate option called the “in-line simulation” mode. Here the sensor will just send alerts about things it would normally block in IPS mode, allowing you to test IPS policies before deployment.

In addition to setting up a separate Microsoft SQL Server database, which Proventia used as the primary repository for captured data, we had to install a Windows client — the SiteProtector console — on our management workstation in order to communicate with both the sensor and the database and to retrieve stored and correlated data. Unlike the other three competitors in this comparison, ISS does not provide a Web management interface.

We concede that the Windows client does enhance security because it creates a strict relationship between the appliance and the console. But it also restricts client platform options for administrators and limits the ability to distribute administrative duties, as each console requires the management client. We’d like to have the option of using a Web management interface.

SiteProtector was easy to configure, but it’s completely dependent on the Proventia appliance and SQL Server database for all functioning and authentication. As we found out, if the SQL Server connection is not established, the appliance simply does not respond to console requests — not even with an error message. ISS should incorporate a pop-up message to inform the user when there is a problem.

This wasn’t the only usability hurdle we stumbled into. SiteProtector uses the database log-in and password combination established by the system administrator. If novice users attempt the log-in incorrectly, they are locked out without explanation.

On the plus side, the management console is designed to handle network vulnerability data from a variety of hardware and software sources, including ISS’ host-based distributed client, RealSecure Desktop, and its vulnerability management software, Internet Scanner, which we reviewed last fall. The SiteProtector console also supports several other information-gathering tools, including the SiteProtector SecurityFusion correlation engine. SecurityFusion helps you prioritize defenses against possible attacks based on other ISS product data. We came to think of SiteProtector as a Swiss army knife of sorts.

During setup, Proventia presented us with lots of options for various network configurations. For example, we could create a policy that was geared for specific router traffic or traffic coming from a specific subnet. But we also found creating and managing policies to be slightly confusing and counterintuitive. Policies we created often didn’t seem to justify the number of steps we were required to take — or the variety of templates we had to wade through — in order to get there.

Among all the products we reviewed, we found that Proventia put the strongest emphasis on network traffic auditing, thanks to deeper protocol analysis capabilities than its competitors’. For example, if you had a zero-tolerance policy for FTP traffic, Proventia could easily supply you with the information necessary to combat violations, even going as far as capturing user names and passwords that are sent in the clear. Of course, the same auditing policy approach works with other types of traffic such as HTTP and POP3.

Application-layer traffic filtering in Proventia is extensive out of the box. The application auditing it provides is nonexistent in StealthWatch and would require creating custom rules in Border Guard and Snort. For example, we could filter on POP3 traffic and inspect headers for source and destination, and we could view quite a substantial amount of information regarding the session transaction — even when not viewing the actual payload.

Proventia also offers a plethora of options for reporting, including the ability to collect application-specific data such as successful FTP log-ons, Telnet users and passwords, or HTTP session information. A 3-D pie chart of current traffic activity gives the user a quick overview and the ability to drill down into the details.

Proventia was the only IDS among the four that didn’t catch the Sasser outbreak during testing. After we notified ISS, its engineers were able to trigger alerts off the Sasser signature, but even this took several attempts. Despite this shortcoming, Proventia earned higher marks in threat detection than Snort did, thanks to its avoidance of false positives. Proventia produced fewer false positives — but also fewer true positives — than either Snort or Border Guard.

Proventia is powerful and flexible but also complex. Its deep packet-analysis capabilities make it a good compliance-auditing tool, but the product didn’t strike us as the best fit for straight intrusion detection. StillSecure’s Border Guard is much better suited to that job, not only because it’s easier to install and configure but also because it’s more straightforward to maintain and monitor on an ongoing basis.

Lancope StealthWatch

Lancope’s StealthWatch takes a different tack to detecting malicious activity than the other three IDS products we tested for this comparison. Instead of relying on signatures or predefined patterns to identify attacks, StealthWatch relies on anomalies — or exceptions to normal traffic trends — as indicators of a threat. This approach makes StealthWatch especially well suited to detecting worm outbreaks and exploits of unknown vulnerabilities.

While all four of our IDS products were online for testing, StealthWatch alerted us to the potential Sasser outbreak before the other devices did. The downside to StealthWatch’s approach is that the device must first learn your network’s normal traffic patterns, commonly called “behavioral baselining.” This process takes time, in some cases as long as several weeks.

StealthWatch uses a distributed architecture for deployment, with a master console that communicates with distributed sensors via specified ports and encrypted channels. The management console is not strictly needed for network safety. We installed the M250 alone on a satellite network, and it was quite effective. 

StealthWatch doesn’t require a separately managed database, and it secures each install by presenting new default command line and administrative log-in combinations. Another plus is its capability of integrating with Snort 2.0.5 and ISS RealSecure Network Sensor 7.0, allowing you to pull information from these signature-based detection systems into the StealthWatch console.

StealthWatch will appeal to the detail-oriented user. But the level of detail bleeds into the configuration process, which is intricate and time-consuming. On the upside, the appliance has an auto-tuning feature that sets the initial “concern index” threshold high enough to avoid false positives yet low enough to continue monitoring suspicious network activity. Lancope helps organize this monitoring by grouping similar hosts into zones.

Just as StealthWatch needs time to learn your network, administrators will need time to learn StealthWatch. The dashboard is split into two components: A concern index focuses on the sources of attacks, while a target index focuses on the destinations. The StealthWatch appliance monitors the behavior of each host on the network, as well as cumulative network activity. The higher the index value, the more likely a source is dangerous or a target is under attack.

Learning to judge index values and set appropriate thresholds doesn’t come easy. Although the dashboard provides a nicely consolidated view of potentially anomalous events, a certain amount of networking expertise is required to interpret what’s presented. Ultimately, StealthWatch requires a technically savvy operator and shouldn’t be used by a novice administrator.

Lancope does provide a number of features designed to make StealthWatch easier to use. The watch list, for example, allowed us to enter a specific IP or port number to monitor on an ongoing basis. We used the watch list to track the Naval Postgraduate School’s e-mail server during the Sasser outbreak. Unfortunately, it’s not as easy to specify hosts or ports to ignore; StealthWatch can ignore alarms from specific machines, but that’s not quite the same. An easy way to whitelist trusted machines would be a good addition.

Reporting was more than adequate but could be improved. Although we could drill down on alerts to discover details of suspicious activity, we would also have liked to see hyperlinks to graphs in the daily reports, for example, so we could drill down to graphical views of traffic and anomalies.

Overall, we found StealthWatch to be an excellent solution, with one downside — the lack of a signature-based detection engine. Its capability of flagging unknown attacks is a huge benefit, but it requires expertise and interpretive skill from administrators. Although the quick-and-dirty identification of known attacks is valuable, this is made unnecessarily difficult by StealthWatch. Whether by integration or parallel deployment, combining StealthWatch and a signature-based IDS would enhance overall security.

Nevertheless, if we were charged with bringing maximum security to a mission-critical network and money were no object, StealthWatch would be our first choice. Its capability of detecting zero-day attacks and all anomalous occurrences, such as our Sasser worm, move it ahead of the pack.

Border Guard is our No. 2. Combining easy setup, smooth management, and powerful reporting, it brings much-needed polish and an additional measure of effectiveness to a solid Snort core. Border Guard is also an excellent value, making it a close second to StealthWatch for any network.

— Mark A. Givens and Charles D. Herring of the Naval Postgraduate School contributed to this review.

(Return to special report)