by Michael Baum

Challenges of Greylisting

analysis
Nov 15, 20054 mins

Protecting e-mail systems from spam is one of the great battles for IT shops at companies of all sizes. There are lots of commercial and open source technologies that can help you out. One of the most popular new techniques is greylisting, the process of setting up your MTA to reject e-mail from unknowns with a “try again later” message. The rejection happens at the SMTP layer so the end-user is not involved. What this process introduces is that, initially anyway, all mail gets delayed until the sender tries again. But, the good news, most spamming software will not try again later. Greylisting gives sys-admin’s the ability to keep a watch on mail logs and see what servers do try again versus those that don’t (likely spammers) and take corrective actions to deal with the spamming servers or just reject the email all together. There is a great Infoworld blog entry on greylisting by Paul Venezia.

So greylisting sounds great, right? Well like most new technologies that enter the IT shop, there are some significant implementation challenges to consider. And troubleshooting these problems can be difficult. Here are some of the more problematic issues I’ve seen.

First consider you may need to relocate your MX Servers. You need to have control over your MX hosts and may need to relocate the hosting of your MX records from a public ISP own or ones you can configure. This is usually not too difficult.

Secondly, you need to understand delays between retries for your MTAs. Greylisting works by examining IP address, sender envelops and receiver envelopes triples from each MTA involved in the greylisting process. The first time someone sends mail to you it will be delayed because the triple has not been seen before. Greylisting works by training the system (automatically) on triples the system has already seen and approved for immediate deliver or rejection. Each unique triple combination is considered different, unless you while list a larger address space e.g. a whole class C. Perhaps the most difficult part of troubleshooting MTA interactions will be understanding your MTA back off algorithms. First attempt and second attempt retry settings can vary widely and depending on where the settings begin and how they escalate, your configuration may wide up delaying mail for unacceptable periods of time or bouncing mail accidentally.

Lastly, if your infrastructure receives mail from other senders running a pool of MTAs, you’ll have some additional configuration issues and troubleshooting tasks. Let’s say, for example, a large enterprise you communicate with has a pool of five or ten MTAs running of different hosts to handle large volumes of mail and/or for redundancy purposes. Now say MTA 1 tries to send a message and MTA 2 tries to resend the message after receiving your greylisting systems retry request. The request could end up bouncing around from MTA to MTA in an MTA ping-pong if it never sees the same two MTAs. Once you find the problem, by looking through lots of mail logs, you might consider configuring your system to look at the inbound IP addresses from this sender and only consider the class C equivalent. Now all class C MTAs will be considered the same triples rather than individual unique combinations of IP address, sender envelope and receiver envelope.

Most greyllisting implementations are based on Perl or C and use something like MySQL as a database backend. Most of your troubleshooting data you’ll need to be concerned with can be found in your MTA logs, the greylisting application logs and the MySQL database logs.

If you’re an expereinced greylisting guru or just getting started and want to be part of the dialog, email me at thebaum@splunk.com.