OS X Security: How I became a spam kingpin, went legit and turned detective

I’m picking up the first-person account of a Leopard Server root exploit where it left off from my preceding post.

I did slam the door on the ClamAV exploiter, and close observation for a couple of weeks allayed my concerns that any lasting hole had been blown in my OS X Leopard Server’s security. I felt quite pleased with myself. My mail server was back on-line and healthy, and days of backed-up e-mails, including the requisite quantity of spam, started streaming in. I was a happy camper in April.

Then came May, which has been an unkind month thus far. Walking past my Xserve one night, I noticed that the CPU activity lights were pegged when the server should have been idle. Eight cores, burning rubber doing nothing? I went to the Activity Monitor GUI, and then to top from the command line. Neither identified a process responsible for sucking up all of my Xserve’s CPU cores. I rebooted, and the problem seemed to go away. But after about ten minutes, CPU utilization began climbing. I disabled Postfix and rebooted again with the same pattern.

It was at this point that I knew I was under attack. It’s at this point that a sensible person like you would pull his WAN cable. But I, in addition to using Xserve as my 24/7 server, use it to unravel mysteries that might make for interesting copy, even if it means feeding my limbs to wolves in the process. I care that much.

Looking at Activity Monitor, top and ps again, I noticed that there were five sshd (secure shell daemon) processes running whose CPU times (total time a process spends occupying a CPU) nearly kept pace with the system’s uptime. I often keep multiple ssh sessions going at once from a couple of machines, so except for the CPU time, it was hard to see five sshd instances as unusual. Until, that is, I used lsof to find out which files each of the sshd processes had open. I found socket connections open to Russia, China, Poland, Sweden, and Italy. My Xserve seemed to be on a promiscuous world tour.

I blew away the sshd processes, including my own (coming in through Remote Desktop instead), and I used Server Admin to disable ssh. The CPU VU meters dropped to minimal. Well, that was easy. “Too easy,” I said in my noir gumshoe manner.

Every so often, ps would show launchproxy, sshd -i or both in the process list, but they’d vanish before I could lsof them. This was a job for Little Snitch. Little Snitch traces outbound socket connections by application, source IP and destination IP. Its killer feature is that it keeps a list of the last several connections. Watching Little Snitch, I could see launchproxy, then sshd pop up from connections from mostly offshore domains. The sockets would close within a couple of seconds of opening. However, Little Snitch couldn’t tell me what these programs were doing. Was spam still getting out?

It was time to move on to tcpdump, which confirmed that yes, my Xserve was making outbound connections on port 25. Instead of a flood of connections from standing sshd instances, each hit of launchproxy and sshd from afar seemed to squeak out one message. The frequency of connections was low during the day, but spiked enormously at night in my time zone. tcpdump -A -s 0 didn’t show the text of any of the messages, which I found odd.

Throughout all of this, I’ve been checking blacklists to see if my IPs were flagged for spamming. I haven’t, knock wood. As I expected, my server passed a variety of on-line open relay tests.

I’m quite sure that the initial attacker messed with my ssh server keys or my server’s default certificate. Beyond that, I still have a lot of work to do. I’ve already done a clean install (I wasn’t given an option to upgrade or archive) of OS X Server Leopard to a new volume, with no WAN services on line yet. I’ll copy configuration files, very carefully, to recover services like DNS, while I expect that I’ll rebuild my mail server from scratch. Meanwhile, I’ll keep scratching around to look for signs that might help you in diagnosing a similar attack.

When I finally do put ssh back on-line, it will be authorized only for a non-privileged user.