Unix: Rootkit detection and prevention

Last week’s posting was a reminder of how stealthy and dangerous rootkits can be. They hide in your operating system and compromise your systems and your data as quietly as possible. It often turns out to be some odd behavior or system peculiarity that clues admins into the presence of a rootkit — if they’re lucky.

If you’ve got a rootkit on your system, what it’s going to do depends on the rootkit itself. It might be set up for keystroke logging or for giving a remote user control of the system any time they want. It might send spam or participate in a DDoS attacks. It might send your critical data to an external server, inject code into your web pages, or make your system unusable. And that’s just a start. Whatever it does, it isn’t good for you or your organization.

Most rootkits can survive reboots. Many go to extreme measure to resist detection not just by Unix admins, but by the tools that admins use to locate and remove them. In fact, the efforts of security vendors to detect rootkits and rootkit architects to avoid detection is something of an ongoing “arms race”. The threat landscape is continuously changing and what worked a few months ago might not work today.

Rootkits will often be specifically engineered to block security tools, especially those designed to detect and remove rootkits. Trying to remove a rootkit can be like trying to remove from your favorite shirt a stain that is designed to resist stain removers — except, of course, that you can actually see the stain on your shirt. What do you do if netstat, du, find, ps, ifconfig, inetd, killall, and lsof all lie to you? How would you even know?

Replacement executables only represent the simplest form of rootkits. These would be easy to detect by comparing checksums with a known good system, though you would need to trust your checksum executable. One option I’ve seen people use for this is mounting a CD with an md5 binary and using it to compare their checksums.

More sophisticated rootkits might:

•  
  • intercept and redirect system calls to a process that is running in memory
  • manipulate kernel objects — hiding processes, changing privileges, etc.
  • modify kernel data structures
  • add drivers to your system
  • use covert channels to communicate with external systems

Fortunately, there are many commercial and free tools that are designed to both detect and remove rootkits. Examples

include RootkitReveaker, GMER, Sophos Rootkit Scanner and Rootkit Hunter.

Some of the techniques that rootkit detectors might use include:

•  
  • using md5 checksums to compare with known good values
  • looking for file “signatures” based on byte patterns rather than checksums (these can lag)
  • looking for known files that might indicate a compromise
  • finding files that have been granted excessive permissions (such as setuid)
  • looking for hidden files in odd places
  • looking for pointers in memory in that refer to locations outside the expected ranges
  • comparing high level (user) views with hardware level views (files, registry keys)
  • for Windows systems, looking for registry discrepancies
  • looking at network traffic and comparing it what the system files acknowledge sending

The success of these techniques will depend on the sophistication of both the tool and the rootkit. A kernel mode rootkit can always alter the results of the commands you run trying to figure out what’s going on.

The best defense is, as you’d likely suspect, to follow the security guidelines that help keep the cyberscum from getting a root level foothold onto your system in the first place. These include:

•  
  • running a firewall
  • using good passwords
  • never sharing the root password
  • using sudo to give limited root privileges, but only as needed
  • exercising the principle of least privilege
  • keeping software up-to-date
  • reducing the attack surface of your system by not running any services you don’t absolutely need
  • knowing what software should be running on every system you administer
  • enabling secure communications links only — shut down everything else
  • patching regularly
  • monitoring log files
  • using an IDS

In next week’s post, we’ll look at some tools and techniques that you can use to help determine if you system might be compromised.

Read more of Sandra Henry-Stocker’s Unix as a Second Language blog and follow the latest IT news at ITworld, Twitter and Facebook.