Java 2’s Verifier becomes confused by German student’s security attack

April 9, 1999 — In early March, German graduate student Karsten Sohr at the University of Marburg discovered a serious security flaw in the Java 2 Virtual Machine (VM). Together with Princeton University’s Secure Internet Programming Team and researchers at Reliable Software Technologies (RST), Sohr developed two attack programs, tested them on several diverse Java platforms, and responsibly reported the flaw to vendors.

The flaw, which attacks Java 2’s Verifier security component, serves to emphasize that even systems designed specifically with security in mind can still fall prey to subtle, clever attacks.

This latest flaw fits a global pattern evident in the chronology of Java’s security holes. That is, discovery of Java holes not surprisingly follows on the heels of new releases, which add complexity and new functionality to the Java language. (For a complete chronology of Java security holes resulting in attack applets, see Chapter 5 of my book, Securing Java in Resources.)

Booby trap: sneaking past the Verifier

The Sohr flaw is an example of a “type-confusion attack” — the kind of attack accounting for a majority of serious Java security flaws. This type of security attack exploits Java’s language-based security model that depends on enforcing type safety. In normal situations, type-safety enforcement ensures that the VM always keeps track of what objects belong to what classes, which in turn prevents programs from accessing memory in inappropriate ways. (For more, see Chapter 2 of Securing Java in Resources.)

To accomplish this, the Java 2 security architecture relies on the Verifier security component to perform static type checks of the objects’ classes. In general, static type checking is a good security measure, and Java byte code was carefully crafted to allow it (to the extent that static checking is possible).

However, if the Verifier slips up in its type-safety enforcement job, all bets are off for Java security. Whenever the VM becomes confused about types, language-based security encapsulation and enforcement go out the window.

This is precisely what happens if the new flaw is exploited. A malevolent soul — the attacker — could exploit the flaw by booby-trapping a Web page with an attack applet so that when a victim surfs to the booby-trapped Web site, the victim’s browser requests the attack applet’s class file. When the attack applet subsequently arrives, the Verifier performs its type-safety checks.

The attacker has crafted the class file to confuse the Verifier by generating an Exception in a particular situation (I’ll leave out exact details since so many browsers still have this security flaw). As a result, the Verifier becomes confused enough that it is possible to sneak unverified code into the runtime environment. This can be virtually any code the attacker wants. That is all a knowledgeable attacker needs to completely destroy Java’s security model.

An attacker carrying out a successful type-confusion attack can do whatever the attacker wants to the victim’s machine, including reading and deleting files, installing viruses, and snooping for any data and activities on the victim’s machine. Such an attack can be carried out in a sneaky enough manner that the victim will probably not even know that Java has been run or that they have been compromised. This is serious stuff.

One awful scenario involves a cracker booby-trapping a hacked Web page. Since literally hundreds of thousands of people may flock to a hacked site to check out the graffiti, as happened when the Department of Justice site was hacked last year, placing an attack applet on a hacked site would be most effective.

Good guys identify affected platforms

Soon after discovery of the problem, Sohr contacted Professor Ed Felten at Princeton University to determine the flaw’s seriousness and to decide how to approach Java vendors. Together with researchers at RST, two pieces of attack code were devised (one application and one applet) and used to test several Java VM implementations.

Attack application results

Viren Shah of RST reports that the application-based attack worked against the following Java platforms:

• JDK 1.1.5 (Solaris)
• JDK 1.2beta4 (Solaris)
• JDK 1.1.6 (Solaris)
• JDK 1.1.7 (FreeBSD)
• JDK 1.2 (NT)
• JDK 1.1.6 (NT)
• Symantec Visual Cafe Version 3

Interestingly, the attack failed when tested against Microsoft Visual J++ 6.0. (This is one time we can all be glad that “write once, run anywhere” is not always true.)

Attack applet results

The applet-based version of the attack succeeded against the following browsers:

• Netscape 4.5 (FreeBSD)
• Netscape 4.5 (NT)
• Netscape 4.05 (NT)
• Netscape 4.02 (Solaris)
• Netscape 4.07 (Linux)

Note: Microsoft Internet Explorer’s version of the Java VM was not susceptible to the applet-based attack.

Vendors respond promptly

The research team notified Sun and Netscape of the flaw in mid-March, well before news of the flaw was made public. This allowed them a head start in fixing the problem. Dr. Li Gong, chief Java security architect at Sun, told CNET, “The impact should be fairly low.”

Because responsible good guys discovered the flaw, Gong’s statement is accurate. Had crackers bent on destruction discovered it first, the story could have been completely different. Imagine, for example, a cross-platform Melissa virus that attacks not just Outlook, but any number of different mailers. Exploiting the new flaw in such a way is well within the realm of possibility.