Can HotSpot jumpstart your Java applications?

May 20, 1999 — Last month Sun introduced the Java HotSpot Performance Engine, a jumpstart technology for Sun’s Java 2 platform that the company says will make conventional just-in-time compilers (JITs) a thing of the past. By using an “adaptive” compiler and new techniques for garbage collection and mutithreading, says Sun, HotSpot can improve application performance by up to 100 percent. Still, experts say those improvements aren’t across the board, and HotSpot still lags when it comes to certain applications and hardware configurations. In other words, there are places where HotSpot is hot … and places where it is not.

“HotSpot is a turbo-charger for the Java 2 platform,” said Sun Java Software Division President Alan Baratz, speaking at the product’s launch in Paris earlier this month. “It can improve performance of Java applications by two times and is blowing away the competition in speed and scalability benchmarks.”

The hot spot

The key to HotSpot, says Sun, is adaptive optimization, which provides a significant performance gain over JVMs that use just-in-time compilers. A JIT translates Java bytecode into machine language on the fly, compiling methods only the first time they’re executed and using the resulting machine code in subsequent method calls, rather than reinterpreting the bytecode. This enables a significant performance improvement over earlier Java platforms. But JITs have limitations. For example, complex graphics processing can slow things to a grinding halt.

HotSpot is Sun’s solution to the problems experienced by JITs — at least on the server side. HotSpot can optimize the performance of many server applications, like databases, that execute the same task again and again, using the same objects and calling the same methods. HotSpot identifies these code “hot spots” in a profiling process that begins when the application is launched and continues dynamically to identify the most frequently executed code and concentrate on compiling it. (For a more detailed explanation of this, see the “How compilers and interpreters work” section of Eric Armstrong’s March 1998 JavaWorld article on HotSpot.)

Garbage collection and multithreading

In addition to dynamic compilation, HotSpot targets two other key areas instrumental in application performance. Sun estimates that 40 percent of hardware resources used by a typical Java application are devoted to multithreading (whereby multiple I/O data streams are handled simultaneously) and garbage collection (which frees memory space occupied by objects that are unlikely to be used again).

According to Sun, HotSpot improves on conventional garbage collection in several ways:

• Generational collection: HotSpot’s “generational” garbage collection relies on the fact that the large majority of objects in most programs are short-lived, stacking new objects together (an efficient allocation process) in a “nursery,” then relocating only the surviving objects once the nursery overflows.

• Improved ‘old-object’ collection: When memory becomes sparse, long-lived objects’ areas also have to be cleaned up. Live objects in these areas are scanned by the collector and the gaps left by dead objects are compacted (rather than collecting live objects into a new list). This reduces memory fragmentation and increases efficiency by eliminating searches on new lists.

• Pauseless garbage collection: Application pauses experienced by the user during old-object collection are now spread out over smaller intervals, each no longer than 10 milliseconds. This type of garbage collection attempts to improve user experience in applications where smooth sailing is critical, such as in games.

HotSpot also attempts to eliminate performance bottlenecks caused by multithreading. Sun claims the new technology incorporated into the performance engine can make “synchronization performance so fast that it is not a significant performance issue for the vast majority of real-world programs,” according to one of the company’s whitepapers.

The speedometers

By some industry measurements, the new features described above seem to work quite well. Preliminary (as yet unapproved by the Standard Performance Evaluation Corp.) results indicate HotSpot has indeed earned Sun bragging rights: it’s the top-performing JVM according to the SPECjvm98 benchmark, and its VolanoMark benchmark figures are even higher. (For a detailed explanation of the VolanoMark benchmark, see the March 1999 JavaWorld exclusive coverage of the VolanoMark benchmark report, linked in the Resources section below.)

Sun reported a SPECjvm98 result of 31.1 for HotSpot on Windows NT at 450 MHz — an impressive 90 percent improvement over the Java 2 SDK running with the Symantec 3.00 JIT compiler on the classic JVM and a 112 percent improvement over JDK 1.1.7 with the same JIT. (See Figure 1 below for a comparison with other JVMs on a Pentium II/350 MHz machine.)

On the JVM front, HotSpot’s SPECjvm98 result of 25.3 on a Pentium II also places Sun slightly ahead of IBM, which previously held the lead with a result of 22.6.

Although HotSpot’s preliminary performance on the VolanoMark tests (which measure JVM speeds as a function of the number of concurrent connections) showed JVM speeds still lagging behind some results posted by Big Blue, HotSpot proved to be highly scalable, a drastic improvement over previous HotSpot results for the Windows platform.

On SPARC/Solaris, HotSpot showed a stunning capacity for maintaining an almost even throughput value for concurrent connection levels ranging from 200 to 2000. That is, as Figure 3 indicates, HotSpot’s ability to handle messages didn’t degrade substantially as a function of the number of concurrent chat sessions.

Sun is calling its VolanoMark results “best-of-industry” despite some individual results that are slower than those for other JVMs. Still, Sun’s JVM performance numbers on recent, published VolanoMark tests are a milestone, says Volano LLC founder John Neffenger, considering that until recently, Sun’s JDK 1.2 had failed to outperform Microsoft’s SDK 3.2 on Windows.

Servers and clients

Number grubbing aside, everyone admits the issue boils down to one question: What will HotSpot do for your application? For some apps, HotSpot can double performance; for others it won’t do much at all.

“HotSpot sped our [Java database] StreamStore up by from 80 to 100 percent,” said Bluestream Software CTO Jim Tivy. “StreamStore requires a lot of repetitive operations; the process of inserting a thousand records or reading 10,000 [records] takes place time and time again, and the act of reading 10,000 records is the same as [the act of] reading 1. [HotSpot] takes a hit in the first few operations while it’s still profiling the app, but begins to warm up after that.”

Sun agrees that databases are ideal candidates for the HotSpot performance engine, saying in its HotSpot literature that server-side applications that operate for long periods of time allow “the performance engine’s adaptive compiler ample opportunity to analyze the program during runtime and make the smartest optimizations.” Additionally, said Tivy, distributing garbage collection over smaller intervals gives the user the impression of more consistent database availability.

Less spectacular client-side improvements are also possible, depending on the application. Cerebellum creates a Java- and CORBA-based application with a server-side component that calls multiple databases with single queries submitted by users through a client-side GUI; company officials claim HotSpot improved application performance by about 20 percent.

Cerebellum saw server-side improvements that it attributed to HotSpot’s being well suited to the design of the company’s product, which repeatedly calls certain code segments for almost all of its operations. On the client side, however, performance improvements are attributed to HotSpot’s generational garbage collection, according to Cerebellum CTO Todd Olsen.

“Enhanced garbage collection helped the most with Cerebellum,” said Olsen. “In [our] highly graphic interface, the performance degradation is caused by a lot of object issues … before HotSpot, you could really tell when the application was doing garbage collection.”

In other cases, however, clients aren’t likely to see much in the way of improvement, according to Sun. Whereas 90 percent of the average server-side application’s time is spent executing Java bytecode, client-side windowing and GUI operations often rely heavily on native code libraries and Java 2D APIs. Application performance in these cases is influenced heavily by the speed at which the OS executes calls to classes in the native libraries — a process that bypasses the JVM. Sun estimates that a typical client spends some 30 percent of its time waiting for native windowing code, as illustrated in Figure 4.

“On a typical client, 60 to 70 percent of the app’s time can be spent in the native libraries and about 25 percent [is spent] executing Java bytecodes,” said David Stoutamire, engineering manager for HotSpot compilation. “Most of the server apps we see are compute-intensive and that’s where HotSpot can speed things up. That’s a lot different from drawing polygons on a screen.”

With other server-side uses of Java, such as applets downloaded through a browser, code can change on the fly in ways that HotSpot simply can’t anticipate, said HotSpot Product Manager Bill Pataky. In addition, Sun points to the underlying operating system and hardware as a significant parameter of application speeds. That particular line of defense was hauled out periodically to defend Java 2’s rather poor performance in early VolanoMark scalability tests.

A free lunch

According to Sun, HotSpot is something of a free lunch in more ways than one. For starters, HotSpot’s memory footprint is comparable to that of Java 2, and HotSpot’s optimizer is small in comparison to the typical Java application, said Sun’s Stoutamire. Additionally, HotSpot’s memory model saves space by using compact, “handleless” objects. The Java 2 SDK uses handles to represent object references, which makes object relocation easier but requires more memory. HotSpot uses direct pointers as object references, updating the pointers during garbage collection.

HotSpot for Windows is available for download free of charge. HotSpot for SPARC will be available sometime in June.

Sun has been evasive about how exactly HotSpot is to be licensed. Company representatives will say only that Sun is investigating the possibility of placing parts of HotSpot in the Sun Community Source Lincensing program, and will form a more specific policy in the next six to nine months.

Sun says its next version of the performance engine (HotSpot 2.0) will improve Java 2 performance by another 30 percent. HotSpot 2.0 is due out later this summer, accompanied, no doubt, by more fanfare and a flurry of new benchmarks.