Negative developer survey stirs controversy

On November 10, Portland, OR-based Market Decisions Corp. (www.mdcresearch.com) issued a press release highlighting the results of a survey based on interviews with 151 Java developers. A few days later, the pointedly negative conclusions in the press release were uncritically published by the respected trade newsweekly ComputerWorld as a news story. Controversy swirled when IBM’s Java Community Web site verbally slapped the Market Decisions’ survey in its “Daily Grounds” editorial later that week, with its not so subtle inference that the report was anything but an “independent research study.”

So just what is the real story? The press release issued by Market Decisions, entitled “Developers Express Widespread Dissatisfaction With Java — Two-Thirds of Java Developers Losing Faith in ‘Write Once, Run Anywhere'”, presented five major findings and three pages of quotations expressing dissatisfaction or frustration with Java. Here are the major findings presented in the release:

• 67 percent said having to “test [Java code] on multiple virtual machines and platforms was an issue affecting their Java development plans”. 22 percent said they were “satisfied with Java as a cross-platform development language”.

• 56 percent were concerned by “incompatibilities between JDK 1.02 and JDK 1.1.”

• 47 percent said the “lack of open standards for Java” was a “problem.”

• 45 percent noted the “poor performance of Java compared with programs written in other languages.”

• 39 percent said they are “merely experimenting with Java or [are] in the early planning stages of a project that uses Java,” and the median — mid-point of the sample — time spent developing in Java was 9 percent of their workday.

The survey sample was culled from three sources, said Bill Svendsen, a statistician and senior partner at Market Decisions. Approximately half of the sample came from randomly dialed US businesses, where interviewers asked for software development and probed until they found someone who admitted to developing software written in Java. About 40 percent of the sample came from randomly selected JavaOne attendees. And the remainder — about 10 percent — came from attendees of some other developers conference, Svendsen said.

Survey respondents reported targeting Java software development at the following platforms:

      Windows 95/NT      87%
      Solaris/Sun OS     36%
      Macintosh          31%
      Windows 3.x        18%
      AIX                17%
      NetWare            11%
      OS/2               11%

Other platforms were mentioned by less than 10 percent of those interviewed.

Svendsen characterized the survey as qualitative and not designed to project what the industry would be doing with Java. Using the traditional 95 percent confidence level, any projection would have a margin of error of plus or minus 8 percent, he said.

So what’s the controversy?

The Daily Grounds editorial at IBM’s Web site suggested that the survey data may have been rigged, raised questions on whether the sample was taken from a list of Microsoft Windows developers, and noted that, “To our shock and amazement, Market Decisions Corporation, the originator of the report, has done no less than three ‘independent’ reports in the last couple of months on the popularity of Microsoft Internet Explorer 4.0, Microsoft Visual J++, and Microsoft Visual InterDev Web Solutions Kit.” The apparent implication: Microsoft had sponsored the survey.

The editorial also criticized the anonymity of the client that paid for the survey and the timing of the release, coming as the ISO voting process on whether Sun could submit Java as an ISO standard was coming to a close.

Market Decisions’ Svendsen said he was surprised by the media response to the release, and defended the survey findings as valid.

In clarifying some issues about the validity of the survey data, Svendsen’s remarks raised other questions about the objectivity of the data’s presentation in the press release. He said the clients — more than one — told Market Decisions to issue the press release, that 99 percent of the company’s research never is disclosed publicly, and that his firm never makes any recommendation on research disclosure. In follow-up questions, Svendsen did not respond to requests to clarify whether the clients were involved in writing or editing the release.

Statements in the release such as, “Java developers are most concerned with Java’s inability to deliver on its purported ‘write once, run anywhere’ promise,” and the press release’s “losing faith” subtitle, imply far more than the data warrants.

Quotations from interviews within the body of the release present a starkly negative perspective — even on issues where less than half of the sample expressed concern. On standards: “I don’t trust Sun to handle it correctly.” On performance: “We cannot recommend Java highly.”

Ron Rappaport, an analyst that tracks Java development for Zona Research, said he has little doubt developers have their complaints, but that does not mean they view Java as a failure.

“Java was rolled out with great promise in 1995,” said Rappaport. “You have a little wine, you have a little cheese, you throw a little party, and everybody expects that promise to be fulfilled the next day — the next week if they are patient. That simply is not the way technologies mature and evolve. Everything happens fast with the Internet, but it still takes a little time for an entirely new language and computing infrastructure to mature.”

Zona’s own survey of 279 Java developers working at firms with more than 250 computer users assessed developer complaints. The June 1997 survey, released in October, asked developers what they saw as the greatest obstacle for Java. Fifty percent expressed concern about migrating applications across all platforms; 43 percent said performance was a serious issue.

That said, 32 percent of the Zona survey respondents said Java was being deployed top-down, as a strategic technology.

“For a technology as new as Java, to have a third of a survey [respondents] say they are deploying it strategically, from the board room — that is both significant and testimony to the impact that Java has had on development. That is extremely impressive,” Rappaport said.

For Scott Hebner, manager of IBM Software Group’s application development marketing, the key is keeping constructive developer feedback in perspective. Hebner, who was not connected with IBM’s Web site editorial on the Market Decision’s survey, said you’re bound to get feedback from technical people when you ask them what is wrong with something.

“Java is the best thing going for building networked applications that must support a heterogeneous environment. It is not 100 percent perfect, but no one should expect it to be. That is what 100% Pure Java is pushing to achieve — getting everybody moving down a common path,” said Hebner, who points to the Summer of 1998 as the threshold for having a fairly mature version of Java.

“We are already seeing big islands of problems being fixed that will in turn merge and become even bigger continents,” Hebner said. “Today you can create code using VisualAge for Java on Windows NT and run that code on OS/390. We are working with Sun and with Netscape and we are getting closer to a unified networked environment based upon Java. In my own little world here at IBM we have VisualAge for Java, NetObjects Fusion, BeanMachine, Netscape browser, VisualAge Generator, and VisualAge SmallTalk. They all support Java. You can truly move a piece of Java code and a JavaBean through all these tools and add value. That was impossible to do before Java.”

Hebner argues that Java is well-suited for certain tasks, and that criticizing it for not yet delivering the panacea of total platform transparency misses the big picture. He notes five key tasks for which Java is a superior development environment:

• Building distributed, network-based applications, where more of the code resides on the server and it is downloaded on demand.

• Helping people integrate heterogeneous environments — platforms, disparate tools, deployment environments.

• Extending the value and the reach of existing core business applications — getting to employees better, building business partner access.

• Enabling cross-platform capabilities — to build an application for more than one platform, and to use common tools, and to leverage common skills.

• Being a better C++.

“When you think about what you can do today with Java — even with its inconsistencies — compared with two or three years ago, most people are very impressed. Progress has been extremely significant,” Hebner said.

Conclusion

It is important to clarify time frames and perspectives when critiquing Java. Are you looking at past experience, anticipated near-term experience based upon the latest tools, or are you looking out six to twelve months? Love ’em or loath ’em, the one thing you have to credit Java supporters with is pushing the technology along rapidly — perhaps even too rapidly.

Developers in the MDC survey complained about the need to test on multiple platforms (67%), and platform incompatibilities (56%) such as JDK 1.02 versus JDK 1.1. Version 1.1 was a major leap forward for Java, revamping AWT including a fundamental change to the event model, and introducing the JavaBeans component architecture. As JDK 1.1 tools are now becoming available, attention is focused on post-1.1 initiatives such as Enterprise Java and the emerging new foundation classes.

Java remains a rapidly moving target because so much energy is being put behind it by major players — Sun, IBM, Netscape, Oracle, and yes, even Microsoft. Developer expectations are very high in part because much has been promised, but also because so much has been achieved in such a short space of time.

In such a climate, few doubt that developers can easily list Java’s faults when invited to do so by researchers. That is a far different discussion for the developer community than the one touted by MDC’s press release — that developers are losing faith in Java. Research from the Gartner Group, Forrester Research, and the Zona Research data cited above, underscore that fact.