Lessons from a lost decade: Developing for a disposable Web

Ten years ago this week, the tech-heavy Nasdaq stock exchange reached its all-time peak of 5,048.52. It was the height of the dotcom boom, and thousands of developers were busy writing millions of lines of code for applications that would — they hoped — revolutionize computing and become mainstays of the Web. So where is all that code now?

Building applications for today’s Web is a lot different than it was in those early days. Gone are the costly, monolithic e-commerce engines and content management systems written in C/C++, largely replaced by open source alternatives in more agile languages, such as Java and PHP. Meanwhile, once-promising technologies such as Napster and server push have joined Flooz, Geocities, and Kozmo.com as so much scrap on the side of the Information Superhighway. Remember ColdFusion?

[ Keep up to date on the latest developments in and insights on software development with InfoWorld’s Developer World newsletter. ]

As in any industry, much of the change has been evolutionary. But when platforms evolve, they often outpace the applications that build on them. Java, for example, is one of the more important technologies to come out of the dotcom era, but modern Java EE applications bear little resemblance to early, servlet-based code. Similarly, code compiled for a dotcom vintage Linux probably won’t run on a modern server. Even the Apache Web server has undergone a major architectural redesign since the Nasdaq peaked in 2000.

What this means is that — in contrast to enterprise applications, which often keep humming along on decades-old legacy servers — most of the code written for the early Web has simply disappeared. It’s been retired, rewritten, or taken out of service when the company that hosted it folded.

On the surface, then, the Web sounds like a massive productivity sink. All that work, all those hours to develop all those applications, and what do we have to show for it? But rather than crying over spilt milk, if developers learn the lessons of the last 10 years, they can gain a better understanding of how best to do business in the modern Web era.

First lesson: Emphasize agility First, it seems clear that a successful Web project must be agile. “Ship early, ship often” is an oft-cited mantra from the open source world, and it works for Web apps, too. If we accept that real-world Web code seldom endures as a monument to its developers, there’s no reason to fall victim to overengineering in the early phases of a Web project. As Joel Spolsky says, “Shipping is a feature. A really important feature. Your product must have it.” For startups in particular, the ability to bring a product to market quickly often outweighs loftier engineering considerations.

That doesn’t mean you should skimp in areas that could affect security or your user’s privacy. In today’s business climate, those kinds of missteps can be fatal. But scalability, for example, is one area that’s often given too much focus in the early days of a Web project. In some cases it can be more prudent simply to plan to rewrite your application for scalability at a later date, rather than delaying the first version while developers dicker over architectural issues.

In recent years we’ve seen this fast-and-loose approach work even for high-profile Web sites. For example, now that it has established itself and has experienced some scaling issues, Twitter has begun phasing out Ruby code for a new platform written in Scala, a multiparadigm programming language that runs on the Java virtual machine.

Second, development shops that measure employee productivity by lines of code written or some similar metric are doing it all wrong. If we assume that code created for Web applications has a short shelf life, the best way to avoid wasting employee productivity is to write less code, not more.

Today’s developers have access to a wide range of libraries, frameworks, servers, and tools with which to assemble Web applications. They should use them. As Web app development increasingly becomes the process of assembling these prebuilt components, developers are able to focus on the business logic that drives their applications, rather than wasting time writing and rewriting aspects of their software that don’t differentiate them from their competitors.

Code is expendable; developers aren’t Finally, companies should evaluate their software investments for what they are: short-term assets. The intellectual property value of any one version of a custom Web application is minimal. Far more valuable are the developers who built it, because they’re the ones who will be asked to rewrite it in response to the ever-changing business and technology landscapes.

Google and Microsoft, among others, understand the value of hiring, retaining, and incentivizing good developers, and they do it well. It’s a shame that so many companies promote their best programming brains into needless middle-management positions or allow them to leave the field entirely, rather than retaining them for their value as developers.

Perhaps the biggest loss of the dotcom era is that, in the aftermath of the bubble, fewer American students than ever are pursuing computer science or information technology degrees. That’s a shame, because while few Web sites of the last 10 years may have stood the test of time, the demand for Web software has never been greater.

Web applications, like all software, must be constantly refreshed as technology advances and business needs change. But that doesn’t mean programming must be a Sisyphean task. For all the software that has fallen by the wayside, today’s Web is far more powerful and advanced than the one of 10 years ago. If we want that progress to continue, it’s up to employers to convince young workers that software development remains one of our fastest-paced and most vital industries.

This article, “Lessons from a lost decade: Developing for a disposable Web,” was originally published at InfoWorld.com. Read more of Neil McAllister’s Fatal Exception blog and follow the latest developments in software development at InfoWorld.com.