Scripting Web services

I’ve just returned from InfoWorld’s conference, Next-Generation Web Services II: The Applications. One of the most memorable talks, for me, was given by Cape Clear’s Annrai O’Toole. He predicts a rosy future for basic application integration using Web services.

The secret sauce is XML’s ability to represent data neutrally. This, he said, drives toward zero the syntactic cost of integration — which has been the major EAI bottleneck. But O’Toole was much less sanguine about business process integration. What needs to be represented here in XML (and then continously and rapidly modified) are not just data structures, but dynamic systems with control flow, parallelism, complex dependencies, and exceptions. O’Toole wonders if it’s even possible to declaratively describe a business process. I share his skepticism.

“We’ve all seen the demos where the stick man does a yes/no analysis of the purchase order and hands it to the other stick man,” O’Toole joked. “That lasts five minutes, then you throw it away and start coding.” He thinks we’ll still need to do a lot of scripting to glue things together. I vehemently agree! Thanks, Annrai, for putting this important topic back on the table.

Almost a decade ago, I began to chronicle the evolution of what I’ve come to call the “components and glue” style of software development. Components are packages of reusable code. CORBA calls them objects; Microsoft calls them VBX/OCX/ActiveX and now .Net components; Java calls them beans. Glue is the scripting technology that rapidly assembles components into the useful arrangements we call applications. Visual Basic, JavaScript, Perl, and Python are glue languages.

These dynamic and flexibly typed or typeless languages are radically more productive than the strongly typed languages typically used to build components. The idea was that progammers would be divided into two groups. A small and elite group of systems programmers, working more slowly in compiled languages, would create libraries of robust, high-performance components. A much larger group of less-skilled application programmers, working rapidly in scripting languages, would use those components to crank out applications.

This arrangement has served us rather well, and will continue to do so in the Web services era. SOAP interfaces can now be easily or even automatically wrapped around every kind of legacy component. And every major scripting language can use those SOAP interfaces. In theory, the stage is set for a burst of productivity.

In practice, though, I worry that the true power of scripting is not yet fully understood or appreciated. Java isn’t a scripting language. Neither is C# or VB .Net, something the VB6/VBScript crowd are loudly complaining about. Their protests should not be lightly dismissed as the whining of a disenfranchised caste of untouchables. They know the loss of dynamic language features is going to hurt their productivity, and they’re right to holler about it.

In the early days of the Web, Perl was called “the duct tape of the Internet”; many people thought that the terms “CGI” and “Perl script” were synonymous. I’ve been wildly productive in Perl for many years, and more recently in Python too, and I fully expect to remain so as Web services proliferate. When the SOAP APIs to Google and then Amazon emerged, a Python programmer named Mark Pilgrim immediately released Python wrappers for them. Like many others, I’ve used Mark’s wrappers to whip up simple applications that called those Google and Amazon services — it’s much more productive than hitting those services with Java or .Net code.

Note that while we normally think of scripting languages as consumers of services, you can also produce services using these languages — with the same hallmark ease and productivity. You might not want to field a Perl- or Python-based Web service under heavy transactional load (though, in fact, there are ways to do it), but not every Web service requires the full panoply of J2EE or .Net middleware. Real business process integration will mean deploying a mixture of heavy, medium, and lightweight services in the cloud, on the intranet, and even (in peer-to-peer mode) on the desktop. Modern scripting languages, the duct tape of the Internet, are SOAP-capable and stand ready to deliver.