Active intermediaries can halt the finger-pointing

Most people agree that two unifying principles make Unix great: Everything looks like a file, including programs and devices, and everything can be pipelined by connecting the output of one thing to the input of another. These principles begat a philosophy: Make simple tools that do specific tasks well, and solve complex problems by assembling the simple tools like beads on a string.

This worked well for decades, but now there’s a major fly in the ointment. The data passing among the tools is not self-describing. If you use Grep to search a log file for entries on a given date and pipe the results to a report writer that grabs the third and fifth fields of each selected line of the report, there is no way for a filter inserted between the two to know that fields are separated by tabs or that the third field represents a discount and the fifth a total price.

From 50,000 feet, it’s fair to say that XML Web services do nothing more, and nothing less, than remove that fly from the ointment. The XML documents exchanged among Web services are self-describing. A discount field will be tagged as such. Any observer watching the data flow can pick out, examine, report on, or even modify the discount field with little (if any) special knowledge of the data. That intermediary might be a person running an XML-savvy tool such as InfoPath, or it might be a piece of software that intercepts and transforms the data.

The nascent Web services industry has so far focused mainly on the technical implications of these “active intermediaries.” They do make it vastly easier to integrate systems that pass around packets of self-describing data. But the reasons for this go beyond the regularity of XML data and the ubiquity of tools that can parse, search, and transform it. XML data flows fundamentally alter the political landscape of IT, shifting the locus of control away from the service endpoints and into the fabric of the network itself.

Closed systems that use proprietary APIs and speak binary protocols are a recipe for finger-pointing. “I can’t adjust the discount until Tweedledum upgrades the purchasing module,” says Tweedledee. “Contrariwise,” says Tweedledee, “I don’t control that logic. My hands are tied.”

We’ve all been on both sides of this dispute, with no Alice to point out that we are only fighting over a rattle. Web services, however, can take us through the looking glass, ending the blame games to reveal the truth.

Frank Grossman, co-founder of Mindreef, likes to talk about how his company’s SOAP debugger, SOAPscope, can produce such revelations. SOAPscope is an active intermediary that can inspect and modify a SOAP transaction in flight. “When you can show the people on both ends that the XML packet isn’t what it should be, the finger-pointing just stops,” Grossman says. At last year’s Emerging Technologies conference, KnowNow co-founder Rohit Khare joked that the real integration challenge lies at “Layers 8 and 9 of the OSI stack: economic and political” (see “Connecting with Web services”).

Of course, the same architecture that can take the politics out of the Tweedledee/Tweedledum conversation can also put politics back in. Active intermediation is the ideal way to monitor and enforce a corporate or federal policy.

To make that happen, the services we want to monitor and mediate must be able to speak XML. That’s the easy part. The hard part will be trusting the intermediaries.

We know how to do it: Use familiar PKI methods to secure channels and documents, and use newer techniques — XML DSig (digital signature) and XML Encryption, for example — to secure elements within documents. But the devil is still in the PKI details.