Building SOA your way

A fault line runs beneath the groundswell that began a few years ago with XML Web services and continues today as SOA (service-oriented architecture). True, nearly everyone agrees that XML messaging is the right way to implement low-level, platform-agnostic services that can be composed into higher-level services that support enterprises business functions. Yet, here’s also a sense that the standards process has run amok.

IBM, Microsoft, and others have proposed so many Web services standards that a new collective noun had to be invented: WS-* (pronounced “WS star” or sometimes “WS splat”). The asterisk is a wild card that can stand for Addressing, Eventing, Policy, Routing, Reliability, ReliableMessaging, SecureConversation, Security, Transactions, Trust, and a frighteningly long list of other terms. Surveying this landscape, XML co-creator Tim Bray pronounced the WS-* stack “bloated, opaque, and insanely complex.”

It wasn’t always so. Simple forms of XML messaging were succeeding in the field long before any of these standards emerged. At InfoWorld’s SOA Executive Forum in May, Metratech CTO Jim Culbert described how his company’s service-oriented billing system worked back in the late 1990s. The messages exchanged among partners were modeled in XML and transported using HTTP with SSL encryption — the method still used for most secure Web services communication today. Seybold analyst Brenda Michelson, who was then chief architect at L.L. Bean, tells a similar story about that company’s early experience with Web services.

Two factors were prominent at the time. First, the Web offered a simple, pervasive integration framework, one later promoted to the status of architecture and assigned the label REST (Representational State Transfer). Second, XML provided a universal way to define services in terms of the data they produced or consumed, rather than in terms of the code that produced or consumed the data. In combination, these factors were — and still are — powerful enablers.

Cranking Up Complexity

How, then, did we arrive at WS-*, which Culbert and others say is a cart that’s gotten way ahead of its horse? One theory holds that the heavy-hitting vendors, working closely with key customers and partners, have ratcheted complexity up to a level that only they will be able to sustain. Because those specs are so far ahead of what most users need today, their development hasn’t been an organic process driven by well-known requirements.

Patrick Gannon, president and CEO of OASIS, the standards body now coordinating a number of the WS-* specifications, reluctantly agrees that users should have been more engaged from the beginning. “I wasn’t involved in creating those specs without formal user requirements on the table,” he says. “But I’m a pragmatist; the specs are there.”

Another view holds that industry heavyweights, who have paid their dues when it comes to security, transactions, and reliable messaging, are indeed qualified to translate their experience in these matters into the language of XML. TN Subramaniam, director of technology at RouteOne, which makes software that streamlines credit management applications on behalf of car dealers, learned that lesson the hard way. At one point he began drafting his own spec for single sign-on, only to abandon it when he discovered SAML, which his joint-venture partners enthusiastically adopted because all their identity management vendors — including Netegrity and Oblix — were supporting it.

“What are the chances,” Subramaniam asks, “that five architects meeting every other day will iron out all the possibilities, versus having a committee thinking it all through in great detail with all the vendors on board?”

It’s tempting to interpret the tension between these two perspectives as a replay of the cathedral and the bazaar — or perhaps instead, WS-Heavy and WS-Lite. In that dichotomy, WS-Heavy would refer to the security, reliability, and scalability that WS-* claims to deliver, whereas WS-Lite would mean the speed, simplicity, and agility that attract labels such as REST, AJAX, and RSS. None of the enterprise architects we interviewed for this story has pledged allegiance to either of these camps, though. They’re intensely pragmatic people who will do whatever it takes to get the job done, and it’s instructive to learn how they are — and are not — making use of Web services standards.

RouteOne: Securing Credit Checks

Although end-to-end SSL is often sufficient, RouteOne’s Subramaniam has two reasons to prefer the more granular approach enabled by WS-Security. First, it’s necessary to digitally sign the credit applications his application transmits, and to do so according to rules understood by service partners. WS-Security defines such rules, although admittedly, and unfortunately, too many of them. One method is to put the signed application into the body of the SOAP message; another is to use SOAP with attachments. In the end, there was no agreement among the service partners, so RouteOne uses both. That’s frustrating, but Subramamian would rather have two rules than none.

The second reason touches on one of the deep principles that motivates the design of the WS-* stack: pervasive intermediation. RouteOne is required to maintain meticulous audit logs and would prefer not to have to encrypt all of them. So it’s using DataPower’s XML router/accelerator to selectively encrypt only sensitive items such as gross pay and Social Security number. Because it’s a standards-based intermediary, the DataPower box can straightforwardly modify RouteOne’s XML message traffic in this way, and it could be swapped out for another appliance that did the same thing.

When services communicate directly, as many if not most still do, there’s no need to define the rules of engagement that enable service intermediation. Today’s most visible exemplars of WS-Lite — Amazon and eBay — use Web services in a point-to-point way. In that mode there’s not much difference between SOAP/WSDL APIs and REST APIs, so it’s not surprising that developers who work with these platforms overwhelmingly prefer the REST flavor. But when you do need to flow your XML traffic through intermediaries, SOAP and WSDL suddenly make a lot more sense.

Subramaniam is a pragmatist, however. Plain XML over HTTP, sans WSDL, also plays a role in RouteOne’s internal and external affairs. Because it’s a no-brainer to put a servlet interface onto an internal legacy system and pull XML data through it, that strategy is used where appropriate. Some of RouteOne’s external partners use the same approach, and because “they’re making money hand over fist” doing so, Subramaniam can’t mandate otherwise. Instead, RouteOne normalizes inbound traffic to SOAP and WSDL in order to enable its expected future use of BPEL (Business Process Execution Language) for service orchestration. Today, partners who don’t present SOAP and WSDL interfaces are not competitively disadvantaged. But the tipping point may not be far off.

RouteOne depends on both SAML and WS-Security, and Subramaniam wishes he could use a standard form of reliable messaging, too. “If I don’t send a message, we are losing money,” he says. Drawing inspiration from ebXML (e-business XML) and JMS (Java Message Service), he specified — and is now using with partners — a scheme that guarantees orderly and reliable delivery of messages. But he’d rather it were otherwise and hopes that OASIS will succeed in merging the two proposals it is now hosting: WS-Reliability and WS-ReliableMessaging. This duplication is “really, really bad,” Subramaniam says. “I wish we had a common spec so I could dump my stuff and just use it.”

Corillian: Point-to-Point Simplicity

Many service-oriented systems don’t require reliable messaging and, according to Scott Hanselman, chief architect at Corillian, his company’s banking middleware falls into that category. Corillian’s product, called Voyager, handles services touched indirectly by 25 percent of all users of online banking, Hanselman says. “But the only transaction they care about is the one at the host.” So he’s not worried about the merger of WS-Reliability and WS-ReliableMessaging. Although he does make use of WS-Security, he regards SSL as equally effective in most cases. That approach precludes routers and intermediaries, he admits, “but rarely do I use them, because nine times out of 10 we’re doing point-to-point messaging.”

He’s also dismissive of UDDI, the much-maligned standard for publishing directories of Web services. What about the argument that services not  found in the yellow pages won’t be reused? Hanselman doesn’t buy it. Finding services isn’t really a problem for developers, he says. Using them easily and effectively is. Imagining a fictional average developer named Mort, Hanselman opines that SOA will be a nonstarter until we can shield Mort from XML angle brackets and X.509 certificates. To that end, he thinks the most important standard is WSDL because it’s a tool-enabler.

Of course WSDL has earned its fair share of criticism, too. RouteOne’s Subramaniam thinks that the “goofy” complexity of WSDL 1.1 made it a ball and chain that SOA has had to drag around, and he hopes that the “much cleaner” WSDL 2.0 will lighten the load. Perhaps, Hanselman says, but “you can’t unring the bell.” Millions of Web services transactions ride on WSDL 1.1 and will for a long time to come. Using WSDL 1.1, Corillian was able to describe the objects, messages, and services at the core of Voyager and to bind those descriptions to internal machinery that doesn’t speak XML. As the need arose, the company created alternate bindings that enable customers to see the engine through a Web services lens. If WSDL 1.1 was an 80 percent solution, Hanselman thinks, then WSDL 2.0 might be a 90 percent solution, but either can deliver crucial leverage.

Ohio State: Securing Vital Signs

The most widely adopted of the advanced Web services standards is clearly WS-Security. Beyond that it’s hard to find practitioners who have worked with the more exotic beasts in the WS menagerie, but Furrukh Khan — who holds joint appointments in the colleges of engineering and medicine at the Ohio State University Medical Center and is broadly responsible for its medical IT — tells a fascinating story about his transition from basic to advanced Web services.

In this scenario, vital signs flowing from monitors are recorded in databases and are simultaneously delivered to smart clients that observe, replay, analyze, and annotate the streams of data. The streams must be delivered to a lot of clients reliably, securely, and in near real time.

A first implementation, based on Microsoft’s WSE (Web Services Extensions), made use of WS-Policy, which hasn’t yet found a home in a standards body but likely will soon. WS-Policy was used to declare the means of authentication to back-end databases — for example, to require X.509 certificates signed by a specified key — as well as the required payload signature and encryption.

The current implementation — based on the beta version of Microsoft’s Indigo, a Windows implementation of a stack of advanced Web service protocols — uses WS-ReliableMessaging to ensure orderly and reliable delivery of messages. And it uses WS-SecureConversation to optimize that secure, reliable channel for high-volume traffic.

Khan explains that WS-Security alone, in concert with WS-Policy, could not sustain near-real-time traffic. The protocol, which required frequent exchanges of credentials with the identity management system, was too chatty. WS-SecureConversation, which enables caching of credentials, streamlines the protocol. That, coupled with a feature of Indigo’s implementation of WS-ReliableMessaging that enables a router to broker a connection between two end points and then get out of the way, resulted in a massive scale-up.

“Before, with WSE, each router limited us to 300 clients,” Khan says. Indigo can support 638 clients per router, he adds, and with optimization, that many clients for each service running behind the router. “So if you keep on adding services, it scales linearly,” he says. The system currently supports more than 1,000 clients, all observing vital signs simultaneously every 30 seconds.

Reflecting on the transition from WSE to Indigo, Khan echoes Scott Hanselman’s point about shielding developers from XML. WSE handled the basic scenarios, he says, but beyond those, “we had to go into the schema and do all the angle brackets.” Thanks to Indigo’s higher level of abstraction, that problem vanished.

More broadly, Indigo made a harder problem — the appropriate use of Web services in concert with platform-native services and transports — tractable. “Behind each Web service there’s an MSMQ [Microsoft Message Queue] and an enterprise service,” Khan says. “In the Microsoft domain, enterprise services are completely different from Web services, MSMQ lives in its own world, and XML has its own toolset.” Different team members had to be experts in different disciplines; no one person could master them all. From Khan’s perspective, Indigo gives “Mort” the leverage he needs.

Providence: Enforcing Contracts

Providence Health Systems deploys what’s becoming a typical two-tiered SOA to support its clinical and business applications and its physician and patient portals. A set of coarse-grained services, which map closely to business processes, are woven from another set of more elemental services. Although some advanced standards are in use, such as WS-Security, Providence doesn’t deal with them directly. “We rely on our vendor’s implementation of the security stuff,” says Mike Reagin, vice president of development at Providence. The vendor in this case is Infravio, whose Web services management system provides the framework within which Providence deploys and manages its services.

Infravio implements UDDI, but Reagin says that, with relatively few services in play, directory lookup isn’t a big deal. Declaring and enforcing policies that control the use of those services, however, is a very big deal, as is monitoring service activity.

In Infravio’s model, services are provisioned as producer/consumer pairs, each of which is governed by a contract. The master patient index, for example, is a common service used by both the physician and patient portals but in slightly different ways. The patient’s health-plan member number, which appears in the patient portal, must be stripped from the physician portal. By creating separate WSDL interfaces for separate consumers, Infravio enables the common service to be reused rather than duplicated. This variation is achieved in a declarative way, rather than by writing code.

Providence’s SOA deployment is, for now, largely internal. Services feed its outward-facing portals but are not yet directly exposed to partners. That day will come, Reagin feels sure, and when it does, he expects that his use of the core standards, SOAP and WSDL, will enable more advanced scenarios: orchestration, reliable messaging, policy-governed security, and auditing. Which pieces of the WS-* stack will enable those scenarios? Reagin doesn’t lose sleep over the question. When the time comes, he’ll buy — rather than build — the needed infrastructure.

Pfizer: Trusting the Fabric

Security and reliable messaging are key requirements for the Pfizer Global Pharmaceuticals (PGP) group. The pharma giant’s SOA deployment meets those requirements with the help of Blue Titan’s Network Director, which manages PGP’s Web services traffic across the enterprise.

On the security front, Blue Titan’s “fabric” enforces a policy that routes requests through a DataPower intermediary for compliance auditing and through an Oblix system for authentication. Martin Brodbeck, PGP’s application architecture director, sees WS-Security as the integration framework for these activities. Although he doesn’t deal directly with related standards, such as WS-Policy or WS-Trust, Blue Titan does in fact support them.

It’s worth noting that a number of standards said to be “vendor-driven” are primarily of interest to vendors. For example, another architect interviewed for this story was hands-on with WS-Security but unaware that WS-Trust plays a role in his implementation. Why? The WS-Trust protocol is spoken only between his security broker, VordelDirector, and his identity provider, Entrust. The messages exchanged between his company and its Web services partner have nothing to do with WS-Trust, says Mark O’Neill, CTO of Vordel. “We and Entrust chose to use it because it’s a spec that we don’t have to work out ourselves,” he says. The WS-Security protocol used by the service end points and the WS-Trust protocol used by infrastructure components are “solving completely different problems — it just so happens that both involve specs that begin with WS.”

Along with security, reliable messaging is a key PGP concern. With various flavors of message-oriented middleware in play, along with multiple versions of some of these (such as JMS), the company values the Network Director RM’s capability of hiding the differences. Although that product’s support for WS-ReliableMessaging is not immediately relevant, PGP is evaluating Indigo, which natively supports the standard. “Blue Titan in concert with Indigo will make RM [reliable messaging] really, really easy to do,” Brodbeck says.

To the short list of important standards such as WS-Security and WS-ReliableMessaging, Brodbeck adds RSS, the wildly popular format for Weblog syndication. That PGP would regard this variant of WS-Lite as strategic may surprise you, but if you think about how collaboration and knowledge management drive the top line in an organization such as Pfizer, it shouldn’t. What PGP envisions, however, is not your garden-variety blogging software. “We have to recontextualize RSS for the enterprise,” says Richard Lynn, PGP’s vice president of global applications and architecture.

PGP’s requirements include virtualizing RSS feeds so that they’re independent of hard-coded addresses, aggregating them for specific business functions and securing them using the same kinds of declarative policies that govern existing Web services. According to Frank Martinez, founder and CEO of Blue Titan, a forthcoming release of Network Director will address these requirements, building on the product’s capability of wrapping WS-Heavy infrastructure around WS-Lite protocols.

Heavy, Lite, or Just Right?

When you regard the WS-* stack as a whole, you have to conclude that the critics are right: It really is a monster. Taming it will require, in part, a unifying conceptual framework. That’s a point that Gannon, Khan, and Subramaniam each make in different ways. Gannon points to a series of blueprints and reference models published by OASIS. These documents aim to help architects understand how the various WS-* specs, which are designed as modular building blocks, combine to solve specific problems. For Ohio State’s Khan, it’s not just about blueprints. He needs a toolkit that tames the complexity and thinks Indigo will be that toolkit.

RouteOne’s Subramaniam hopes that a recent initiative called JBI (Java Business Integration) will be a unifying force in the Java world. What’s hard about Web services, he says, “is that you have to see the whole picture — WSDL, and then SOAP, and relevant parts of WS-Security, and BPEL.” He’s anxious for vendors such as SeeBeyond, which was recently bought by Sun Microsystems, and webMethods to embrace JBI. “When you can see how it all fits together in the big picture of JBI, a very nice infrastructure emerges,” he says.

Of course, toolkits and frameworks are double-edged swords. Even when wire protocols are standard and open, you can get locked in to proprietary abstractions layered on top of those protocols. That’s why pragmatic architects and developers who don’t yet need advanced WS-* features tend to focus on the basics: SOAP and WSDL. “If you need some kind of envelope, why wouldn’t you use SOAP?” Subramaniam asks. “And if you need to describe your interfaces precisely, why wouldn’t you use WSDL?” Frank Grossman, co-founder of Mindreef, says that most of the customers who use his company’s SOAPscope diagnostic suite have adopted this strategy, which he adroitly labels “WS-JustRight.”

For Grossman and others, WS-JustRight means using SOAP and WSDL to strike a balance between formal contracts and agile interoperability, while laying a foundation for future use of more advanced SOA features. PGP’s Brodbeck agrees that WSDL is the key enabler of reusable business transactions and processes.  He also extends the definition of WS-JustRight, however, to include enterprise-enabled RSS as the key enabler of reusable content.

For many practitioners, WS-JustRight now includes aspects of WS-Security, too. For a few, it includes reliable messaging, transactions, routing, and policies related to these features. The definition will evolve over time, but the only one that really matters now is the one that’s just right for you.