Lessons from an SOA pioneer

Today, SOA (service-oriented architecture) is the undisputed champion of IT trends. But IT professionals have seen other megatrends come and go, some successful, some disastrous. Many companies remain skeptical about SOA, in part because most deployments are recent, leaving any assessment of long-term viability inconclusive.

Yet a handful of deployments that began before the SOA acronym was coined are beginning to suggest how effective the service-based approach to application architecture and business agility can be over the long haul. Among these, Con-Way Transportation Services stands out as an ambitious, successful reinvention of one enterprise’s application infrastructure.

Whereas most enterprises in the 1990s were developing or deploying client/server architectures, Con-Way decided to take a more component-based, distributed approach to app dev and delivery. As Con-Way’s SOA evolved, it helped insulate the company from the disruptive effects of new technologies, mainly because SOA’s basic premise of abstracting services makes it easy to adapt as technologies change.

Con-Way’s initiative also demonstrates that the key to successful SOA deployment is to focus on the business processes that are the heart of the services, rather than on specific technology platforms.

Reworking a legacy

In 1997, as a newly spun-off subsidiary of logistics management company CNF, the Con-Way shipping unit relied on corporate systems for inventory management, billing, order tracking, and the like. That limited the organization’s flexibility and its responsiveness, recalls Praveen Sharabu, Con-Way’s director of enterprise architecture, because CNF’s IT group and software development efforts had to balance Con-Way’s needs against those of the other units, which at the time included the Emery Worldwide air freight service and the Menlo Logistics warehouse management service.

With an initial IT staff of six, Con-Way’s executives decided to keep development in-house rather than rely on the parent company. (The IT staff is now about 100, including developers, architects, and networking administrators.) Con-Way would continue to rely on CNF for its Oracle Financials and PeopleSoft applications and databases for financial reporting and HR functions because all CNF units followed the same standards. But it wanted control of the applications used to run its business: customer tracking, billing, shipment management, and other core functions.

The IT staff calculated that, following a conventional app-dev approach, reworking Cobol applications for shipment management to meet requirements would take five years — not exactly an agile turnaround. But several new staff members, including Sharabu, had experience with component-based development using Texas Instruments’ IEF (Information Engineering Facility) environment from previous jobs. They thought it would make sense to develop their mainframe apps as coarse-grained components, so they could roll them out incrementally, rather than waiting until the complete system was done before seeing any benefits.

They brought in an independent consultant familiar with both component architectures and object-oriented development. He showed them how to develop components that were abstract enough to be reusable, so the IT staff could avoid creating multiple versions of essentially the same code, which would both be a long-term management problem and take more development time in the long run.

The team modeled the shipment functions across the transportation lifecycle — from pickup request to pickup, intermediate transit and storage, delivery, and finally to billing and payment. “We created our methodology at that time,” Sharabu says, by mapping out the business logic and figuring out what software components were needed to support it. “The granularity is very important. There’s a tendency to make components too small or too big. The big lesson is that the development team has to be very close to business. They have to know the business processes; the granularity comes from the processes.”

The importance of architecture

Not far into the effort, team members realized that they had underestimated the importance of architecture. A couple of staffers focused on data modeling to ensure that the data structures were consistent across the various components. But it soon became apparent that data modeling per se was insufficient, that the real effort needed to be on architectural design and management so that efforts weren’t duplicated and components weren’t too tangled in dependencies. To this day, every development effort goes through the architectural team, which acts as both a design resource and requirements enforcer. “They retain the big picture of the implementation,” Sharabu says.

Development itself is not about writing code but about developing business logic. The team uses the Computer Associates COOL:Gen development platform — a later incarnation of the IEF platform it started with — to specify the logic, also relying on it to generate the component code. The mainframe components are still output as Cobol code because Con-Way runs its DB2 databases and CICS applications on IBM mainframes for their high transactional performance. “The code generated by COOL:Gen is perfect,” Sharabu says, so there’s no requirement for extensive Cobol expertise. The “applications” at Con-Way are essentially wrappers of specific business logic that call up various components as needed, passing data from one component to another to accomplish a task such as looking up a customer’s orders for the past month and generating the related invoice.

In some cases, the components communicate with packaged applications that Con-Way decided to buy rather than to attempt replicating. For example, although Con-Way is primarily an IBM shop, it uses BEA Systems’ WebLogic server for some services.

Evolving services

In 1999, when three of the seven mainframe application development phases were complete, Con-Way realized that customers wanted to access order and tracking information via the Internet, which was then just getting initial traction as a service-delivery platform. So Con-Way began creating components that acted as proxies between the Web’s graphical interface and the green-screen presentation of CICS. “All we had to do was expose our components to the Web,” Sharabu says. Written in COM and Visual Basic, these initial components were made available to customers to run on their computers, with the service distribution managed at Con-Way using Microsoft IIS.

Soon after, Con-Way switched to Java because the IT staff decided that client-side Java made the most sense, providing both a graphical UI and the zero deployment advantages of a thin client. “Most of the Java functionality was over UI and presentation, pulling together the business components,” Sharabu says. “We still have it the same way.” Customers typically have browser-based SSL connections to Con-Way’s systems, although a few integrate directly using SOAP and SSL. Data is typically presented through XML or as report files in popular formats such as Microsoft Word and Excel.

Con-Way dropped the IIS server in 2000 because Microsoft ASPs were “not cutting it” as usage increased, Sharabu says. Instead, the company deployed an IBM WebSphere server and began using EJBs to encapsulate the business and presentation logic for Web-based services.

In 2003, Con-Way extended its architecture to work with a data warehouse that replicated data from its parent company’s Oracle system. It deployed a Tibco EAI server to connect its mainframe applications to CNF’s Oracle Financials and PeopleSoft applications so that the two companies’ systems could interact. (Before that, Con-Way and CNF just exchanged data.) The initial interface components were hard-coded; Con-Way is now creating components that deal with the applications’ APIs, so as the applications change, the components don’t have to.

The company has also expanded the use of data warehouses and data marts, as customers demand more information in near real time. To support that, Con-Way has created Web-based components that allow customers to run queries and reports against these data marts. Con-Way is also exploring the use of an event-based architecture on its Tibco server to actively deliver information and execute component services, such as sending a confirmation when a package is delivered or updating the shipping manifest if a mismatch is detected between a customer’s stated and actual package size and weight. “Having components lets us prioritize and stage our propagation,” Sharabu says.

An early, but careful, adopter

Although the Con-Way effort began eight years ago, the basic architecture has remained stable and has allowed the company to change its technologies while maintaining the underlying business logic and adding new business logic as the market demands. The SOA also allows Con-Way to maintain several types of technology, each suited for specific operations, without creating an overly complex integration challenge. And its IT staff is able to respond to business needs quickly by truly understanding the business processes.

Con-Way is also careful to avoid leaps into the latest standards and technologies promoted under the SOA label. “We are not in a hurry to embrace every new technique, process, standard, or technology that comes down the pike,” Sharabu says. “We’d rather examine it and match it to our business relevance and adopt it only if it makes sense to do so.”

In fact, Sharabu insists that if an SOA’s focus becomes specific technologies — rather than the architecture — IT has missed the point. “Companies tend to look for a Holy Grail. But this is really about building a solid foundation,” he says. A successful SOA implementation allows IT to adopt new technologies as appropriate, while keeping the old ones as long as they continue to work.

The componentization of business processes into discrete pieces of technology allow businesses to focus just on changes to business processes and truly valuable new technologies, not on consistent redevelopment of the same systems based on the technology du jour, he says. “The trick is to not let new technology changes affect what already accomplishes your business processes.”