Java Tip: Write an SOA integration layer with Apache Camel

In this Java tip, learn how to use Apache Camel to develop a quick-and-easy integration layer for a service-oriented architecture. Apache Camel lets you configure Camel integration components for web service endpoints; message transformation, routing, and aggregation; and exception handling. All with a minimum of Java coding.

Introduction

Integration is a top priority for enterprise projects that seek to connect multiple web service endpoints quickly, efficiently, and maintainably. From a development standpoint, integration can also be a serious challenge. One way to ease that challenge is by using an integration framework like Apache Camel.

Apache Camel’s API and out-of-the-box components implement many common enterprise integration patterns (EIPs), making it relatively simple and easy to do integration tasks such as connecting web services, performing XSL transformations, logging audits, and more. In this Java tip we introduce a process for using Apache Camel as an integration solution. We start with a business integration problem involving multiple web services. We then map the problem, briefly discuss the components needed to resolve it, and implement a solution based on Apache Camel’s routing engine.

An enterprise integration scenario

Say that we are given an airline portal that wants to develop a web service for its online reservation service. The service will aggregate fare quotes from multiple airline companies in order to provide portal visitors searching for flights with the best available fares. This scenario involves connecting multiple web services: The portal will invoke a ReservationService to collect a list of quotes based on the given search criteria. The portal ReservationService will then invoke the ReservationServices exposed by various airlines, consolidate the quotes, and return the results to the portal user. This is a typical integration scenario where a composite web service invokes multiple external web services and aggregates the responses.

Figure 1 shows a flow diagram of an SOA architecture for the portal reservation service and two airline reservation services. (Note that we are using a contract-first or Design by Contract for this demonstration.)

Figure 1. Flow diagram of an SOA architecture (click to enlarge)

Creating a custom solution for this business problem would mean writing a lot of Java code. We would need to map and transform the Request objects from their source formats to the target format; use Java threads to invoke both of the airline web services in parallel; set up event listeners to wait for both services to respond; aggregate the responses; and also manage supporting tasks like logging and exception handling.

Using Apache Camel as an integration framework shortcuts a lot of that programming. We can simply leverage Camel’s API and components, which are designed to resolve most web services integration tasks with standard enterprise integration patterns (EIPs). Rather than Java code, we’ll work with a streamlined domain-specific language (DSL). Apache Camel provides DSLs based on Java, Scala, Groovy, or XML. In this case, we’ll use Camel’s XML-based Spring DSL to specify the routing and configure our integration layer.

Enterprise integration with Apache Camel

Figure 2 shows how we plan to implement our requirements using Spring configuration and built-in Camel components.

As shown, a portal user (Consumer) sends a GetQuote request consisting of a starting location, travel destination and dates, and (optionally) the preferred airline. The response consists of an array of price elements with the airline name. The portal ReservationService accepts the GetQuote request. Based on the Consumer‘s preferences, it invokes the ReservationServices of either Airline A or Airline B or All. It then aggregates and returns the responses obtained from these web services to the Consumer.

The three web services in our SOA are integrated using Camel routes, which are defined in XML files. We’ll also need to write some Java code for the processors that comprise the routes.

Components of the Camel SOA integration layer

Before we proceed into the demo we’ll briefly introduce the Apache Camel components that we’ll use to create the SOA integration layer for our reservation service implementation:

• Camel CXF Endpoint: Camel uses Apache CXF for web services implementation. We’ll use the the Camel CXF component to configure all three of our web services (the portal ReservationService and the two airline ReservationServices) and use them in the Camel routes.
• Camel Bean: Camel’s component for using JavaBeans. We’ll use the Camel Bean to set up custom processing in some of the steps that follow.
• Camel Direct: Connects two routes via direct synchronous invocation.
• Camel XSLT: Loads and uses XSLT sheets via Spring’s resource loading mechanism. We’ll use the XSLT component to do request-and-response transformations.
• Camel Multicast EIP: This pattern allows us to send messages to several different destinations for processing. We’ll use multicasting to send requests from the portal ReservationService to our two airline ReservationServices.
• Camel Aggregator EIP: This pattern combines the results of individual but related messages into a single outgoing message. We’ll use Aggregator to combine the responses from our two airline Reservation Services.
• Camel WireTap EIP: This pattern lets us inspect messages as they flow through the system. We’ll use this EIP to log messages at appropriate points in the flow shown in Figure 1.

The steps to implement our SOA integration layer are as follows:

• Step 1: Generate Java types for the three WSDLs
• Step 2: Configure the Camel CXF endpoints
• Step 3: Configure the Camel routes to use multicasting
• Step 4: Configure the Camel routes for message transformation
• Step 5: Write the XSLT sheets for message transformation
• Step 6: Use WireTap to log messages
• Step 7: Create the CamelContext XML
• Step 8: Build and deploy the application

 Step 1: Generate Java types for the three WSDLs

To start, we use the Maven cxf-codegen-plugin and the wsdl2java tool provided by CXF to generate the Java artifacts from the three WSDLs found in the article source code. To execute the POM, use: cmd : mvn generate-sources. CXF generates the Java artifacts and puts them in the specified directory. We then use Maven to build our artifacts.

Listing 1. Maven plugin for generating Java stubs from WSDLs

  <plugin>
                <groupId>org.apache.cxf</groupId>
                <artifactId>cxf-codegen-plugin</artifactId>
                <version>2.7.0</version>
                <executions>
                    <execution>
                        <id>generate-sources</id>
                        <phase>generate-sources</phase>
                        <configuration>
                            <sourceRoot>${basedir}/src/main/java</sourceRoot>
                            <wsdlOptions>
                                <wsdlOption>
                                    //ReservationService invoked by Portal
                                    <wsdl>${basedir}/src/main/resources/wsdl/ReservationService.wsdl</wsdl>
                                </wsdlOption>
                                <wsdlOption>
                                    //ReservationService exposed by Airline A
                                    <wsdl>${basedir}/src/main/resources/wsdl/ReservationServiceAirlineA.wsdl</wsdl>
                                </wsdlOption>
                                <wsdlOption>
                                    //ReservationService exposed by Airline B
                                    <wsdl>${basedir}/src/main/resources/wsdl/ReservationServiceAirlineB.wsdl</wsdl>
                                </wsdlOption>
                            </wsdlOptions>
                        </configuration>
                        <goals>
                            <goal>wsdl2java</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

 Step 2: Configure the Camel CXF endpoints

Next, we define the endpoints for ReservationService, ReservationServiceAirlineA, and ReservationServiceAirlineB shown in Listing 2 below. These endpoints will be used subsequently in our Camel routes. Note that the Service interface class generated in Step 1 is referenced by the camel-cxf endpoint.

The camel-cxf endpoint uses the default dataFormat=POJO. In this example we used the data format of PAYLOAD to get the raw XML into our Camel endpoint and process the XML in our Camel routes. Enabling schema validation validates the incoming request against the XSD schema defined in the WSDL, which reduces the time we would otherwise need to invest in validation.

Listing 2. Camel CXF endpoints for web services

//Endpoint of Mock ReservationService for Airline A
    <cxf:cxfEndpoint id="airlineAEndpoint"
        address="http://localhost:8089/QuoteService" serviceClass="com.sample.aira.quote.AirLineAQuote"
        wsdlURL="wsdl/ReservationServiceAirlineA.wsdl">
        <cxf:properties>
            <entry key="dataFormat" value="PAYLOAD" />
        </cxf:properties>
    </cxf:cxfEndpoint>


    //Endpoint of Mock ReservationService for Airline B
    <cxf:cxfEndpoint id="airlineBEndpoint"
        address="http://localhost:8088/QuoteService" serviceClass="com.sample.airb.quote.AirLineBQuote"
        wsdlURL="wsdl/ReservationServiceAirlineB.wsdl">
        <cxf:properties>
            <entry key="dataFormat" value="PAYLOAD" />
        </cxf:properties>
    </cxf:cxfEndpoint>

    //Endpoint of ReservationService
    <cxf:cxfEndpoint id="reservationService" address="/QuoteService"
        serviceClass="com.sample.air.quote.AirLineQuote" wsdlURL="wsdl/ReservationService.wsdl">
        <cxf:properties>
            <b><entry key="dataFormat" value="PAYLOAD" /></b>
            <b><entry key="schema-validation-enabled" value="true" /></b>
        </cxf:properties>
    </cxf:cxfEndpoint>

 Step 3: Configure the Camel route(s) to use multicasting

Apache Camel allows us to use the Multicast pattern to send a single message to more than one endpoint. Here, we use Multicast to send the request from our portal ReservationService to the airlineARoute and airlineBRoute. Setting the parallelProcessing=true attribute means that we can use multicasting in parallel, thereby optimizing the response time. Both routes in turn transform the received request into the destination-request format and invoke the Reservation Services from Airline A and Airline B in parallel.

As shown in Listing 3, we use the processor in Apache Camel to set the header of the exchange and to specify a Consumer‘s choice of airline (AirlineA, AirlineB, or All). This allows us to configure the choice when construct in the Camel route to direct the message to the user’s choice of airline, with easy XML configuration. Camel processor is a Java class that implements Camel’s Processor interface and is used to transform the message (exchange) flowing through the Processor or add important information in the header, which can be used in the Camel route to branch out the flow, based on business inputs.

Because we’ll receive a response from more than one web service, we also need to aggregate the responses before we send them back to the original service. For this we use Camel’s Aggregator pattern implementation. Camel’s support for aggregation is provided by the strategyRef attribute of the Multicast tag, which is shown in Listing 3 below.

The strategyRef attribute points to an implementation of the AggregationStrategy class, which you can view in the article source. The overridden aggregator method gets responses from airlineARoute and airlineBRoute. The responses can be aggregated in this method and set in the Camel Exchange.

Listing 3. Camel route to conditionally invoke the target web services

<camel:route id="airLineQuoteRoute">
            // Web service Consumer endpoint representing the ReservationService exposed to outer world
            <b><camel:from uri="cxf:bean:reservationService" /></b>

            //Use the Camel wireTap component to take a snapshot of the message 
            <wireTap uri="direct:logInfo"/>

            //Camel processor that does business validations, extracts the information from incoming XML and sets the Exchange header
            <camel:to uri="bean:requestValidator" />
            
            //Conditional Route Flow
            <camel:choice>
                <camel:when>
                    <camel:simple>${header.airLines} == 'AirLineA'</camel:simple>
                    <camel:to uri="direct:getQuoteA"></camel:to>
                </camel:when>
                <camel:when>
                    <camel:simple>${header.airLines} == 'AirLineB'</camel:simple>
                    <camel:to uri="direct:getQuoteB"></camel:to>
                </camel:when>
                <camel:when>
                    <camel:simple>${header.airLines} == 'All'</camel:simple>
                    <camel:to uri="direct:getQuoteMulticast"></camel:to>
                </camel:when>
            </camel:choice>

            //To handle exceptions occurred
            <camel:onException>
                <camel:exception>java.lang.Exception</camel:exception>
                <camel:log message="Exception occurred ${exception.message}" />
            </camel:onException>
        </camel:route>
        
        // Routing logic using the Camel Multicast component
        <camel:route id="getQuoteMultiCastRoute">
            <camel:from uri="direct:getQuoteMulticast" />
            
            // Mulitcast call for both Airline A and Airline B 
            <b><camel:multicast strategyRef="aggregateData" parallelProcessing="true">
                <camel:to uri="direct:getQuoteA" />
                <camel:to uri="direct:getQuoteB" />
            </camel:multicast></b>

            <camel:to uri="bean:processResponse" />

            <camel:to uri="xslt://process_response.xsl" />
            <camel:onException>
                <camel:exception>java.lang.Exception</camel:exception>
                <camel:log message="Exception occurred ${exception.message}" />
                <camel:to uri="bean:exceptionHandler" />
            </camel:onException>
        </camel:route>

 Step 4: Configure the Camel route for message transformation

In Step 3 we used multicasting to invoke the two routes for the Airline A and Airline B ReservationServices. Each of these routes transforms the incoming XML to the target format, invokes our two airline reservation services, and transforms the response XML. Apache Camel allows the use of XSLT sheets via the Camel XSLT component. We use XSLT sheets to transform the request-and-response message from the source format (that is, the portal ReservationService) to the target formats required by the airline ReservationServices. As stated in Step 2, the ReservationService endpoints use a PAYLOAD dataFormat, which allows us to process and transform the raw XML using XSLT sheets.

Listing 4. A Camel route for message transformation and invoking target web services

//  Routing logic for Airline A Web Service
        <camel:route id="getQuoteAirLineARoute">
            <camel:from uri="direct:getQuoteA" />
            // To create "StreamSource" (Stream of XML) of the request, and send the message to Camel XSLT component
            <camel:to uri="bean:getXMLStreamSource" />
            
            //loads pre-defined XSLT sheet
            <b><camel:to uri="xslt://airlineA_req_transform.xsl" /></b>
            
            //set the necessary headers to set the namespace and operation to be invoked on the target Web Service 
            <camel:setHeader headerName="operationNamespace">
                <camel:simple>http://aira.sample.com/quote/</camel:simple>
            </camel:setHeader>
            
            <camel:setHeader headerName="SOAPAction">
                <camel:simple>http://aira.sample.com/quote/getQuoteOperation</camel:simple>
            </camel:setHeader>
            
            //Use wiretap component to take a snapshot of the request message (optional)
            <wireTap uri="direct:logInfo"/>
            // Invoke Airline A Web Service with the Transformed request
            <b><camel:to uri="cxf:bean:airlineAEndpoint" /></b>

            //Use wiretap component to take a snapshot of the response message (optional)
            <wireTap uri="direct:logInfo"/>
            //do response transformation
            <b><camel:to uri="xslt://airline_resp_transform.xsl" /></b>

            //Handle  Exceptions 
            <camel:onException>
                <camel:exception>java.lang.Exception</camel:exception>
                <camel:to uri="bean:exceptionHandler" />
            </camel:onException>
        </camel:route>
        
        //do similarly for Airline B Route
        <camel:route id="getQuoteAirLineBRoute">
            <camel:from uri="direct:getQuoteB" />

            <camel:to uri="bean:getXMLStreamSource" />

            <b><camel:to uri="xslt://airlineB_req_transform.xsl" /></b>

            <camel:setHeader headerName="operationNamespace">
                <camel:simple>http://airb.sample.com/quote/</camel:simple>
            </camel:setHeader>
            
            <camel:setHeader headerName="SOAPAction">
                <camel:simple>http://airb.sample.com/quote/getQuoteOperation</camel:simple>
            </camel:setHeader>

            <b><camel:to uri="cxf:bean:airlineBEndpoint" /></b>

            <b><camel:to uri="xslt://airline_resp_transform.xsl" /></b>

            <camel:onException>
                <camel:exception>java.lang.Exception</camel:exception>
                <camel:to uri="bean:exceptionHandler" />
            </camel:onException>
        </camel:route>

 Step 5: Write the XSLT sheets for message transformation

In this step we apply XSL transformation to generate the request type of the destination web service. Apache Camel allows the use of XSLT sheets via the Camel XSLT component. The input will be the raw XML obtained from the ReservationService endpoint. This approach eliminates the need to do XML-to-Java binding and is more efficient and maintainable. As previously mentioned, the transformation rules are defined in custom XSLT sheets, which are loaded using the Camel XSLT component. With this approach we are able to update the mapping rules by making changes in the XSLT sheet itself, without updating the Java code. For example, we use the XSLT sheet to transform the request XML of the portal ReservationService into the destination-request XML of the Airline A and Airline B ReservationServices. We apply a similar technique for response transformation. (See the article source to view a sample XSLT sheet.)

Note: The Camel XSLT component uses optimized Spring resource loading to load the XSLT sheets. It also uses a caching mechanism, which is enabled by default.

 Step 6: Use WireTap to log the message

Wire Tap is an EIP that creates a copy of an original message and routes it to another destination. Here, we use Camel’s wireTap component to log the messages flowing between our three web services at appropriate points. We actually called the wireTap component in Step 4 to direct a message to a route as shown below. The route in Listing 5 uses a JavaBean to log messages into an appropriate data store, such as a file or database.

Listing 5. Message logging with Camel’s wireTap component

//Use wireTap in the main route where necessary, and send to a desired route using Camel's direct component
<b><wireTap uri="direct:logInfo"/></b> 

//Log the message in this route
<route id = "auditLogging">
    <from uri="direct:logInfo"/>

    //log the request object in database / file
    <to uri="bean:logAudit"/>   
</route>

 Step 7: Create the Camel Context XML

In this step we create the camel-context XML. We add the processors, beans, routes, and endpoints in the camel-context.xml.

You might recall that we’ve used processors in our Camel routes to do various tasks. For instance, in Step 3 we used a Camel processor to add a header to the Camel Exchange, in order to conditionally execute our system flow and invoke a response from one or more desired airline web service(s). In this case the processor extracted the preference from the request XML and set the header, allowing us to configure this rule in a Camel route.

Additionally, in Step 4 we used a Camel processor to change the message into a format suitable for the XSLT component, and we used a Camel processor in the onException block in that route to extract the exception object and create a SOAP fault. In Step 6 we used a Camel processor to log the message into a data store. See the source code for a detailed listing of the processors.

Listing 6. camel-context.xml

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xmlns:cxf="http://camel.apache.org/schema/cxf" 
    xmlns:camel="http://camel.apache.org/schema/spring"
    xsi:schemaLocation="
       http://www.springframework.org/schema/beans 
       http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://camel.apache.org/schema/spring 
       http://camel.apache.org/schema/spring/camel-spring.xsd
       http://camel.apache.org/schema/cxf 
       http://camel.apache.org/schema/cxf/camel-cxf.xsd>

    //Camel CXF endpoints used in Step 2
    <cxf:cxfEndpoint ...

    //Java beans and Camel Processors used in Steps 3,4 and 6
    <bean id="requestValidator" class="com.sample.processor.RequestValidator" />

    //the camel-context that encloses the Camel routes defined above
    <camel:camelContext id="camelContext"
        xmlns="http://camel.apache.org/schema/spring">
    //the routes defined above
    <route>
    ...
    </route>
    <route>
    ...
    </route>
    </camelContext>
</beans>

In Listing 7 we create the web.xml to load camel-context and deploy the application as a WAR file.

Listing 7. web.xml

<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app version="2.4" xmlns="http://java.sun.com/xml/ns/j2ee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">

    <display-name>Camel Routes</display-name>

    <servlet>
        <servlet-name>CXFServlet</servlet-name>
        <servlet-class>org.apache.cxf.transport.servlet.CXFServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
        <servlet-name>CXFServlet</servlet-name>
        <url-pattern>/*</url-pattern>
    </servlet-mapping>


    //camel context xml
    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>/WEB-INF/camel-context.xml</param-value>
    </context-param>

    //the listener that kickstarts Spring 
    <listener>
        <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    </listener>
</web-app>

 Step 8 : Build and deploy the WAR file

We are now ready to deploy our Camel SOA integration layer. First we’ll add the necessary dependencies to our project’s pom.xml (see the article source code), then we’ll use our Maven POM script to build the application.

Executing the Maven POM script with cmd : mvn install generates a WAR file. Repeat this for the three projects: ReservationService, ReservationServiceAirlineA, ReservationServiceAirlineB. The script compiles the Java code and copies the camel-context.xml from Listing 6, the XSLT sheets, and the compiled Java classes into the WEB-INF/classes folder in the WAR file. We can then deploy and run the integration layer in a web server like Tomcat.

To deploy the sample app on a Tomcat server, simply copy the WAR files generated to the webapps folder of your server and start it. Next, access the WSDL of the deployed service at the following URL:

http://localhost:8080/ReservationService-1.0-SNAPSHOT/QuoteService?wsdl

Use the WAR files to deploy the mock web services for Airline A and Airline B. The WSDL URLs will be:

http://localhost:8080/cxf-sample-airlineA-1.0/services/AirLineAQuote?wsdl

and

http://localhost:8080/cxf-sample-airlineB-1.0/services/AirLineBQuote?wsdl

Use the request XML of the reservation service in the source code to check out these web services.

Conclusion

In this Java tip we’ve shown you briefly how to use Apache Camel to build an integration layer in an SOA web services implementation. We used the Camel CXF PAYLOAD data format for direct access to the raw XML passed between web services, and XSLT sheets to do the request-and-response transformation. This approach removed the need for XML-to-Java binding, with XML validation happening at the CXF endpoint or in the XSLT as needed.

We also demonstrated Camel’s implementation of the Multicast and Aggregator integration patterns, which we used to route a single web service request to multiple endpoints in parallel and aggregate the responses. We used the Camel wireTap component to log messages at points of interest in our system. We also used the onException block in our Camel route to handle exceptions and throw SOAP faults, as necessary. Using Camel’s Spring DSL, we were able to configure these components using XML, with very little Java code. See the Resources section to learn more about Apache Camel and download the source code for the demo.

Anirban Konar has more than 17 years of experience in IT and has designed and developed Java-based solutions since 2001. A Sun Certified Java Programmer and Enterprise Architect, Anirban currently works as a senior architect for Cognizant Technology Solutions, India.

Devaradjan Venkatesan has over eight years of experience in IT and currently works as a technology specialist for Cognizant Technology Solutions, India.