Connecting applications on both sides of the firewall

This week on the New Tech Forum we’re featuring Maneesh Joshi, senior director of product marketing and strategy for SnapLogic, a cloud data integration startup. As companies move forward with a mixed bag of applications and services, both in-house and in the cloud, the problem of creating data connections between these apps will arise over and over, as each new application is integrated into the enterprise.

SnapLogic has designed a software stack that provides data connections between disparate data sources, allowing companies to control data flows between different in-house applications, or cloud applications, or a mixture of both. Joshi, whose resume includes a stint at Oracle as a developer, provides a clear explanation of how SnapLogic’s technology works. — Paul Venezia

Elastic integration for the 1,000-app enterprise

The SaaS model continues to increase in popularity, thanks to the promise of agility at low risk. But SaaS applications complicate the already intricate web of on-premise enterprise applications. Business applications popular in the 1990s — such as Oracle eBusiness Suite, PeopleSoft, Siebel CRM, or SAP R/3 — are each being supplanted or extended by hundreds of SaaS point solutions. For example, Okta focuses specifically on enterprise identity management for the cloud, whereas Zuora focuses on subscription billing alone. With this explosion of SaaS offerings, the application integration challenge has never been harder. It’s no longer a distant prospect to imagine enterprises running 1,000 different applications.

SnapLogic’s Elastic Integration provides an easy on-ramp to the cloud by providing a flexible integration layer that connects any combination of data sources more quickly and cost-effectively than traditional integration solutions. SnapLogic’s integration infrastructure and SnapStore enable customers to integrate data quickly using a drag-and-drop visual designer and over 160 prebuilt intelligent connectors called Snaps that do not require coding.

SnapLogic’s Web-based architecture and data containerization technique stand apart from other integration approaches. SnapLogic enables connections between an ever-growing set of cloud applications, as well as other data sources such as legacy enterprise systems, social media, and machine-generated events.

As businesses adopt SaaS applications, in most cases existing on-premise applications are not being ripped and replaced, but augmented with SaaS. The result is often referred to as a “hybrid cloud” world. The ideal integration solution that binds all of this together will be more effective if it is elastic, both on-premise and in the cloud. At SnapLogic, we believe hybrid is the state and elasticity is the solution.

Three-way elasticity SnapLogic delivers elasticity in three ways. First, by leveraging the Amazon’s EC2 (Elastic Compute Cloud) infrastructure, SnapLogic can to scale up and down to meet the customer’s integration needs in the cloud. Second, with appropriate hardware configurations, or by leveraging private cloud infrastructure, customers can benefit from elastic integration in on-premise data centers as well. Third, by using a technique known as “function shipping,” customers can move functions closer to where data resides and reduce network bandwidth consumption.

Imagine that a CSR (customer service representative) is reviewing his customers along with information about the products they own using the SaaS-based ServiceNow portal. Part of the content displayed in the portal needs to be refreshed from Salesforce.com with customer and products-owned data. While fetching customer data from Salesforce.com, the CSR would like to enrich it with recent clickstream activity from the customer support portal. Among other things, this clickstream of support-related activities can flag any unresolved issues.

The challenge is that both applications, Salesforce and ServiceNow, are in the cloud, while the large clickstream data set resides in an on-premise Splunk data store. Rather than loading this large data set into the cloud and joining it with customer and product data — and tying up network bandwidth — SnapLogic allows customers to ship a function to the Splunk instance and return much smaller query results. With this approach, SnapLogic helps customers optimize their integrations around data gravity and move closer to where data density is highest.

Designing connections and using snaps In the spirit of cloud computing, SnapLogic users are provisioned with a dedicated sandbox in a multitenanted environment for development, simulation, and test of integration pipelines. Using SnapLogic Designer, pipeline building is a simple configuration exercise that requires no programming expertise. Customers benefit from universal access via a 100 percent HTML5-based user interface, with access secured by SSO-based policies and encryption of metadata.

SnapLogic Designer’s drag, drop, and configure functionality enables customers to snap integration pipeline components together easily. These components, called Snaps, are intelligent prebuilt connectors that are available in an online marketplace called SnapStore. The SnapStore is an easily browsable and searchable catalog of more than 160 prebuilt, precertified Snaps that connect to a variety of enterprise applications, social websites, databases, and other technology protocols.

Snaps shield both business users and developers from much of the complexity of the underlying application, data model, and service. Snaps are easy to build and modify, and they’re based on an open and standardized development environment.

Graphical administration The SnapLogic Manager provides comprehensive operations support where pipelines can be visually migrated from development to testing, staging, and production. The management console also includes sophisticated error-management capabilities, where errors can be identified and corrected easily. The console can be used to view end-to-end pipeline execution, auditing, and logging information. Integration metrics and statistics are displayed in graphical fashion and proactive alerts assist with exception management. The console is also protected with SSO-based policies.

SnapLogic simplifies the process of taking an integration pipeline through its lifecycle by separating pipeline metadata and environment configuration. The environment-specific configuration — for example, test endpoints are replaced with production endpoints when deploying to production — are captured in what are referred to as environment documents. This makes moving of pipelines across environments a seamless process of changing the associated configuration file as opposed to changing the pipeline metadata.

The future is now With the explosion in SaaS applications, the 1,000-app enterprise is no longer a pipe dream. It represents a true paradigm shift, where the integration layer moves closer to where new data is generated in the cloud. On-premise integration solutions that were purpose-built to run behind a firewall are ill-suited for the new world of cloud services.

Implementing incremental solutions to handle this new reality will help customers extend life of their existing investments, but sooner or later, a tipping point will be reached beyond which new solutions such as those offered by SnapLogic will become necessities. Customers need to research their options and invest in the right set of technologies that will help secure their long-term interests. Eventually, they will need to embrace the new paradigm rather than resist it.

New Tech Forum provides a means to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all enquiries to newtechforum@infoworld.com.

This article, “Connecting applications on both sides of the firewall,” was originally published at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.