Think again: Data integration is different in the cloud

It’s been nearly 20 years since I wrote the book “Enterprise Application Integration,” yet after all that time data integration remains an afterthought when it comes to cloud deployments. I guess that’s par for the course, since security, governance, monitoring, and other core services are often afterthoughts as well.

When moving to the cloud, enterprises focus on the move itself, rather than on what they need after they get there. Although this may be a common plan, it’s not a best practice.

Data integration is essential because you’ve rehosted some of your data on a remote cloud service. The inventory system that’s stilling running on a mainframe in the datacenter needs to share data with the sales order system that’s now on AWS. In other words, your data-integration problem domain is now bigger and more complex.

The trouble is that traditional approaches to data integration, including traditional data-integration technology providers, are typically no longer a fit. Even data-integration technologies that I’ve built in the past as a CTO would no longer be on my short list of data-integration technologies that I would recommend today.

That’s because the use of the public cloud changes how you do data integration. For example, you need a much more lightweight approach that can deal with more types of data.

Also, having the data-integration engine in the enterprise datacenter is no longer efficient; for the same reason, it should not be placed at a cloud provider that has centralized access to all systems that are being integrated.

Cloud-based data integration also requires different types of security and governance services. Although most data that moves from system to system in an enterprise is not encrypted, you need to encrypt pretty much everything moving to and from systems in the cloud.

The list goes on.

The result is that cloud data integration is not your father’s data integration. It requires different approaches and different technologies. Although the old guard has done a pretty good job of cloud-washing their datacenter-centric solutions, you need to look beyond them, at data-integration technology that was built specifically for the cloud.