How to choose between custom and commodity clouds

Not all cloud infrastructures are built the same way, and not all applications, services, and frameworks are built for cloud computing.

In this week’s New Tech Forum, Brent Bensten, CTO of custom cloud provider Carpathia, discusses the phenomenon of “bottom dollar” cloud computing and what that means from a performance, security, and regulatory perspective — and how to make sure the cloud resources you choose are the right tools for the job. — Paul Venezia

Don’t be fooled by the one-size-fits-all cloud

The public cloud is finally gaining acceptance. Freed from the shackles of the IT department, employees and their business units can now obtain the resources they need at a low cost of entry and hassle.

But what’s good for the individual is not necessarily good for the enterprise. As familiarity with cloud architectures rises, so does the awareness that the public cloud does not suit every IT function, particularly when it comes to high-volume, low-latency applications like big data and rich media processing.

Cloud wars and the cost of commoditization

The main drawback of running high-end applications on public cloud resources is lack of customization. This is primarily due to the race-to-the- bottom pricing that top providers like Amazon, Google, Microsoft, and others have engaged in recently. It is now possible to find storage resources for about 2 cents per gigabyte per month and database operations for about a penny per 1,000 transactions.

Ultimately, this price war will influence all IT spending, whether for Web-based services or end-to-end IaaS (infrastructure as a service) ecosystems. From a purely operational perspective, it tends to mask two crucial aspects of the public cloud: First, the resources available for bottom dollar are usually low quality — with limited availability, latent performance, and other detriments that make them unsuitable for modern production environments. Second and even more important is the fact that most public clouds are built on commodity infrastructure designed to support low-cost, scale-out, virtual architectures.

This makes sense for the cloud provider because it keeps hardware costs down and can be configured to meet the generalized computing needs of the widest array of users. The top providers long ago figured out that basic commodity hardware, much of it sourced directly from original design manufacturers, provides both the scale and the horsepower to support higher-level virtual environments.

Performance pitfalls of generalized service

For basic, noncritical applications and data, the public cloud provides a viable solution to rising volumes. But as many early adopters are finding out, it is woefully inadequate when it comes to high-order enterprise functions like data analytics and advanced business-process applications, almost all of which require highly specialized hardware configurations in order to deliver optimal performance. This level of customization is simply not possible on generic, commodity-based cloud platforms.

Big data analysis is a perfect example. Imagine a Hadoop cluster in a generic public cloud compute environment. That infrastructure could just as easily go toward Web applications, database management, or a hundred other uses. But to get top performance from Hadoop, you need to deliver the proper mix of CPU, RAM, storage, and network support depending on the nature of the loads you are running.

Optimum Hadoop performance can be achieved only with a customized, purpose-built configuration. To meet that challenge, many enterprises and government agencies seeking more powerful and cost-effective access to large-scale data processing capabilities have turned to HaaS (Hadoop as a service) solutions. Engineered to meet stringent federal requirements, HaaS solutions can processes data many times faster than commodity-built Hadoop platforms and ensure mission-critical availability to meet the performance and compliance demands of any organization. Such approaches provide a viable solution for matching the level of management, support, and reliability you expect from the rest of your IT and cloud infrastructure.

Balancing security and compliance

Lack of customization presents challenges beyond poor performance. Security and compliance issues are starting to crop up as well, particularly in such highly regulated areas as health care and government contracting.

HIPAA compliance can be particularly problematic in the public cloud. The latest rule changes greatly enhance patient privacy and confidentiality when it comes to the storage and sharing of personal information. By nature, public clouds are built on a shared infrastructure model with high levels of multitenancy on virtual compute, network, and storage systems. There may be some circumstances in which this will satisfy HIPAA requirements, but in most cases it will fall short. This means there is no way to prevent a data breach without a sophisticated security regime on either the hypervisor or even the application and data layers.

The sad fact is that few cloud providers are giving any more than lip service to security at this point. Again, this is a function of the generic nature of the commodity public cloud. In nearly every case, enterprise-class security must be tailored to the specific data and application environment of the organization. Anything more than basic, rudimentary security measures will tend to drive greater incompatibility between the host environment and the cloud customer — potentially opening up the organization to a costly data breach.

This same dynamic applies to other compliance rules and regulations, such as Sarbanes-Oxley, the Graham-Leach-Bliley Act of 1999 governing financial institutions, and the PCI Data Security Standard of 2004. In nearly every case, the security and compliance measures of commodity public clouds leave the client enterprise open to violation of these newly strengthened business and financial regulations.

It is also important to understand that most public cloud service-level agreements lack proper definitions for key data parameters such as ownership, portability, governance, and control. Also, visibility into cloud environments tends to be sketchy at best. Will resources be shifted to lower tiers of service? Will your data be offloaded to another site or perhaps even to another cloud provider?

In addition, migration to and from the cloud may or may not go smoothly depending on the level and type of access provided — and the ability of the provider’s infrastructure to import or export large quantities of data within a reasonable time period.

As recent headlines attest, public clouds have experienced their share of breaches and failures over the past few years. In most cases, these were caused by technical failures or human error, although there is the possibility that a cloud provider could simply close up shop, as Nirvanix did in 2013. Fortunately, most of the losses due to public cloud failure involved low-level or backup data. But if enterprises start to place more critical workloads on the public cloud, such losses would be more keenly felt.

Cloud decision points

To be fair, local data center infrastructure has a less-than-spotless record when it comes to availability. The difference is that the enterprise is in control of its own destiny when it comes to in-house systems, rather than relying on a third-party contractor. The type of workload should help determine whether you deploy in your own data center or in a customizable public cloud where you have control over the infrastructure.

Take big data as an example. Massive scale-out infrastructure is clearly needed to handle large volumes and can be delivered most economically on a pay-as-you-go basis rather than as a fixed capital expense. Meanwhile, a wide range of systems and techniques like data-scrubbing and tokenizing can be employed to move data sets across diverse infrastructures as their values change during the analytic process.

At the same time, the enterprise is secure in the knowledge that its cloud environment — even those portions that are hosted on third-party architectures — occupies dedicated, fully customizable infrastructure, subject to the same security and governance policies that oversee the broader data ecosystem.

In some cases, such an arrangement can be maintained at lower cost than a commodity public cloud, depending on the volume and type of data and the duration of the service. For example, paying 15 cents per gigabyte per month for enterprise-class storage is a bargain for relatively low levels of capacity, but a 30TB block would run upward of $4,500 per month, plus networking and other charges. The same amount would cost half as much or less on a hosted private solution and would generally come with dedicated, high-throughput networking support.

Regarding the cloud, there is no one-size-fits-all approach. Each architecture has its strengths and weaknesses in terms of accommodating the plethora of data requirements that the modern enterprise brings to the table. As a rule, however, the public cloud is suitable for environments that exhibit incremental growth over time or are subject to large peaks and troughs in data traffic. Private and hybrid infrastructure, on the other hand, can step in at a moment’s notice to handle the high-level, high-performance functions that the knowledge workforce relies upon every day. These environments can be built to order and provide for regularly updated, enterprise-class applications and services, often at less cost than traditional licensing agreements.

The cloud is here to rescue the enterprise from steadily increasing data volumes and complexity. But as with any other IT resource, it’s important to deploy the right tool for the job.

New Tech Forum provides a venue to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to newtechforum@infoworld.com.

This article, “How to chooose between custom and commodity clouds,” was originally published at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.