Big data gets Amazoned with Redshift

Big data is, well, big. And messy. And intimidating to many IT shops and would-be business users. Its complexity also suggests high prices, which also gives would-be adopters pause. So Amazon.com’s announcement of its low-cost Redshift big data service as part of the Amazon Web Services (AWS) suite is a big deal.

Retailers have long complained about being “Amazoned” — that is, undercut in price and surpassed in both availability and service by Amazon.com’s online store. AWS has “Amazoned” the hosted services world as well, making what we now call IaaS cheap and easy. Now big data looks as if it might get “Amazoned,” too.

Amazon’s sales pitch for Redshift lays down the gauntlet for the big data vendors such as IBM, Oracle, and Teradata, which typically come out of the pricier, more complex business intelligence or data warehousing worlds: “Amazon Redshift is a fast and powerful, fully managed, petabyte-scale data warehouse service in the cloud. Amazon Redshift offers you fast query performance when analyzing virtually any size data set using the same SQL-based tools and business intelligence applications you use today.”

Redshift is a clear attempt to steal big business in big database from the traditional enterprise providers. In other words, Redshift targets competitors that pull billions out of enterprises through ongoing licenses and support fees.

To net some of those billions, Amazon has made it easy for you to test-drive Redshift, by incorporating into to the AWS Management Console, where you can launch a Redshift cluster. Clusters start with a few hundred gigabytes of data and scale to a petabyte or more, for less than $1,000 per terabyte per year. (I suspect most enterprises will blow well past a terabyte, so estimate your likely usage before you make the move.)

Most enterprise hate to pay those huge fees to support huge databases. In the past, there were no other options. But now there is. If Redshift lives up to just a portion of the expectations set by Amazon, it could force enterprises to rethink how they consume database technology. The cost benefits alone could drive some quick changes.

What may keep many enterprises off Redshift is the fact that most of those charged with maintaining data warehouses in larger enterprises are not big fans of cloud computing. They consider the technology too new, insecure, and unreliable.

On the other hand, if Redshift can do what the on-premise databases can do at a fraction of the budget, those concerns will evaporate — or be overruled by business units. In that case, bye-bye Oracle, hello guys who also sell books.