Powering real-time marketing with big data analytics

With more than $300 million in annual revenues and 1,600 employees, Merkle is a leading customer relationship marketing agency for such clients as Dell, Geico, DirecTV, and Chase. The company uses ParAccel‘s big data analytic platform to gain a 360-degree view of consumers so that Merkle’s clients can work toward real-time marketing and measure campaign effectiveness more precisely.

Merkle has a long history of curating consumer information to provide “data as a service” for marketing purposes. Historically, this was achieved through monthly batch processing of huge flat files.

[ Download InfoWorld’s Big Data Analytics Deep Dive for a comprehensive, practical overview of this booming field. | Harness the power of Hadoop with InfoWorld’s 7 top tools for taming big data. ]

In order to evolve from a marketing database company to a customer relationship marketing firm, Merkle needed to reconcile and integrate big data sources. Digital consumer information such as IP addresses, cookies, and emails had to be merged with traditional offline information such as name, address, and phone number. Ultimately, clients needed deeper marketing engagement, such as emails and banners customized for individual consumers.

Reaching for real time To meet those goals, Merkle needed processing to move from monthly batch jobs to near-real time, with the ability to integrate all interactions and achieve a 360-degree view of each consumer’s behavior. To achieve these goals, Merkle builds data warehouses for big data analytics, some of which are deployed at its clients’ sites and some of which are hosted by Merkle.

Merkle’s challenges in selecting appropriate technology included the cost of a big data analytics environment, predictable high performance and scalability, and specialized requirements unmet by current analytics offerings.

Ultimately, Merkle chose the ParAccel Analytic Platform first for its efficient, scalable architecture as a massively parallel processing (MPP) columnar analytic database. “Merkle originally selected ParAccel because of winning execution speed and price performance,” comments Peter Rogers, VP of Technology at Merkle.

For real-time analytics of structured big data, MPP columnar analytic databases have become a common choice. Columnar storage means that the relational database houses data in columns rather than in rows, yielding faster queries than in transactional systems. In addition, the data is compressed to further increase querying efficiency. Finally, due to the nature of MPP, you can “scale out” in linear fashion by simply adding more commodity hardware.

Stretching big data dollars The affordability stood out for Merkle: At about $4,500 per terabyte, the ParAccel Analytic Platform came in at a fraction of the tariff demanded by competitors. That cost advantage made storing and analyzing data more affordable, which has resulted in a 300 percent growth in data processed since switching platforms. Coupling the higher capacity with the more than 500 sophisticated, easy-to-apply analytic functions built into the ParAccel platform, Merkle can offer more complete solutions to its clients.

One of Merkle’s specialized requirements involved leveraging existing skills in T-SQL, the query language used by Microsoft SQL Server. Merkle already used Microsoft SQL Server extensively and wanted its data management professionals to use ParAccel with minimal additional training. ParAccel’s professional services delivered a T-SQL parser so that developers can write jobs in T-SQL, which are transparently translated to the variety of SQL that runs most efficiently on ParAccel.

On the front end, Merkle uses MicroStrategy for visualization and business intelligence. For data integration, Merkle loads data using native ParAccel tools so that all transformations are performed within ParAccel. This ensures high performance and enables Merkle to hit its data loading windows by obviating the need to transform data on intermediate servers.

Today Merkle has five ParAccel clusters maintaining a total of 50 terabytes of compressed data (200 terabytes of raw data). On a daily basis, the company processes 1GB to 250GB of raw data, depending upon client demand. And those clients are happier, thanks to an integrated, 360-degree view of customers based on real-time interactions — along with analytics that enable more precise measurement of the effectiveness of campaigns.

This article, “Powering real-time marketing with big data analytics,” was originally published at InfoWorld.com. Read more of Andrew Lampitt’s Think Big Data blog, and keep up on the latest developments in big data at InfoWorld.com For the latest business technology news, follow InfoWorld.com on Twitter.