6 tips for working with IoT and big data

Is your company looking to get started with the Internet of things and big data, or are you looking to improve on how you’re handling it now? Here are six tips from the pros that should help anyone:

Use the right people. Data scientists are in short supply and get very sizable salaries. But you don’t need to hire data scientists, says Andrew Brust, Senior Director of Technical Product Marketing and Evangelism at Datameer, a big data Analytics and Visualization company. Instead, look at your existing staff for people with data warehouse and IT experience, and are willing to learn, and train them.

Be smart about data capture. Carefully design exactly how you’ll capture IoT data. GE, for example, uses small data-collection appliances that determine what kinds of data to collect, what protocols to use for collection, and how the data should be stored. And keep all of your data, even if you don’t know how you’ll use it, recommends Mike Maciag, Chief Operating Officer at Altiscale, which offers a cloud-based Hadoop platform. As your company strategies change, you may well find a need for it.

Provide an abstract data layer. IoT comes in many different protocols and data standards that aren’t always compatible with one another. Sometimes the data is highly structured, and other times it isn’t. Your best bet is to provide an abstraction layer that can handle multiple data types, including new ones you haven’t yet encountered.

Choose the right platform. Your company may not want to spend its time and money building a large data analytics platform on its own. Consider using one of the many cloud-based ones currently available.

Start with a small pilot, then build out. Intel’s Sharma says many companies bite off more than they can chew when taking on IoT big data projects. Instead, he says, start small with a pilot. Once you’ve got all the problems ironed out, roll it out to the rest of your enterprise.