Is Big Data Better Data?

February 10th, 2014

Is Big Data Better Data? Ellis Friedman BurrellesLuce Fresh Ideas

What do you think when you hear “Big Data?” If you’re anything like me, you hear some syllables that conjure up an image of lengthy Excel sheets. If you’re more precise, you know that it’s all that data that organizations and computers collect on a daily basis.

But is bigger data better data? There are a lot of factors that determine whether investing in big data will actually help your organization and its efforts.

The biggest issue is that big data can easily turn into GIGO – Garbage In, Garbage Out. You may be pulling in gobs of data every day, but if you don’t know how to use it, don’t get to see it, or don’t know what your most important metrics are, that data is all but useless.  Harvard Business Review has an excellent article (registration required) tracking seven case studies of companies that used their data, how they did it, and whether they used that data effectively or not.

The gist? Using big data – and even small data – effectively takes a lot of preparation, implementation, and adjustment. In many instances, it won’t pay off to jump right into big data if you don’t yet have a grip on your small data. Here are some key takeaways, whether your data is big or small:

Start small

Start by learning to use the data to which you already have access. Most existing CRM or ERM systems obtain a lot of useful data, but many organizations don’t know how to access and/or use it. According to Kapow’s survey, only 23 percent of respondents think their big data initiatives have been a success. So before you go investing in big data, think about what you’d like to do with that big data once you have it. Whether its reforming processes, reorganizing, or restructuring, use the data you already have to start small reformations.

Dedicate resources wisely

Be prepared to dedicate a significant portion of your resources to coaching and training. HBR found that the most important factor in successfully becoming a fact-based decision-making organization was consistent, continuous coaching aimed at improving performance of every individual, especially those who are decision-makers.

Provide real-time feedback

Start providing daily feedback before trying to implement new big data changes. Determine one key metric to focus on (the metric will be different for different departments and different levels) and provide the department with the updated metric every day. HBR found that this not only helped managers determine how to best spend their time, but caused those at lower levels to increase precision and efficiency. Just make sure it’s the right metric; it may take some finessing.

Shift the culture

An organization will not magically change by virtue of investing in big data. HBR found that if the organization had a tradition of fact-based decision making, performed engineering and research functions or was web-native, then it was poised to gain the most from big data.

So don’t just look at the shift to big data as an investment and software issue; instead, organizations need to consider it a major shift in company culture. Like any major culture shifts, it will take a while. Give it time, and allow any revised data processes the leeway to produce flawed data – it will improve, and those involved with the data will, with the proper coaching, seek to improve that data.

According to the Kapow survey, 85 percent of business and IT leaders agree that big data helps make intelligent business decisions and foster a data-driven organization. And for organizations that are data-driven and fact-based decision makers, there is a lot of potential in big data.

Last month I attended the Big Data in Motion Summit, where the speakers were Jack Norris, chief marketing officer for MapR Technologies; Pat Pruchnickyj, product marketing director at Talend; and Clarke Patterson, senior director of product marketing at Cloudera. All speakers expressed enthusiasm for the impact big data can have on organizations. And little wonder – they all work for companies that provide big data solutions.

The conference was intended to educate attendees about big data’s potential, results, and myriad of advantages, though the speakers mostly talked about platform options and advantages of their own services. Patterson pointed out that 64 percent of organizations invested or were planning to invest in big data in 2013, so of course, getting the down-low on services is pretty necessary.

Norris explained that the need for big data is driven by three V’s:

Volume (by 2020 enterprise data volume will be four times higher than it was in 2009)

Variety (data is both structured and unstructured and gathered from a myriad of devices, processes, and sources, and stored in different ways)

Velocity (large organizations produce massive amounts of data. Facebook gathers 100 terabytes per day, WalMart has 1 million transactions per hour)

There are plenty of organizations that stand to gain from big data if it’s implemented wisely, but it’s not the “big” part of data that provides benefits – it’s learning to use data big or small in the correct way. No matter the size of your data, you still need to know your key metrics and how to base decisions around those metrics. The opportunities data provides come down to leveraging the data you have into powerful insights and harnessing it in an efficient, fact-based way.

Leave a Reply