Smartphones and cloud applications are adding to the creation of data which must be organised and interpreted. Photo: Tim Chong
The collision of consumers, business and information on the internet has created the latest IT buzz phrase: Big Data. Big data is everything out there online, in the cloud, in the air.
As with many buzz words, the definition can depend on who requires it. For a chief executive, big data can be a headache; for a technology officer, it's a challenge (and also maybe a headache) or for an analyst, it's some kind of paradise.
Werner Vogels, a data guru as chief technology officer for Amazon Web Services, has been touting his interpretation of big data for almost two years. For him, managing a behemoth like Amazon, it's not exactly what big data is, but what can be done with it.
“Big data is the collection and analysis of large amounts of data to create a competitive advantage,” he told a conference earlier this year.
“I am an infrastructure guy and for me big data is when your data sets become so large that you have to start innovating how to collect, store, organise, analyse and share it.”
For Vogels, big data is all about opportunity: “If only [businesses] could process all the data that they are sitting on, that could give them a competitive advantage - with the idea that bigger is better.”
To demonstrate just how much bigger can be better, Vogels recommends to audiences they type into Google the search term “funny amazon recommendations” and see the suggestions. The results expose a business weakness and provide the benefit of harnessing big data for an e-tailer like Amazon.
“There is not enough data to provide accurate [consumer] recommendations,” he said. “As such, cloud computing and big data are closely linked, because for a company to be able to collect, store, organise, analyse, and share data, they need access to infinite resources.”
According to research by Gartner, big data emerged as a significant challenge for the IT industry soon after it became popular in 2009. Gartner reports that in February, 2011, a Google search on "big data" yielded 2.9 million hits. Today, that same search turns up 25 million hits.
Gartner's somewhat dry definition in a July 2012 report states that big data “is high-volume, [high] velocity and [high] variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.”
In a blunt suggestion to business leaders, the authors, Mark Beyer and Douglas Laney, wrote: “The most important aspects of big data are the benefits that can be accrued to the organisation. The only reason for pursuing big data is to deliver against a business objective. For example, knowing how a large population responds to an event is useless unless a business organisation can benefit from influencing that event or its outcomes.”
Perhaps, though, the simplest definition of big data is that it is information that can help you answer small questions.
Richard McLaren, the chief data and technology officer at Australian digital media company Mi9, a joint venture between Microsoft and Nine Entertainment, told ITPro the advantage of harnessing big data correctly is the detail it can provide.
“The definition is so amorphous that it can only be something that we don't properly understand,” he said, perhaps only half-joking.
“But it is not just the availability of the information or the rate of the information. It is the competitive advantage that allows us to ask simple questions about our consumers, about our business, about our content, and allow us to get extraordinarily insightful answers. Answers that are, we use the phrase, 'otherwise unknowable'.”