It’s recognised that big data not only exists but that it has great potential to disrupt well-established vertical markets. Google commented on the fact that every two days, users generate as much information as was created between the dawn of civilisation and 2003, says Davide Hanan, MD at QlikView SA. 
Cisco predicted that by 2015, there will be 15-billion connected devices in circulation. While, until recently, the big data phenomenon was a hype generated by the technology industry and media, it has been rapidly unfolding and growing behind the scene.
How do users know this? Because all industries and sectors are expected to have realtime analysis of what’s happening. In the same way that customer service has evolved to be 24/7, realtime data analysis for better opportunity and anomaly spotting is something that the majority of us take for granted.
Let’s take a couple of examples. In retail for example, have users not come to expect Amazon and other online retailers to recommend our next purchase based on prior buying habits? It’s quite telling that Amazon claims 30% of its sales are generated as a result of its recommendation engine: “customers who bought this book also purchased”.
While most consumers now take it for granted that they receive personalised and tailored offers in the retail, utility and telecoms industries, it’s not as welcome in other vertical sectors, for example public sector or government-based industries.
Those industries and vertical sectors that have easy and available access to customer or industry data to compare and contrast should be taking advantage of this. At a time when cost comparison and running as leanly and efficiently as possible is crucial, all companies need to be looking for opportunities where they can also benefit and find growth. And many are.
As mentioned, the retail sector in particular is doing well out of analysing customer behavior and patterns and being able to offer consumers coupons almost in realtime. Two great examples are UK-based retailers, sandwich shop EAT and retailer giant Sainsbury’s.
Using business intelligence technology to pull together data on ingredient purchasing, weather, store visits and staffing, EAT has been able to purchase seasonally and staff according to demand. The other example is Sainsbury’s, which is using big data to help it set prices, nearly in realtime, and shift inventory by giving loyal shoppers customised coupons.
There are definite similarities to be drawn between the retail space and the telecoms and mobile industries. What the latter has an advantage on, however, is the consumer and user data. Thanks to (mostly) long-standing customer relationships, the billing history can throw up a huge amount of insight on the individual mobile user and their habits.
In fact, this information is so valuable that European mobile giant Telefónica last year was set to launch a global big data business unit aimed at selling information on customers using its mobile services.
Some of the immediate benefits on offer would have appealed to the likes of local councils who want to measure how many people visit the high street after the introduction of free car parking, farmers’ markets or late night shopping for instance.
What the big data potential does demonstrate is an opportunity for open innovation. By empowering all the users within a business, innovative transformation should theoretically become feasible. Not only do organisations in all sectors have to remain agile and able to adapt quickly to stay ahead of the competition in their specific vertical sector, but also companies’ traditional roles need to evolve.
This comes back to the diversification challenge and opportunity – and several failed attempts.
As touched on earlier, the private sector has arguably greater licence to analyse user patterns to better tailor and personalise services, because customers opt in to a brand. That said, governments and the public sectors around the world are not letting the big data opportunity pass them by.
Police departments are using computerised mapping and analysis of variables like historical arrest patterns, paydays, sporting events, rainfall and holidays to try to predict crime ‘hot spots’ and deploy officers there in advance with big data analysis. From Stockholm to the UK and across the US, police forces are being very effective in managing their resources for better deployment.
Opening up data is key to this, as is being transparent in how data is being stored and analysed. The UK public sector is driving this with ‘Open Data’ initiatives, giving interested parties access to data and, with the right tools, there’s no reason why users cannot draw their own analysis and reasonings, making for a more engaged society.
Which bring us back to what big data is and where its responsibility sits. One thing big data giant EMC advises against in any case is treating any initiative to address big data as an IT issue.
While the IT department and so-called data scientists have an important role to play in enabling access to data access and then drawing analysis from multiple rows of data, it is down to the business users to spot anomalies and opportunities from the data available to them.
The more users can refer to big data as information that is available and the business insights harnessed from the analysis the opportunity for different vertical sectors all across the globe, the better placed they will be to take advantage of what big data ultimately enables.
The best big data is the data generated as a by-product of operational, customer and supplier processes. The data that people naturally share, and are willing to, in return for a better experience or end product. And the best big data is when it becomes information that is readily analysed by business users for useful insights.