In 2011, McKinsey released their 'Big data: The next frontier for innovation, competition, and productivity' report, which outlined the ways big data was going to impact the world and the implications of the changes. We are now five years on from the original predictions - five years that have seen big data become one of the most discussed subjects in society and the need for it in companies increasing exponentially.
So how accurate was McKinsey all those years ago? In their research, they offered seven key insights into big data and the future of big data, and we wanted to look at how they compared to what actually happened in the intervening years.
1. Data have swept into every industry and business function and are now an important factor of production
This was an accurate prediction as data has now become adopted across practically every industry. Not only that, but it has been the catalyst for industries themselves and companies like Hortonworks have been valued in the billions. However, it is not necessarily as widely used as some may think as, according to a survey from the economist, 58% of companies are making limited progress in big data adoption.
However, there are some industries who have stormed ahead in this regard, with telecommunications having 62% of companies in an advanced stage, retailing with 68% and IT and technology with 57%.
2. Big data creates value in several ways
This prediction involved several different aspects, each of which has come to fruition, these include:
- Creating transparency
We have seen through the opening up of government data and company records that big data has certainly created a more transparent society.
- Enabling experimentation to discover needs, expose variability, and improve performance
Again, this is certainly the case, we have seen that companies using data effectively report significant gains. Data has also allowed companies to make improvements in design, from Ford utilizing it to improve car design through to web designers using heat maps to optimize sites.
- Segmenting populations to customize actions
This was well underway when the report came out, with the prime example being the Democratic party using voter data effectively to target specific narrow demographics.
- Replacing/supporting human decision making with automated algorithms
AI and machine learning may not be at the level we know they will be, but the development has been rapid and is only going to speed up. We have seen almost every major tech company experimenting in this area.
- Innovating new business models, products and services
Data has allowed companies to find new markets, alterations to products and the creation of totally new products through the way that people use existing products today.
3. Use of big data will become a key basis of competition and growth for individual firms
This has more or less been and gone, to the extent that the companies who jumped in early have found significant benefits and now those who don't will always struggle to even enter many markets. There have been several court cases about the theft of data, with a prime example being when fitness tracker makers Jawbone sued their competitor Fitbit, as they poached staff members who then stole data from the company upon leaving.
4. The use of big data will underpin new waves of productivity growth and consumer surplus
This point essentially discusses the indirect benefits of big data, which is hard to quantify but is certainly correct. One of the examples that is given is the use of traffic data being fed back to a user who then saves time on a journey, which is exactly what we have found with a number of traffic apps on phones and satnav units.
5. While the use of big data will matter across sectors, some sectors are poised for greater gains
Once again, this is certainly correct and we have seen it with retail and financial services in particular. The reasons for both of these are varied, but one of the common elements is the transactional element of both businesses, making the use of data much simpler than more complex industries like manufacturing and healthcare. Due to this, they have become two of the top performing and strongest adopters, whilst others don't have quite the same level of maturity.
6. There will be a shortage of talent necessary to take advantage of big data
To some extent this has occurred, but not to the levels that many had predicted. We have seen through the pure number and breadth of companies who we have been involved with that, although there is certainly a gap, it hasn't really stopped companies taking advantage of data. Thanks to new consultancies, easy UX for data platforms and SaaS platforms, to some extent companies have alleviated the absolute need for the in-house data scientists that are lacking.
7. Several issues will have to be addressed to capture the full potential of big data
The healthcare industry is the prime example of this prediction coming true. The potential that data could have on the industry is huge, but until there is a way to effectively prevent the identification of individuals it simply isn't possible.
However, the largest area of improvement needed is simply to get people to trust companies and governments with their data. We saw with the recent Safeharbor changes that even the courts don't believe enough is being done to protect data from the US government and the theft of data from Ashley Madison saw the destruction of hundreds of thousands of lives. Until the issue of effective protection and safeguarding is addressed, it will be impossible for the full potential to be realized.