When Did We Hit Peak Big Data?

Have we hit the peak and are we seeing a decrease?


Big data is not new. When I first came across the concept a decade ago, I needed to explain it with real intricacy to anybody who asked, describing the various ways that it was going to impact their lives. Today, it is part and parcel of what people understand. In a conversation it no longer requires any kind of explanation, it is a concept that the vast majority of people know.

We are, in many ways, now living in a post big data world. We used to write that those who hadn’t yet adopted big data needed to do so soon or they would soon become irrelevant. Today we can look back at the companies who didn’t heed the warning and are now no longer in existence or have fallen back into irrelevance. Think about the Blockbusters, Love Films, and Woolworths of the world. It is clear that one of the key reasons they all failed was an inability to adapt to the new landscape that quickly unfolded in front of them. A part of this was big data adoption, but an even larger part was the ability of big data to help identify these changing trends.

However, today this is simply not the case, there are practically no large companies who don’t use data, don’t forecast, and don’t utilize data insights in their strategy. The very term big data now seems relatively archaic, with the individual parts that once made up the overall big data landscape becoming industries in their own right, such as AI, machine learning, and data warehousing.

So when exactly did we hit peak big data?

To investigate this we took a look through Google Trends, which analyzes search term popularity over time. One thing that is interesting is that searches for big data are currently peaking today, but the element that tells us that the peak was before this is because the countries pushing this search terms popularity are Singapore, India, and Hong Kong. Although each have a relatively advanced economy, we need to look at the US for the peak, as they are the most advanced data economy in the world and generally set the agenda for the rest of the world on these issues.

The data shows that the uptake in people searching for the term started in Q4 of 2011, when the quantity of searches nearly tripled from 9/100 to 37/100 in 12 months. In 2013 this number peaked at 69/100, before hitting 91/100 in October 2014, then having peaks and troughs between 2014 and 2017, with March 2017 hitting 100/100. What this graph clearly shows, is that there is a clear plateau in the number of people searching for the term.

This plateau began in March 2015, which saw 95/100 and has only been bettered 4 times in the following 2 years, suggesting that the time of growth may be over. We could therefore argue that March 2015 was where we hit ‘peak big data’.

Looking through Google Trends also threw up some interesting insights that produce an interesting timeline. This timeline suggests the dispersal of big data to the more separated fields that we are more familiar with today, acting as elements within big data to some extent, but broadly seen as independent fields. For instance, we can see that machine learning, although not currently at the same level as big data, has been experiencing consistently steep growth since January 2015, currently sitting at 77/100 compared to big data’s 100/100, but it may soon overtake given the upwards trajectory.

Using Google Trends as a marker may not be the most scientific way of analyzing when big data hit its peak, but does give us some indication of how many people are searching for it, which seems to have hit a bit of a plateau in the last 24 months. What we can see though is that as the umbrella term has begun to plateau, more specific terms have grown in popularity, so although the use of ‘big data’ may soon start the shrink, the use of big data data looks like it still has a bright future. 

Looking small

Read next:

Expert Insight: 'An Effective Visualization Results From A Great Deal Of Curiosity And Exploration'