The Future Of Big Data

Where will we be in five years time?


The last five years have seen tremendous advances in our ability to collect and analyze data, with many organizations entering the higher echelons of maturity. One new study by Progress Software shows an increase in big data adoption among organizations from 50% to 61%, and with new technologies being introduced seemingly every day, it will like soon become accepted that all facets of decision making are driven by data in every industry.

The next five years promise even more exciting developments in the data arena. We asked eleven experts in the field from organizations leading the way with their data initiatives what they think will be the major advances over the next few years.

Shivanku Misra, Director of Data Science & Analytics at Heineken

Data is exploding, and the next 5 years will witness what one may call the renaissance of data science. New sources of data will generate an enormous amount of content, and unfortunately (or fortunately) there will be as much signal as noise in this data. With 'Internet of Things' and 'Artificial Intelligence', it will become a challenge for data scientists to isolate the 'Right Signal' at the 'Right Time' (vs. today, where we still consider isolation of noise from the signal a challenge).

Saket Kumar, Chief Data Scientist At Google

I am excited about the intersection of video/multi-media consumption and analytics. Image and video recognition is still work in progress. There is a lot of exciting stuff that can be done with respect to what ML sees in videos and actual consumption/interaction response of the consumers.

Ritesh Sarda, CIO at Sun Life Financial

It is all about using data to improve client experience. Developing the capability to interact with clients in a way that anticipates their needs takes some time to achieve. Robotics and artificial intelligence is a game changer and requires even more complex data transformations with natural language algorithms and machine learning capability. With the internet of things (IoT) getting immersed into daily life, there is lot more data produced by machines rather than humans and it requires the distributed power of big data platforms to capture huge data loads and make it available for analytics.

Saurabh Bhatnagar, Senior Data Scientist at Rent The Runway

Deep Learning will have a major impact on how we think about learning patterns in big data in the far future. But that will first happen in a few companies and slowly trickle down to others.

In the near future, common statistical techniques will be more automated. For example, to see if two trends are related, the correlation coefficient can be computed and a fit for each trend computed automatically. Things like that will get easier with smarter tools.

Nikhil Garg, Software Engineering Manager at Quora

I'm very excited to see more progress in the area of end-to-end differentiable network architectures. I feel that our rate of innovation will greatly accelerate if we could somehow also offload the learning of network architectures to machines.

Rob Finora from ShareThis

AI will significantly shorten the cycle around extracting insights or building advanced models, to the point we will see data become a disruptive force in the way business is conducted across many industries.

Miao Song, CIO, ASPAC at Johnson & Johnson

I think one is culture shift and the second is that there will be more utilization of external data. Taking the consumer industry as an example, I think there will be many opportunities in external data. For example, how you leverage data in social media to generate consumer insights, and how to eventually reflect that in your R&D, product development, customer behavior analysis, artificial intelligence, and machine learning and natural language processing to really work on data. Even in product design, we will look at consumer behavior and the external data. So I think it’s gonna be more external focused rather than internal focused in the next 5 years.

In the pharmaceutical space, the volume of data continuously mined and analyzed will help to set our strategy around clinical trial data. We are also going to collaborate with institutions or research organizations more closely. It is not just in the internal data, but the whole ecosystem that has to work together. In the medical space, I think IoT data, sensor data and the data we collected from devices will be much better utilized to help the practice develop and help business model development in the next 5 years.

Doug Ireland, VP of Finance at Prezi

Having data on demand in an organization is becoming a given in highly technical organizations. This trend will continue to expand to more traditional industries and smaller companies. The next step will be having the correct visualizations available on demand, too, to be used and reused in multiple channels and media of communication.

Tamara Gruzbarg, Senior Vice President of Data, Analytics and Business Information at Time Inc

I expect to see a continuous drive to uber-personalization: not only ads and e-mail messages, but actual products created ‘just for you’ (not customized based on your inputs but personalized based on your behavior).

I would also not be surprised if data starts being used in crafting content itself (including style, tone and sentiment). This would raise even more concerns around influence and manipulation of public opinion.

Gregg Bierman from Vistaprint

We will ramp towards the pervasive use of AI as standard practice in practically every type of public or private enterprise – so pervasive that we hardly think about it anymore.

Ken Cherven, Data Visualizations Specialist at GM

A lot has been made of the self-service possibility for all business users to instantly turn into data visualizers, but I don't see this materializing. What I believe will happen is the continuing evolution where different parts of the business begin engaging in data visualization at a greater rate. This will include parts of the enterprise that are not customer-facing, but would benefit greatly from more sophisticated visualization of their business metrics. Finance, IT, and engineering functions could all function more efficiently if they can move away from databases and spreadsheets and toward visualization platforms.


Read next:

Why Blockchain Hype Must End