Talking about the future of anything is like trying to identify a specific type of flower from 100 feet away. You can see that it is a flower, its basic shape, but there is no way to tell exactly what it is.
It is why when Big Data was first conceived in its current form, it would have been impossible to comprehend where we would be now. This means that predicting the exact future of it can never be 100% accurate, but we believe that there will be two key areas that are going to see significant changes.
Data privacy used to be a simple thing. You received people’s data, then once you had used it, you deleted it. Today this is not so clear cut as the use of the data is likely to go well beyond the initial use case. For instance, using historical data to predict future outcomes is the entire basis of predictive analytics. However, this requires the data to essentially lie dormant for an indeterminate amount of time.
This goes against what current data guidelines suggest, which means that future guidelines and laws will need to amend this. What these rules dictate will have a significant impact on the way that companies and governments use their data. We have seen with some of the court cases being brought against governments that this kind of mass data mining is not necessarily appreciated and this ire towards unsolicited data collection may turn to companies in the future.
Clear laws not only about the collection, but also the storage and distribution of data are going to be key to the overall success of any future business use cases for Big Data. We have seen the damage that hacks can have on people not only financially, but also emotionally, with the hacks on both Target and Ashley Madison being prime examples.
Speed & Volume
At present, data is collected quickly. Every time you do anything on the internet information is gathered about you and your actions. However, when you compare the amount of information being collected today to only 3 years ago, it is almost unimaginably more. In the coming years, this pace of change is only likely to increase, meaning that by 2020 we are going to be seeing a significant increase in the amount being collected and an improvement in the technology being used to collect and sort it.
Frameworks will need to be more robust and the analytical process will need to be significantly improved in both speed and accuracy. With the use of in-memory databases and platforms such as Apache Spark, this adoption of quick and robust technology means that companies are aware of this necessity whilst software providers are launching the foundations for technology to make it a possibility.
Alongside this huge influx of new data and speedier analysis is going to naturally be quicker and more complete sets of conclusions and actionable, processed data. This change is going to allow boards to become better educated and more agile, making important decisions in a fraction of the time, with increasingly reliable data. As the influx of millennials into the workforce continues, boards are going to become increasingly led by more progressive and technically savvy people, which will allow this kind of thinking to become the norm, rather than the future.