It has been a year of great upheaval in the business community, and data analytics has seen its fair share of change. Big data may have fallen off Gartner’s hype cycle, but data science is in, and companies continue to invest heavily in both the technology and the skills needed to exploit all the information they are gathering. There have, however, been a number of setbacks. Significant data breaches saw the personal details of millions of people stolen from the likes of TalkTalk and Ashley Madison - major companies which prided themselves on the security of their systems.
In 2016, there will be new challenges for analytics practitioners to deal with - and new technologies, new methods of practice, and new ideas to help overcome them. We’ve outlined our predictions for what 2016 will bring analytics.
Greater Data Democratization
Data democratization has many benefits. It enables a company to take a holistic approach to data analytics that helps it become more agile, and drives faster, smarter decision-making processes across the organization. Gartner has predicted that by 2017, most business users and analysts will be able to access self-service tools that prepare data for analysis.
Central to the push for data democratization is data virtualization. Data virtualization is any approach to data management that allows an application to retrieve and manipulate data without the need for technical details about the data. According to Computerworld's Forecast 2016 survey, firms are set to increase the amount of money they budget for virtualization projects next year, with 35% of those polled saying that they were increasing spending, and 64% saying they were beta-testing or piloting some kind of virtualization across desktop, server, storage, mobile or network.
The Cloud Will Become The Home Of Data
Cloud data and cloud analytics have been threatening to take off for several years, but progress has been stilted thus far. In 2016, we should see the cloud become the home for data and analytics, with myths that previously held companies back from adopting it, such as security, being dispelled. The cloud is no longer just a deployment model, it’s an engagement model, bringing you closer to both internal and external customers. The speed of analysis that it enables, and its scalability, mean that companies which do not adopt are likely to fall behind with their analytics. IDC’s recent NA Global Technology and Industry Research Org IT Survey found that 71% of respondents are using, planning, or researching cloud solutions, while 64% cited the importance of having subscription access to software.
Smarter Data Governance
For data democratization and cloud technology to evade concerns around privacy, companies will have to adopt smart and transparent data governance in 2016. In the past, many have considered governance and self-service analytics to be natural enemies, but it appears that the disparity that existed between business and technology is narrowing. Good data governance can help nurture a culture of analytics that meets the needs of the business, by giving people the peace of mind to access and use the data. There is also likely to be a new raft of regulations brought it on a national level, as governments look to take control of their citizens’ data, in the same way the EU has done by overthrowing Safe Harbour.
Spark Will Take Over
Real time tests have shown Spark to sort 100TB of data in just 23 minutes. Hadoop took 72 minutes to achieve the same results. Spark also did this using less than one tenth of the machines - 206 compared to 2100 for Hadoop. It guarantees up to 100 times faster performance for several applications, making it ideal for machine learning. Uptake to date has been exceptional, and it is still growing. According to a recent survey of 2,100 developers by Typesafe, 71% of respondents say they some experience with the framework. It has now reached more than 500 organizations of all sizes, who are committing thousands of developers and extensive resources to the open source project, and this is likely to increase further in 2016.
AI And Deep Learning For Business
Given the volume of data available, and the need for ongoing, real time analysis, the use of AI techniques and processes to deal with it is expected to mushroom in coming years. There have been substantial advancements in AI and deep learning by the major tech firms over the past few years, but its use in business has been more limited. Advances in sorting speed, enabled by software like Spark, should soon see this change. IDC predicts that the global market for content analytics, discovery and cognitive systems software will reach $9.2 billion at a Compound Annual Growth Rate (CAGR) of 15% by 2019 - double what it was in 2014.
Deep learning will replace much of the data analysis that was previously done by humans, which will mean people having to operate at a higher level and work on things like accelerating their data strategies. Deep learning will also play a central role in cyber security. According to an FBI official quoted in The USA Today, more than 500 million records have been stolen from US financial institutions over the past year as a result of cyber attacks, and the average consolidated total cost of a data breach was $3.8 million according to IBM - up 23% on 2013. Deep learning can flag network anomalies, track user behavior, and detect zero-day malware. Anti-virus company Invincea, for one, will add deep learning-based capabilities to its end-point security product in 2016, and others are developing similar solutions.
Information of Anything
According to Gartner, by 2020 there will be 25 billion devices generating data about nearly everything imaginable. Companies that manage to make sense of this avalanche of data first will have a huge competitive edge, but firms are also increasingly having to look beyond the information produced by devices, sensors and machines. They are now looking to incorporate ALL data, such as that produced by server logs, geo location and data from the Internet. This provides a huge challenge, and firms will need to ensure that they are prepared.