With 2016 drawing to a close, we can look back on what can only be described as a mixed year for data analytics. The polling data in the run-up to Brexit and Trump’s election, for instance, have seen many dismiss the power of data, with Mike Murphy, a Republican strategist who predicted a Clinton win, saying after the election, ‘My crystal ball has been shattered into atoms. Tonight, data died.’
However, reports of data’s demise are premature. Data analytics has not spent the last several years being exulted by organizations of every hue for no reason, and a recent IDC report still had the big data and business analytics market growing at a rate of over 11% in 2016 and at a compound annual growth rate of 11.7% through to 2020.
In 2017, there will be new challenges for analytics practitioners to deal with. There will also be new technologies and new ways of working to help overcome them. We’ve outlined our predictions for what 2017 will bring analytics.
Unstructured Data Will Dominate The Analytics Landscape
Unstructured data is any data that does not fit into relational databases. It is estimated that 90% of all data is either semi-structured or unstructured. This includes videos, powerpoint presentations, company records, social media, RSS, documents, and text - all of which are vital to understand for businesses. While structured data analytics describes what’s happening, analysis of unstructured data gives you the why.
However, much of this wealth of valuable insights is currently going untouched. In a 2015 IDG Enterprise study on big data and analytics, 83% of IT professionals who responded said they have made structured data initiatives a high priority for their organizations, yet just 43% said the same of unstructured data initiatives.
The reason for this is simple. The tools needed to analyze such a large scale of data have not existed. However, machine learning and data visualization tools are now making it possible. In 2017, with these tools improving exponentially in quality and decreasing in cost, we expect to see far more companies putting unstructured data at the top of their agenda.
Embedded Analytics Set To Take Off
Embedded analytics is the fastest growing area of Business Intelligence (BI). In a study from self-service analytics firm Logi Analytics, more than 66% of IT teams said they are now using embedded analytics in their organizations, while almost 30% said they were considering it.
Embedded analytics consist of any consumer-facing BI and analytics tools that have been integrated into software applications, operating as a component of the native application itself rather than a separate platform. Embedded analytics allows end users to utilize higher quality data because the standards of governance are improved. They can also pinpoint insights quicker as time is not wasted requesting reports from external agents, and it allows findings to be distributed to those who need it across the organization.
The popularity of embedded analytics has grown exponentially over the past several years, and we expect this curve to go up over the course of the next year. Logi Analytics’ study found that business users are now adopting embedded analytics at twice the speed they are traditional BI tools, while Gartner’s 2016 Embedded Analytics Report also recently found that 87% of application providers claimed embedded analytics is important to their users, up from 82% in 2015.
The Data Scientist’s Role Evolves
According to a Forrester survey, businesses will invest 300% more in artificial intelligence (AI) in 2017 than they did in 2016. This has significant ramifications for analytics, with machine learning able to analyze data at a scale humans simply couldn’t. As Forrester notes, it will ’drive faster business decisions in marketing, e-commerce, product management, and other areas of the business by helping close the gap from insights to action.’ In their 2015 survey, just 51% of data and analytics decision-makers said they could easily obtain data and analyze it without the help of technologist, yet they anticipate this rising to 66% in 2017.
But does this mean the end of the data scientist in 2017? Another recent poll from KDnuggets asked when most expert-level Predictive Analytics/Data Science tasks currently done by human Data Scientists will be automated. A not insignificant 51% of respondents said that they expect this to happen within the next decade, while just a quarter said they expect the process to take over 50 years or never. Joel Shapiro, Executive Director of the Program on Data Analytics at Kellogg's School of Management at Northwestern University, notes: ‘In the right cases, data can be automatically generated and analyzed. But analytics is fundamentally about using analysis to do something differently. I am very skeptical of off-the-shelf analytics products that claim all you have to do is load in your data and it will spit out actionable insight.’ In the short term, data scientists are unlikely to be replaced, however, as more of the traditional reporting and queries are carried out by AI, we expect to see many data scientists see their role become more creative over the next year.
Behavioral Analytics Advances
The amount companies spend on digital ads is expected to grow to as much as $77.37 billion in the US alone next year, and understanding the audience is vital to ensuring this is money well spent.
The ability to predict someone’s personality presents a clear opportunity for targeting advertising, enabling marketers to segment audiences according to personality type rather than by age or gender, which is crass and highly unreliable. The benefits of personalization are well documented, with a 2015 Harris Poll study finding that 95% of respondents would be more likely to respond to personalized outreach. The Aberdeen Group also found that agencies best at personalization achieved up to a 36% higher conversion average and a 21% stronger lead acceptance rate.
Psychologists have attempted to understand different personality types and behaviors using checkboxes for decades, and digital marketers now have a significant amount of data about their customers available that could enable them to do the same. In a media release earlier this year from Universiti Teknologi Malaysia, research scholar Dr Ikusan R. Adeyemi said, ’Our research suggests a person's personality traits can be deduced by their general internet usage,’ and it could do so using machine learning algorithms by analyzing just half an hour of web browsing. With marketers now more au fait with analytics and tools that can judge people’s personality increasingly refined, next year should see a real drive in the practice.
Prescriptive, Not Predictive, Analytics Rules The Day
Predictive analytics have dominated the data landscape this year, but they can only take you so far. In 2017, more companies are going to start looking at prescriptive analytics, with Gartner predicting the market to grow to $1.1 billion by 2019 - 22% CAGR from 2014.
Prescriptive analytics uses the insights revealed by predictive analytics and provides a call to action based on what it finds. It analyzes current data sets for patterns and evaluates the outcomes of the multiple scenarios that could be enacted based on decisions that could be made based on the data, providing decision makers with hypotheticals as to the impact of each option. Just 10% of organizations currently use some form of prescriptive analytics, according to Gartner, but this will grow to 35% by 2020, and with the increasing buzz we have seen already this year, it seems likely companies will look to implement prescriptive analytics in their droves next year.