Tomorrow’s Analytics

How has data analytics changed over the past 25 years?


25 years may seem like a long time ago – Douglas Adams completed the last book in the Hitchiker series, South Africa formally ended Apartheid and Euro Disney opened! On the other hand, the Tories only just managed to win a general election so maybe nothing much has changed…

One significant thing about 1st April 1992 was that Capgemini created its Analytics team. My mathematical modelling team was brought over from British Coal which had a long history itself, formed soon after the company was founded in the late 1940s. 25 years on, I am still with the Capgemini Analytics team, and at an event to celebrate the anniversary, I found myself chatting with some analytics colleagues – Lee Brown and Yigit Gungor.

Lee started in 1996 on analyzing data – back then models were handcrafted in Structured Query Language (SQL), and could only be based on limited sets of data. He joined Capgemini in 2005 from a small team that focussed on data warehousing and business intelligence. His team focussed on the ways to manipulate data, and their success saw a data team of about 20 people evolve into the officially titled Insights and Data practice with over 400 members.

Yigit is the baby of the group – one of a new breed of data scientist and someone whose career started after the concepts of data work were being developed. His former group built forecasting models and helped to generate more revenue for clients.

The early part of our chat focussed on what has changed over the last 25 years. Computer power has increased, the prediction of Moore’s Law has proved generally correct, and we can certainly do much more with our computers now than we ever could. Driven by the introduction of smaller and more powerful computers, we began to see the ‘democratization’ of technology as people started to use mobile and now wearable devices. They then started to interact with programs and applications, which in turn generated yet more data.

Software has also improved for consumers and the data analytics community alike. 25 years ago, we had basic spreadsheets to handle simple calculations, and some had access to complex mathematical software packages for bigger problems. Since then we have developed sophisticated data visualisations which help us to truly understand what is going on, and computers that enable us to quickly use multiple analysis methods on vast data sets to determine causation and make predictions.

Yet in some sense, the applications of analytics remain similar. My last job working for British Coal was to analyse the life and subsequent cost of heavy-duty coalface machinery. And Capgemini still does analytics on asset industries, looking at the life and cost of things like rails and pylons. However, what has changed is the way that data can provide insight. 25 years ago, the work that I did showed the historic costs and expectations of heavy-duty machinery. However, what we can now do is not just look at what has happened in the past, but start to look forward to what may happen in the future – we can assess pylons, rails, and water pipes to predict which ones might fail, and therefore which to replace.

Lee’s view is that the market is currently challenged with organisations wanting to become more and more data driven. Companies are not looking for insight for the sake of it, but for insight that will drive business benefit and transformation. As it has become easier to manipulate the data and create complex models, the focus has moved from the skill of doing the technical things, to creating the links with business – providing the insight that they need. Customers now have the choice of commercial or open source analytical products but the best solutions often incorporate both, although this requires work on setting up the standards and tools to integrate different things together in a seamless way.

Skills are more important now too. 20 years ago people who manipulated data were just complex programmers, nowadays there are whole different categories of data analytical roles, from Data Scientist to Data Engineer among others. The job of Data Scientist was recently named by Harvard Business Review the sexiest job of the 21st century, although even here there is contention with Data Engineers claiming that they are just as important.

So how do companies embed these things within their organisations? That’s what drives Lee. Sometimes it takes a proof of concept – Lee recently ran a project where the application of technology and analytics showed a client in 2 weeks what they had been studying for 30 years. It’s the ‘penny drop’ moment that we need to get true transformation within companies. Some firms are up for this – Transport for London only recently released a study into whether or not escalators should be standing only – but the fact that TfL are looking at this sort of thing is evidence that some organisations are already developing a different mind-set.

Yigit takes this on – the way that we need to embed these new tools and techniques is to use and develop a different mind-set of consultants and then employees. 20 years ago we needed people who had strong technical skills to utilise complex and non-interacting programs. As this became easier, we saw the different data jobs providing expertise in manipulating the data to provide insight. However, the final piece of the jigsaw is to have people who can understand all of that, and how to apply it to business problems - This is how businesses will succeed with analytics. Although these concepts are relatively new, we can combine them with other business change techniques to make a powerful force that can be applied within organisations.

One thing that interests me is how society will adapt to the changing world of data. We all agreed the world is becoming more personalised. Data tools and techniques mean that companies can now understand the cost and benefit at an individual level and provide services accordingly. Yet, society is undecided on this level of personalisation: Personalisation benefits some, but may cost others. And society having to grapple with whether or not this is right. Insurance provides some good examples here. In 2012 the EU introduced rules to say that car insurance companies cannot differentiate between males and females. Even though women were likely to have cheaper quotations, companies were forbidden from offering these to them. In a similar way, there is real concern in the USA that allowing insurance companies to personalise healthcare will lead to a ‘death spiral’ as people in certain groups will be isolated, and this will lead to larger and larger costs being applied to them, and so the insurance cover may become unaffordable.

Society often uses law to consider how to address these issues, but they move slowly, and may become too cumbersome to address concerns. The legal example is interesting. 25 years ago the law was starting to catch up with how to stop people infringing copyright by recording music off the radio – radio organisations often used to talk over the music to prevent this, but there were few prosecutions. However as the ability to infringe copyright has become easier (for example copy and pasting a picture from the internet) the way that this is regulated or enforced has become less clear. And what’s more, where regulation is applied it may not necessarily resolve the problem. In the case mentioned above, 5 years after the EU banned insurance firms from giving different quotes to males and females, the difference between rates offered to men and women has increased – likely driven by gender related things such as jobs rather than gender specifically. This demonstrates that using old institutions to solve analytical problems may not always work. As another example, finance companies are starting to implement artificial intelligence solutions. But how will these be regulated? Who is ultimately responsible for the algorithm – especially if they are artificial intelligence, and so could be said to develop solutions that are not directly human driven! These sorts of challenges will need to be met by politicians in the future, who will need to develop social solutions that can keep pace with technology. General Data Protection Regulation is another recent example that firms are starting to grapple with. We will have to see whether or not it will work…

We all agreed that the last 25 years had seen many changes in the analytics industry (well except Yigit who was only seven 25 years ago!). In that time the industry has seen major disruptions, some of which it led, and technologies, tools and techniques have evolved and will continue to do so. One thing that will remain constant however is the role of analytics to create a competitive advantage either through reducing operating cost or improving customer experience.

Iain Hubert – Senior Consultant, Analytics Team

Yigit Gungor – Senior Consultant, Analytics Team

Lee Brown – Head of Big Data Analytics

Looking small

Read next:

Expert Insight: 'An Effective Visualization Results From A Great Deal Of Curiosity And Exploration'