Why Is Retail Experiencing A Data Scientist Shortfall?

Although it's a universal issue, retail is being hit particularly hard


Supermarket chain Sainsbury's is one of the largest retailers in the UK, with over 24 million customer transactions taking place per week. Like all retailers, however, it is now operating in an incredibly challenging space. People’s shopping habits are changing dramatically, with new and aggressive competitors entering the market and eating into their revenue. In view of this, Sainsbury’s is engaging in a huge drive to better understand its customers and personalize the customer journey, and is on the hunt for a data scientist to enable it.

The problem Sainsbury’s is likely to face is one that many across retail, and indeed all industries, experience. While it is true that there is a shortage of data talent across the spectrum though, it appears that the problem is particularly pronounced in retail.

The reasons for this are many and complex, but much of it comes down to the specific skill set required. Sainsbury’s criteria is for someone who can ‘solve complex retail problems with a modern data-science stack, work in a collaborative environment, support digital experience, DevOps and development teams, and utilize agile delivery with a focus on meeting business needs and driving value.’ Essentially, their skill set must combine business acumen, technology skills, intuition, and math. These qualities are not easy to come by all at once. It is sometimes easy to see data scientists as some sort of amorphous blob of skills, easily transferable from industry to industry because, if there’s one constant in the universe, it’s numbers. This is true to a degree - maths and computer science certainly are essential skills - but each industry requires knowledge specific to it in order to use the data as it needs, to identify how the patterns can be leveraged, and what the data is really needed for.

Ideally, retailers will have access to customer data which reflects how well they are reaching and nurturing customers, which it can leverage to improve the consumer experience, encourage repeat business, and make people more willing to buy their products. They build reports looking at metrics including conversion rate, average order value, recency of purchase and total amount spent in recent transactions.

It is not only customer data where they are useful though. They also need to be collecting, managing and analyzing supply chain data to enable executives to make more intuitive, accurate and reliable decisions, and adjust inventory according to real-time changes to supply and demand so that they can have what they need exactly when they need it.

It is not only an issue of struggling to find candidates with the requisite skills though, it is also a challenge to entice candidates. Tom Redd, global vice president of strategic communications at SAP Retail, identifies two reasons data scientists may not be looking to retailers for work. He notes that, ‘One is because most people do not think of retailers analyzing this data. Most people assume that retailers outsource the data analysis so they are not the ones doing the hiring. The other one is that universities are just now gearing up to develop programs addressing this issue.’

To entice data scientists, retailers need to be attuned to the culture, and let them know that they understand the importance of their work and are willing to get behind them, both in terms of money and personnel support. There is also the option of utilizing the experience of those within the company and training them up. Centers of Excellence are now commonplace for many departments, and data experts can share knowledge, and vice versa, so both groups can build the necessary skills.

According to the Forrester Analytics' State of Customer Analytics 2014 report, analytics is no longer an option, but a necessity for any organization to compete. That was two years, and it has become even more of an imperative since. Every organization in every industry needs a senior-level data specialist on their marketing team. A retailer using big data to the full has the potential to increase its operating margin by more than 60%, and this is an opportunity that should not be wasted.

University lecture small

Read next:

How Are Higher Education Institutions Using Analytics?