5 Ways To Solve The Data Skills Gap

The lack of talent in the field is a growing problem. We asked the experts how to get around it


The skills gap is often cited as a major problem organizations have with realizing the potential of their data. One study by McKinsey projected that by 2018, 'the US alone may face a 50% to 60% gap between supply and requisite demand of deep analytic talent.’ The Bureau of Labor Statistics (BLS), meanwhile, estimates that the number of roles for individuals with this skill set will grow by 30% over the next seven years.

Ensuring that there is a sufficient pipeline of candidates to fill these roles is going to be one of the most pressing issues of the next few years, particularly as organizations look to implement machine learning and AI. According to Capgemini, 'nine in 10 companies said they planned to improve their data science expertise', but the solutions to the problem are not necessarily obvious. We asked the experts what they believed could be done.

Greater Consistency In Expectations Of The Data Scientist

Big data is still an incredibly new practice and ‘data scientist’ is a role that has only recently established itself. As such, job descriptions are often extremely wooly and can vary wildly from company to company, written by people who are sure that they need someone to do something with their data, but unclear of what this is and who can do it.

With expectations so askew, many are saying they need a data scientist when in actuality they only really need an analyst. Equally, many believe that they need someone with industry experience to do their job, when really they just need to get the data to a place where analysts are in a position to unlock insights from the datasets and start to identify trends. Data scientists are effectively there to provide a bridge between the programming and implementation of data science, the theory of data science, and the business implications of data. If you restrict yourself to data scientists from one industry, you are always likely to see a shortfall.

Caroline Clark, Machine Learning Engineer At Argos

'In my experience, companies aren't always clear in the job description about what they're looking for from a candidate. For example, do you need someone with strong software development skills as well as a background in data science? It can certainly be more difficult to find someone with skills in a broad range of areas, but this may not be what you need, so I'd recommend understanding the requirements of the role and targeting candidates based on this. I spend a long time reading the job description when looking for a role and I think this can help to attract the right candidate.'

Joseph Nathan, Former Director of Information Management at News UK

'The issue with hiring data scientists is really deciding whether you want to hire a data scientist who is a specialist in, for example in the case of my industry, media. Do you want someone who has been exposed heavily to media and data in media, or do you just want a data scientist who's a generalist? We used to think that it was better to have that niche skillset, to have a data subject matter expert as well as a subject matter expert in that industry. But what we're actually finding is those data scientists who have worked in other industries - whether it's automobile, travel, or something else completely different to the industry you want them in - they actually bring a different external lens. So then it's really about whether or not that data scientist will fit within the team you have built, whether that's engineers, or business partners, or product owners.'

Let Automation Solve The Issue

Gartner estimates that over 40% of data science tasks will be automated by 2020, resulting in increased productivity and broader usage of data and analytics by citizen data scientists. Alexander Linden, research vice president at Gartner, noted that, ‘Making data science products easier for citizen data scientists to use will increase vendors' reach across the enterprise as well as help overcome the skills gap. The key to simplicity is the automation of tasks that are repetitive, manual intensive and don't require deep data science expertise.’

Even data scientists themselves are concerned. In a KDnuggets poll released last year, 51% of respondents said that they expect most expert-level Predictive Analytics/Data Science tasks currently done by human Data Scientists to be automated by 2025. Just a quarter said they expect this to happen in over 50 years or never. There is some debate around to what extent exactly automation will take data scientist's jobs, but equally, if even just the simplest tasks are automated then this should go some way to reducing the demand for data scientists, while also increasing supply by eradicating some of the more mundane parts of the job and making it more appealing.

Walter Storm, Chief Data Scientist at Lockheed Martin

'There are already great advances in automating many pieces of the analytics chain, and emerging machine learning technologies are showing great promise. We need to remember however, that 'learning' in this context is but a metaphor. Machines don’t 'learn' the way humans do - as much as we are inspired by biology, the machine is still assessing probabilistic relationships derived from transformations of data. As more of these algorithms become automated, and more purpose-built hardware is created, I believe the role of the (back-office) data scientist will shift from implementation to more integration and more front-office work.'

Dr. Zhongcai Zhang, Chief Analytics Officer at New York Community Bancorp, Inc.

'Data scientists, sometimes referred to as unicorns, play a crucial role in the provision of analytics within an organization. Machine learning clearly has an impact on how we do analytics and this impact is poised to increase (directionally) as such machine learning algorithms start to mature. However, such impact is very gradual and likely to be confined to those with a mere technical focus. There are, broadly speaking, two types of data scientists: one more technically oriented and one more business savvy. For data scientists with strong business acumen, they are going to see the demand for their skills on the rise. In the next few years, we are more likely to see a continual increase in demand for data analytical skills afforded to us by business savvy data scientists.'

Upskill Current Employees

One of the primary benefits of automating the data scientist's key tasks is that it will encourage more 'citizen data scientists'. Gartner defines a citizen data scientist as ‘a person who creates or generates models that leverage predictive or prescriptive analytics but whose primary job function is outside of the field of statistics and analytics.’ By democratizing data and encouraging everyone to be a data scientist, this should go some way to alleviating the impact of any skills gap. You can also retrain current employees, either in-house or externally. There are a wealth of training programs available that can provide them with data science skills to go alongside their existing business experience, and many schools now cater for this need, having retooled their programs to accommodate students in full-time employment. Indeed, it is possible that this is a preferable solution to hiring a fully-trained data scientist with no business experience. Software giant SAS, for example, runs three courses through its Academy for Data Science - six-week courses to become certified as a SAS Big Data Professional or an SAS Advanced Analytics Professional cost $9,000, while the eight-week SAS Certified Data Scientist will set you back $16,000. Larger companies also have the option of setting up analytics centers of excellence to ensure that more layman users receive some training in the use of data.

Jason Perkins, Head of Data & Analytics architecture at BT

'Data Science is a young field that continues to evolve across multiple dimensions including the data science methods and technical capabilities. Our strategy is to democratize data science to more of our data citizens so they can use its capabilities to unlock the value in our data. In order to do this then we need to simplify data science and provide more support for our analyst community.'

Peter Jackson, CDO of Southern Water

'At present, there probably is a skills shortage, but this will fill in time, perhaps through more automation or quicker development environments or better data management. Often the best way to fill the current gaps is through the upskilling of internal resources. They understand the data and the business. Alternatively, the gap can be filled with Analytics-as-a-Service.'

Get More Women Into Data

There is also an argument to be made that a skills gap exists because data science suffers from the same issue all STEM industries suffer from - a lack of women. Just 18% of computer science degrees go to women, although this approaches more than 40% when it comes to statistics degrees. We need to improve this pipeline as currently, an entire gender is, for whatever reason, underrepresented. If even 20% more moved into the field, it would have a dramatic impact on reducing the deficit between supply and demand in data. Carla Gentry, a successful data scientist and founder of Analytical Solution, for one, explained that, ‘More women are becoming interested in the big data field because it's an interesting subject, filled with lots of potential. I think 'we' see the whole picture of these possibilities because as wives, mothers, etc. we have to see the macro view all the time. Therefore seeing the big picture comes naturally, in my opinion.’ Gentry does, however, continue to say that, ‘But, we do have an uphill battle to gain a foothold in this field, as I am constantly reminded even after 17 years in data analytics. Until our own field (tech/data science/analytics) recognizes us for our talent, how do we think others will? There are too few truly talented, experienced people in Big Data to silence the share women have attained. It's time to start highlighting talent and not gender. We need all hands on deck if we plan to take Big Data analytics to the next level.’

Sivan Aldor-Noiman, Director of Data Science for the Data Science, Center of Excellence, The Climate Corporation

'I'd say that many women I have seen want to be able to see themselves advancing but looking at the top it's very hard to picture themselves there because they don't see a good role model. The solution is training managers and executives on how to make the environment more supportive, helping women find their voice and confidence in their abilities, and training them as well on how to be successful.'

Give It Time

Ten years ago, just a handful of colleges in the US offered Big Data/analytics degree programs. Today, almost 100 schools have data-related undergraduate and graduate degrees, as well as certificates for working professional or graduate students wanting to augment other degrees. These courses are reportedly full, which suggests a healthy pipeline of job candidates, and as publicity grows around the field and its high average salary, demand is likely to grow again. Of course, these students take time to filter through into the jobs market and in as new a discipline as data science, it could just be that if we wait, we will see a flood of candidates in the next few years.

Nick Read, Director of Data Sciences at Sony Interactive Entertainment

'Recruiting people into data science roles is tough. I’ve been at Sony for three years and we’ve grown our team from about three to a dozen. I also recruit people in Silicon Valley, where if anything it is even tougher. What I have noticed over the last year or two, however, is that we get a lot of good graduates coming out of Masters programmes. The people with ten years experience who have got the really hard to find technology skills, that’s where we’re really seeing a deficit. I think it is going to change over the next two or three years because a lot of graduates are studying data science today, which they weren’t three or four years ago.'


Read next:

Why Blockchain Hype Must End