There is plenty of supposed evidence that a substantial gap exists between the number of data scientists needed and those available. A study by McKinsey, for one, projects that ‘by 2018, the US alone may face a 50% to 60% gap between supply and requisite demand of deep analytic talent.’ Indeed, the 2015 MIT Sloan Management Review found that 40% of companies are already struggling to find candidates to fill their data analytics roles, and if the number of speakers at tech conferences complaining about a shortage is anything to go by, the situation is not improving.
This alleged shortage of candidates is baffling. Just a few years ago, Harvard Business Review called Data Scientist the sexiest job of the 21st century, while the role also topped Glassdoor’s list of ’25 Best Jobs in America 2016’. The job is rare in that it is both intellectually fulfilling and also highly lucrative, with salaries averaging around the $120k mark. Even a summer internship will often pay somewhere in the region of $6,000 to $10,000 a month. It seems to tick all the boxes, so why is there a lack of suitable applicants?
Many reasons have been given for the alleged shortfall. There is a common misconception that maths and science subjects at school are only for the ‘ultra bright’ - those getting straight As throughout their school life. Some argue that this can be off-putting to those considering a data-related degree. Evidence for this is thin on the ground. While ten years ago, just a handful of colleges in the US offered Big Data/analytics degree programs, now almost 100 schools have data-related undergraduate and graduate degrees, as well as certificates for working professional or graduate students wanting to augment other degrees. Universities do not just set up courses for no reason, they do it because the demand is there, and these courses are largely full, suggesting a healthy pipeline of job candidates. Of course, these students take
There is also an argument to be made that a skills gap exists because data science suffers from the same issue all STEM industries suffer
Gentry may well be right, and it could be that managers are still reluctant to hire female data scientists because of some prejudice. If this really is the problem, it’s one that is easily solved simply by hiring more women. This is a failure of management, and it could be that similar failures are leading to the perception that a gap exists that’s not really there. Managers are often not ‘data people’, and many simply do not know what they’re looking for in a Data Scientist. The term itself encompasses a variety of things, but ask a dozen hiring managers what these are and you’re unlikely to get the same answer twice. A statement made by Tom Pohlmann, Head of Values and Strategy at Mu Sigma, is especially telling. He argued that ’it can be difficult to find candidates with the creativity and experimental
The talent is clearly there. Maybe it is the case that companies are simply not looking hard enough, but it could also be that darker forces are at play. If we assume that companies are not so overcome with prejudice that they won’t look at women for data science
Many studies suggest that the skills shortage in
Michael Teitelbaum, a demographer at Harvard Law School and author of the 2014 book ‘Falling Behind? Boom, Bust, and the Global Race for Scientific Talent’, argues that a skills shortage in STEM is a falsehood being propagated by companies wanting to flood the market with cheap
Whether or not there is a real intent to exaggerate the scale of the skills gap, the simple truth is that the talent is there. If companies really are struggling to employ data scientists, it’s entirely their own fault.