Solving The Machine Learning Skills Gap

Organizations are trying to adopt machine learning in their droves, but without the talent they will likely fail


The number of jobs that machine learning could render redundant over the coming decades is a growing cause for concern. According to research by PwC, 38% of US jobs will be automated by 2030, while other parts of the world fare little better. In Germany, it is 35%, and in the UK, 30%. However, it may be inevitable that jobs will be lost, but as with all periods of great technological advance, new jobs will also be created. Many of these will in fact be focused on developing and supervising machine learning algorithms, helping businesses to integrate and implement the technology and bring in efficiencies hitherto unimaginable.

This is already happening to an extent. According to the job search website, June 2015 to June 2017 saw a 500% rise in the number of job postings in the field of AI. Of these job postings, 61% in the AI industry were for machine learning engineers, 10% were for data scientists and just 3% were for software developers.

However, machine learning is suffering from the same problem STEM has suffered from since time immemorial: A lack of qualified people to exploit it to its full potential. There is a dearth of people who understand first where is appropriate to apply it, and second how to apply it. According to a survey from Tech Pro Research, just 28% of companies have some experience with AI or machine learning, and more than 40% said their enterprise IT personnel don’t have the skills required to implement and support AI and machine learning.

Machine learning is, as you would likely imagine, extremely complicated, and not something your run-of-the mill computer engineer is going to be capable of without proper training. It requires someone with a background in computer science, likely with a doctorate in the sciences, as well as a significant amount of practical experience working with data at scale. Given that there is already a dearth of qualified data scientists, there is little to suggest that the situation is going to be any different when it comes to machine learning. And this is already hampering the technology. Just 15% of organizations manage to bring their big data projects to production, according to Gartner analyst Nick Heudecker, and he believes this number is likely to be far lower when it comes to machine learning. His fellow Gartner analyst Merv Adrian blames this on the lack of available talent, noting recently that, ‘For me it's mostly about skills. Missing skills.’ Sumit Gupta, IBM VP for Cognitive Systems, agrees, arguing that, ‘We really need to nurture and harness partners and startups with the skill sets to implement these AI offerings. Even The US Government has expressed concerns about the lack of AI talent. Last week, at a Senate Commerce, Science and Transportation subcommittee hearing, both Washington policymakers and AI experts agreed that the lack of tech talent could see the US overtaken by the rest of the world when it comes to AI technology.

Machine learning jobs come in many forms. If organizations are going to realize its potential, they will need to have the necessary data structure in place to deal with the unearthly amount of it they will need learn from in order to improve and work properly. Most companies undertaking machine learning projects already own and store vast quantities of data, but few enterprises have such copious quantities of data, and even then it is often siloed and requires aggregating, which is a lengthy and difficult process that few are resourced for. It also requires experts to cleanse the data to get it up to the quality it needs to be, and AI supervisors to ensure that the right data is being fed in, and to feed in new rules, business logic, and feedback that will make it able to do the job. And that’s just for developing the technology, actually integrating it into society requires even more people. Governments will need to introduce regulations around machine learning, which requires expert knowledge and real-world experience. City planners will need to understand driverless cars and how they work in order to adapt the infrastructure. The list goes on and on.

The issue is especially pressing in cybersecurity. According to an ITRC Data Breach Report, US companies and government agencies suffered a record 1,093 data breaches last year, up 40% on 2015. IBM/Ponemon found that the average global cost of each lost or stolen record containing confidential and sensitive data was $154. In healthcare it was $363. There are simply not enough security experts to defend organizations’ vital infrastructure and systems. Marc van Zadelhoff, General Manager, IBM Security noted that, ‘Even if the industry was able to fill the estimated 1.5 million open cyber security jobs by 2020, we'd still have a skills crisis in security.’ Machine learning algorithms could comfortably fill this gap, but they need experts to control them, and there are already too few with cybersecurity knowledge, let alone those with machine learning expertise on top. Indeed, according to Jon Oltsik, senior principal analyst at ESG and founder of the firm's cybersecurity service, just 30% consider themselves ‘very knowledgeable’ about machine learning.

The solution, as always, is education. Cognitive technology behemoth IBM has already entered into partnerships with academic institutions, and more need to take the lead. The US was doing well, but it seems to have gone backwards. Under President Obama, it was on track to train 100,000 new STEM teachers by 2021, and American universities began graduating 100,000 engineers every year for the first time in the nation’s history. High schools in 31 states even introduced computer science classes as required courses. But it is unlikely that this trend will continue, given that this administration’s choice for Education Secretary is Betsy DeVos, who has historically taken an anti-science pro-religion view of education, saying in 2001 ‘There are not enough philanthropic dollars in America to fund what is currently the need in education ... Our desire is to confront the culture in ways that will continue to advance God's kingdom.’

Universities are not the only place prospective machine learning experts can go to, though. Organizations such as Udacity, Udemy, and MIT’s OpenCourseWare offer intensive courses that have become extremely popular in recent years, and they often enable you to better tailor your education to suit the needs of an individual employer. While there has been criticism that such courses are too expensive and therefore not open to enough people, they have the expertise to go some way to countering government malaise in the area.

Ultimately, if machine learning is going to reach its full potential, we are going to need to have the talent in place to get there. At the moment, this is nowhere near happening, and this is because governments are still not doing enough to ensure that there is a pipeline of STEM graduates capable of performing the role. In the future, this will have to change, because those who aren’t capable will likely find themselves irrelevant in the world of work.


Read next:

Why Blockchain Hype Must End