Expert View: Can Data Science Be Automated?

We talk to 7 of the top data minds to find out


The top spot in Glassdoor’s 2017 rankings of the best jobs in the US has gone to the data scientist, its second year at number one. With businesses placing an ever-growing importance on data, this is unlikely to be a surprise to many. The rankings are determined by three key factors – number of job openings, earning potential, and career opportunities rating - and with a median base salary of $110,000 and 4,184 job openings, those who have chosen it as a career path look like they’re onto a winner. However, are there clouds on the horizon?

Writing in Forbes last year, Bernard Marr argued that advances in Natural Language Processing (NLP) and data visualization meant data would be increasingly processed automatically and visualized without human input, ready for business experts to analyze. He noted that ‘the human interface — the data scientist — may soon be as mythical as that unicorn, and simply unneeded in the big data landscape where lay persons can conduct their own analytics at will.’

Marr’s assertions are backed up by several surveys. According to a report from Gartner, over 40% of data science tasks will be automated by 2020. Alexander Linden, research vice president at Gartner, noted that, ‘Making data science products easier for citizen data scientists to use will increase vendors' reach across the enterprise as well as help overcome the skills gap. The key to simplicity is the automation of tasks that are repetitive, manual intensive and don't require deep data science expertise.’ Even data scientists themselves are concerned. In a KDnuggets poll released last year, 51% of respondents said that they expect most expert-level Predictive Analytics/Data Science tasks currently done by human Data Scientists to be automated by 2025. Just a quarter said they expect this to happen in over 50 years or never.

We spoke to 7 data science professionals about whether automation would begin to push human beings out of the field.

Cameran Hetrick, Senior Director of Data Science & Analytics at thredUP

Absolutely not. Analytics and algorithms will always need to be managed by a person. Sure there is a lot of automation that can be done to remove the human touch, but we will always need an analytics person to translate a business question into data questions and data results into business strategy. For algorithms, they too need need goals. A human must be involved to construct an algorithm and manage it to ensure that it meets all the goals of the organization.

Jay Barua, Vice President at GoNoodle

I do not believe so and some amount of human intervention and involvement will always be needed to understand and separate the ‘gray’ between the black and the white.

Joel Shapiro, Executive Director of the Program on Data Analytics at Kellogg's School of Management at Northwestern University

Well, yes and no. In the right cases, data can be automatically generated and analyzed. But analytics is fundamentally about using analysis to do something differently. I am very skeptical of off-the-shelf analytics products that claim all you have to do is load in your data and it will spit out actionable insight. That’s really dangerous. Most businesses have unique processes, goals, and contexts that make the link from data to action fraught with nuance. Analytics still rests fundamentally on good critical thinking skills – how to ask great questions and rigorously assess evidence that can lead to action. It’s hard for me to imagine how context-specific critical thinking could be automated - I think good data scientists will serve this role for a long time.

Johann Posch, Senior Staff Data Scientist at GE Digital

Maybe somedays. Self adjusting machine learning models and system which automatically re-train already exists.

Dr. Zhongcai Zhang, Chief Analytics Officer at New York Community Bancorp, Inc.

Data scientists, sometimes referred to as unicorns, play a crucial role in the provision of analytics within an organization. Machine learning clearly has an impact on how we do analytics and this impact is poised to increase (directionally) as such machine learning algorithms start to mature. However, such impact is very gradual and likely to be confined to those with a mere technical focus. There are, broadly speaking, two types of data scientists: one more technically oriented and one more business savvy. For data scientists with strong business acumen, they are going to see the demand for their skills on the rise. In the next few years, we are more likely to see a continual increase in demand for data analytical skills afforded to us by business savvy data scientists.

Khalifeh Al Jadda, Ph.D., Lead Data Scientist at CareerBuilder

With the progress we have seen in the predictive analytics, machine learning, and Big Data I think we can fully automate the analytics. I can’t say we are there yet, but we are on our way.

Walter Storm, Chief Data Scientist at Lockheed Martin

There are already great advances in automating many pieces of the analytics chain, and emerging machine learning technologies are showing great promise. We need to remember however, that 'learning' in this context is but a metaphor. Machines don’t 'learn' the way humans do - as much as we are inspired by biology, the machine is still assessing probabilistic relationships derived from transformations of data. As more of these algorithms become automated, and more purpose-built hardware is created, I believe the role of the (back-office) data scientist will shift from implementation to more integration and more front-office work.

You can hear more from thredUP, GoNoodle, GE, and a host of other industry leading analytics professionals in San Diego this February 22-23 at the Predictive Analytics Innovation Summit
Big data hype small

Read next:

Is Big Data Still Overhyped?