Can Data Scientists Move Between Industries?

Do they really need business experience?

Get this and many more burning questions answered at this years Chief Data Officer Summit in San Francisco and Singapore

Data scientists often move between industries, but can they really do their job effectively without any business experience?

In a recent report by digital transformation firm Atos, 40% of businesses said they are using data analytics in their key functions, while 23% plan to implement it within the next year. By 2020, 90% of companies say they will do the same. However, many of these will likely fail. Gartner predicted last year that as many as 60% of data projects would not make it to fruition, languishing in the piloting and experimentation phases before being discarded. A recent KPMG survey, meanwhile, found that just 40% of executives have a high level of trust in the consumer insights from their analytics, and most said their C-suite still don’t fully support their current data analytics strategy.

One of the most commonly cited factors in the success of a data initiative is the talent you employ to carry it out. Such talent is not easy to come by. The McKinsey Global Institute predicts that the shortage of data scientists in the US could increase to 250,000 by 2024, while other estimates are little better.

There are a number of solutions to the shortfall, one of the most common of which is to retrain current employees, either in-house or externally. Training can provide them with data science skills to go alongside their existing business experience, and many schools now cater for this need, having retooled their programs to accommodate students in full-time employment. Indeed, it is possible that this is a preferable solution to hiring a fully-trained data scientist with no business experience. In a recent MIT Sloan Management Review article, it was argued that even those businesses with established data specialists "struggle to realize the full organizational and financial benefits from investing in data analytics." This suggests one of two things. Firstly, it could be that talent is simply not a deciding factor in a data initiative’s success. Even if you believe it has been overestimated, this would still fly in the face of common sense and experienced wisdom. Alternatively, it could be that the skills that companies are looking for in a data scientist are outright wrong. They’ve hired data talent, unfortunately, it’s just not the talent they need.

Big data is still an incredibly new practice and the ‘data scientist’ is a role that has only recently established itself. As such, it is still often vaguely defined, and some organizations are unsure of exactly what it is data scientists actually do. The job spec subsequently often varies wildly from company to company, who have little idea of what they’re wanted for, only that they need someone to something with their data. Their expectations are askew, with some required to perform tasks better left to engineers and analysts.

Equally, however, this is not just the fault of ignorant companies. A data scientist’s role is to take raw data and marry it with analysis to make it accessible and more valuable for an organization. To do this, The Guardian explains, "they need a unique blend of skills – a solid grounding in maths and algorithms and a good understanding of human behaviours, as well as knowledge of the industry they’re working in, to put their findings into context. From here, they can unlock insights from the datasets and start to identify trends." Data scientists are effectively there to provide a bridge between the programming and implementation of data science, the theory of data science, and the business implications of data. Yet many data scientists seem to be ignoring the last part, believing they can move from industry to industry, and it is likely they are not performing their role to the optimal level as a result. As Tye Rattenbury, director of data science at Trifacta, notes, "Too many data scientists are stuck in maintenance mode - organizing and collating data, rather than actually spending time analyzing it."

Ultimately, there is a gap in expectations. Data scientists believe they need limited knowledge because they won’t have to do much analysis, while businesses expect them to provide analysis impossible to accrue without industry knowledge and experience. This gap needs to be closed if companies are going to fully realize the benefits of their data. In a recent interview with us, Walter Storm, Chief Data Scientist at Lockheed Martin, noted that "If they (data scientists) have a passion for coding, deep learning, and technology itself, then the industry they’re in doesn’t matter. I often refer to this as 'back office' data science. However, the greater challenge is the 'front office' data science work. It is this data scientist that is the translator - speaking both the language of the business and the language of data science. The front-office data scientist must know the industry, have a firm grasp of economics and finance, and be able to validate, integrate and use advanced models within a broader decision support framework." Data scientists must appreciate that the majority of companies out there will be looking for someone to provide analysis and firstly demonstrate they have some in interviews, secondly make a concerted effort to accrue knowledge while in employment. Companies, meanwhile, need to figure out whether they need someone for the front or back office before they hire anyone. It may well be enough for them to train a member of staff with industry experience in data, or it may be preferable to train an experienced - and usually more expensive - data scientist in how the industry works. Employers looking for a data scientist need to explain what they are looking for in detail early in the interview process. A data scientist doesn’t really need to be concerned if they don’t have the skills the company is looking for, because they are extremely unlikely to be short another job opportunity given the talent gap noted earlier, and will likely tell you that you need something else. It may even be worth hiring a consultant at the early stages of a data project to identify objectively what you may need. Gary Damiano, vice president of marketing at NoSQL database specialists Couchbase notes that, "The two things you need to consider in hiring a data scientist are: how are you going to use them and how does their skillset match the use?" Consider carefully what you’re looking for, because it may be that you don’t even really need one at all.

Get this and many more burning questions answered at this years Chief Data Officer Summits in San Francisco and Singapore

Read next:

Why We Need Data Visualization To Understand Unstructured Data