The big data market: The skills organizations are hiring

Finding talent to turn data into actionable insights


Businesses across every industry are facing unprecedented disruptions; with tech giants such as Netflix, Amazon and Uber transforming how services are delivered and uncovering new patterns in consumer habits in the process. Business models are being rewritten, and efficiencies are being found through data-driven operations, as customers increasingly demand personalized experiences.

At the same time, the much talked about General Data Protection Regulation (GDPR) is shaking up how businesses collect, process, manage and secure their data. Designed to give EU citizens more control over their personal information and cut through complexities which organizations face when complying with opposing local laws, the regulation is having a significant impact on businesses’ internal processes. Against this backdrop, it’s hardly surprising that big data professionals are highly sought after by employers. In fact, demand for big data expertise has skyrocketed by 78% in the last 12 months, according to Tech Cities Job Watch report. Organizations are under increasing pressure to make better use of their data to both maintain a competitive edge and remain compliant with new regulations.

It’s the insight that can be gleaned from datasets that makes big data powerful. To achieve this, employers must engage with the right professionals – those who can turn raw data into business gold. Here, I consider the reasons why employer demand for big data talent is on the rise, as well as specific skill sets that are currently needed in enterprises.

Visit Innovation Enterprise's Big Data & Analytics Innovation Summit in Sydney, September 17-18, 2018.

Turning data into actionable insights

With the steady rise in adoption of sensor technologies, including the Internet of Things (IoT), we can expect datasets to grow and, similarly, the demand for big data specialists. More specifically, contractor demand has grown by 128% in the past year and permanent roles by 68% over the same period. This increase outstrips demand across other important skills markets, including Mobile (26%) and Cloud (30%).

Sought after skills

As business leaders continue to grapple with the complexity of running increasingly data-led organizations, the most in-demand skills for specific big data roles include Amazon Web Services, Python, Hadoop, Spark, Cloudera, MongoDB, Hive, Tableau and Java. Given the specialist nature of these skills, big data professionals command some of the highest salaries in the technology industry – earning an average of $87,950 – more than the other four technology disciplines tracked in the Tech Cities Job Watch report – Cloud $78,873, IT security $77,371, mobile $69,820 and web development $56,320.

Stagnating salaries

Whilst big data salaries remain the highest of the five core disciplines, remuneration has actually stalled in recent months. Permanent salaries increased by just 0.1% in the past year, and contractor day rates were down 5% over the same period.

This could be due to the fact that some big data projects have been put on ice in the run-up to the General Data Protection (GDPR) launch, and those with data skills have been reassigned to ensuring that the necessary data processing changes are successful. Thus, those capable of working on higher value – and, by default, higher paying – big data projects are making do with ‘GDPR gigs’. Though one would have expected the demand to have peaked sometime prior to the GDPR launch, organizations cannot risk being understaffed should their GDPR implementation not perform as intended.

It could also be because of a rebalancing in the market, and the fact that big data experts have seen the potential for high earnings and are upskilling themselves with the qualifications they need to secure those lucrative positions. In years gone by, big data skills were scarcer and employers had to put their hand in their pocket to secure the best. Now, candidate supply more closely meets demand, which could explain pay stagnation.

Keeping top talent energised and engaged

Hiring big data specialists will enable organizations to compete in the digital era, and businesses should look for candidates who demonstrate an aptitude and enthusiasm to learn new skills, rather than someone who simply has the right credentials on their CV. Team leaders should encourage staff to get involved with different projects and assignments around the business to widen their understanding and provide opportunities to develop their skillsets by completing the latest industry-certified courses.

In this data-driven world, organizations that can turn information into insight, better business decisions and personalized customer experiences are the ones that will truly thrive in the future of work. Big Data skills remain in high demand and businesses must keep a close eye on the market − to ensure they stay one step ahead of the competition and continue to position themselves as an employer of choice.

Why china is at the forefront of the ai race small

Read next:

Why China is at the forefront of the AI race