With the constant drive for improvements, efficiency savings and speed of service, it is not a surprise that the use of data within companies has become one of the key themes running through successful organizations. In fact, the two most valuable companies in the world - Apple & Alphabet - can lay much of their success at the feet of effective use of data.
When companies look at these organizations, or Facebook, Uber and AirBnB, they see data machines, companies that can rely on the data they have. It is therefore no surprise that leaders across the board want to create a data driven organization, where decisions can be made through efficient use of metrics. However, achieving this is not as simple as starting an analytics program, it requires the entire company to focus on data.
To do this, you need to follow three key principles:
Focus: Look at the right KPIs and know the difference between your data and your metrics. Use outputs that can consistently be measured and which align with your core mission.
Often companies fall into the trap of collecting as much data as they can, then hoping that at some point in the future they will be able to make sense of it. This seldom, if ever, works as it creates such a messy and unworkable system. The companies who have become truly data-driven are the ones who have identified early on which core metrics they need to look at, then built around them.
It is not to say that in the future a company will not expand into other areas, but trying to look at too much too soon means spreading yourself too thin and missing many of the fundamentals.
Speed: The rate of your insights must match the rate of your decision making.
The speed of decision made today is as important as making the right decision in the first place. A correct decision that's a few days too late will be the equivalent of making the wrong decision in a shorter amount of time. It is the concentration on the speed of decisions that has seen the rise in the use of in-memory systems like Apache Spark. Some Companies have realized that their legacy systems are simply not fast enough to give them the data they need at the time they need them.
Facebook have got around this through creating their own in-memory system; Scuba. Given that the site has over 1.5 billion active users, using traditional systems to analyze data across the site would take days, if not weeks, to analyze. Therefore, the Scuba system was created to effectively utilize their data in a timely fashion.
Lior Abraham, Senior Software Engineer at Facebook at the time of Scuba's creation put the need for system clearly -
'Traditionally, we’ve relied on pre-aggregated graphs or tools that query from samples stored in a MySQL database. While these approaches were often sufficient for basic queries and small datasets, as data grew in size, and as we needed to ask more sophisticated questions, they became way too rigid and slow.'
Empathy: Interact often with your customers/members/beneficiaries and understand what success looks like for them.
To become a data driven organization, the company needs to have an ultimate purpose. Creating a system where you can see metrics quickly and clearly is not useful unless you know firstly what they represent and then what you are aiming to achieve with them. For customer facing companies this often means a better user experience when people are using your product or site.
Knowing what your customers/users want is not just about looking at the data and making educated guesses, but about finding out directly from them what they want and then using the data you hold as an indicator of how you can achieve it. Amazon are a clear example of this, through using data they have managed to optimize the use of their sites through elements like recommendation engines, button placement and even the 'buy it now' button. This came from talking to customers and understanding what drives them, rather than what the company wants to drive them.
A company can never be truly data driven until it is genuinely being driven towards something, and the only way to know where it is being driven is through finding out where people want to go. Clearly, some of this will come down to innovative ideas, like the iPhone, where people don't know what they want until they have it, but these examples are as famous as they are rare.