Our uses for data have evolved rapidly over the years as they have seeped into every single facet of modern existence. From Netflix algorithms capable of trapping us in black holes of entertainment to machine learning software capable of diagnosing cancers, big data has revolutionized life as we know it.
However, these advancements wouldn't have been possible if we hadn't also revolutionized the way we collect and analyze data. As these changes have become more pronounced, they have allowed the applications for all kinds of data to widened greatly, to the general betterment of society as a whole.
One of such people who is fighting the good fight when it comes to figuring out new and more effective ways around this traditionally tedious task is Keith Tsui. Keith is the Data & Analytics Director at Lan Kwai Fong Group where he is responsible for various data-driven initiatives. He also heads up the newly formed ‘LKF Labs’, a multi-award-winning team that focuses on innovation and technology projects, including analytics, VR/AR, and gaming.
He also happens to be one of our speakers at these years Big Data & Analytics Innovation Summit, so we asked him how data gathering and analyzing has changed over the years.
You first joined Lan Kwai Fong Group as a consultant in 2012. How has data gathering evolved since then?
Regarding data gathering, in the beginning, we were exploring all the various innovative ways we could collect it. After doing many pilot projects though, we eventually discovered that some data is more useful than others. While one big data approach is 'collect everything you can, and data mine later', it is not always feasible with limited resources and ROI constraints.
Sometimes when there aren't many actionable insights from certain data sources, then one may have to decide to shift resources elsewhere with higher ROI.
And what about with regards to data analyzing?
As for data analysis, I have found that in the last two or three years, the software tools have really begun to mature, in both proprietary and open source areas. Cloud-based big data solutions from providers such as Google Cloud and Alicloud have also vastly lowered the barriers to entry and made the big data infrastructure reachable to many who previously didn’t have the scale and resources.
So have recent advancements and falling costs in software tools made it more feasible for organizations with fewer resources to 'collect everything and data mine later', or is it fundamentally flawed for them to use it?
As tech historian George Dyson once said, 'Big data is what happened when the cost of keeping information became less than the cost of throwing it away.' On one hand, the price of software tools, especially storage, has indeed fallen to very affordable levels lately. However, on the other hand, collecting data, particularly real-world offline information, the cost is often not negligible. One example would be installing CCTV to collect various customer in-store behavior patterns, there is still the actual cost of sensor hardware that one needs to take into account. Even though sensor systems have also gone down in price, there are still non-trivial costs that one needs to take into account when considering ROI.
It could be argued that with better data gathering/analyzing procedures, the more advanced machine learning becomes. What, in your opinion, is the most exciting application of machine learning/AI in your industry right now?
For most day-to-day customer-facing impact, chatbots have of course become quite useful with dealing with tedious, yet costly 'Frequently Asked Questions' interactions. This significantly freed up staff and resources to deal with more critical matters. Over the last two years, I’ve also witnessed machine learning make massive leaps in its effectiveness around visual analytics as well. Both the sensor and computational hardware used to be very expensive, but lately, the prices have become progressively cheaper and now the technology is within reach of even SMEs.
And what will you be discussing in your presentation?
In my talk, I discuss how big data can be used to help understand and thus better serve visitors to one of Hong Kong’s most popular spots for nightlife, Lan Kwai Fong