I recently attended the Big Data Innovation Summit in San Francisco. Over the coming weeks, I will be sharing the insights I gathered from some of the most impactful presentations.
Walmart is the third largest commerce store in the US and currently the world's largest private employer. Somnath Banerjee is Director of Machine Learning at Walmart Labs, the technology division of the company. He and his team deal with all the data generated from the mobile app and website. They also create the technology that powers the stores, Walmart-pay, store pick-up, and Google Home.
The dawn of e-commerce
By the start of the millennium, e-commerce sales accounted for a paltry 2% of retail spending. Today, Amazon alone is a half a trillion dollar company, and as of 2017, e-commerce was 9% of total spending. This has led to the shuttering of thousands of brick and mortar stores across the US.
First, you have to understand why customers like E-commerce so much. There are four key reasons:
- Catalogue size
- Better product information
- Lower prices
The issue of catalogue size is evident at Walmart. There are three avenues to products for Walmart customers, and the website's catalogue size dwarves the rest combined:
- Neighborhood stores: 30,000 products
- Big 'super' stores: 150,000 products
- Website: +100 million products
The next challenge, therefore, becomes how to help people find products in a catalog this size.
- Search is by far the most popular and important way people look for items
- People who search on the site are 40% more likely to buy a product
- Half of Walmart's total revenue comes from search
- You can also browse through categories or recommendation modules on the website
However, the search function is a very complex technical problem. Common issues include:
- Ambiguity in search
- Differences in expression
- Catalogue quality
- If it's a new query or item
In the early days of e-commerce, the search function used to be hidden because it rarely functioned the way the customer wanted it to. Today, it is always in the most prominent location on any given e-commerce site.
With the growth of the internet, more big data was created and collected. New technology was created to both store and analyze said data, such as AI. As data science matured, we began to see more and more value in the data collected, so we collected more.
The two core building blocks of product search are capture intent and display results.
Capture intent functions like autocomplete are not just for user convenience, they provide further validation for the query. It helps preempt bad spelling, allowing the search to function better.
Display results is the core of big data and AI tech. This is the sequence:
Search query - text matching - relevance ranking - results
- Text matching is easy for humans, but typically difficult for machines
- It took decades of research to compute text similarity
- More powerful approaches have since been invented using deep learning
- Machines can recognize images at a higher proficiency than humans
- Relevance ranking - text similarly vs item popularity
- More advanced big data and AI progressions have led to better 'Query understanding'
- Once the machine can understand the query, relevance ranking improves significantly e.g NLP advancements
And the future?
- Visual search is now live on almost every major e-commerce site:
- 600 million searches powered by image understanding on Pinterest as of Feb 2018
- 30% higher conversion rate than text base searches
- 5x higher conversion rate for shoppers clicking on visually similar products
- 50% higher CTA of shoppers who click visually similar products
As omnichannel retail evolves, we will eventually graduate away from search. Alibaba even has started testing their latest prototype for where they see the evolutions of search. Their tech is now capable of detecting what customers are holding as they walk around shopping and can recommend variations of it on a screen, in real time.
"The way the market is evolving is so dynamic, how do you even get ahead of the curve in such a dynamic field?"
"You have to commit to things wildly ahead of the curve. Commit to new technology research even when you don't believe its 3/5 years close to coming to fruition."