Dora Petrella

How We Built a Machine Learning-Based Recommendation System (And Survived to Tell the Tale)

How do you find the perfect substitute for an out-of-stock item? Learn how we adapted a natural language model to solve this critical e-commerce challenge.

How We Built a Machine Learning-Based Recommendation System (And Survived to Tell the Tale)
#1about 5 minutes

Defining the business need for product recommendations

A recommendation system for substitute products is needed across multiple touchpoints to prevent lost sales from out-of-stock items.

#2about 2 minutes

Analyzing the limitations of the existing recommender

The previous system, based on the Jaccard coefficient, produced low-quality recommendations, particularly for new or unpopular items.

#3about 5 minutes

Using the Prod2Vec algorithm for recommendations

The Prod2Vec algorithm, adapted from Word2Vec, learns product relationships by analyzing co-occurrence within user session context windows.

#4about 2 minutes

Improving predictions with Meta-Prod2Vec and metadata

Incorporating product metadata like category and brand into the model (Meta-Prod2Vec) significantly improves recommendation quality for long-tail items.

#5about 2 minutes

Implementing the end-to-end MLOps pipeline

The production system uses dbt for data transformation, a Vertex AI pipeline for model training, and Elasticsearch for efficient vector similarity search.

#6about 3 minutes

Evaluating model performance with offline and online metrics

Offline metrics like NDCG confirmed model quality, while mirror traffic analysis showed a 45% increase in product recommendation coverage.

#7about 3 minutes

Visualizing product relationships with embedding projector

Using TensorFlow's Embedding Projector tool reveals how the model groups similar products into distinct clusters in a high-dimensional space.

#8about 3 minutes

Adopting pragmatic baselines and automated data analysis

Key project takeaways include using simple business-logic baselines for benchmarking and automating exploratory data analysis within the ML pipeline itself.

#9about 1 minute

Understanding the project team and final timeline

The project was completed in nine months by a cross-functional team of data engineers, data scientists, and software developers.

Related jobs
Jobs that call for the skills explored in this talk.

Remote

Name of

Name of

Remote
Intermediate
PHP
Java
+1

Matching moments

Real-world examples of machine learning in e-commerce
02:56 MIN

Real-world examples of machine learning in e-commerce

Data Science in Retail

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
Real-world applications and key takeaways
01:54 MIN

Real-world applications and key takeaways

Machine learning 101: Where to begin?

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
How AI powers e-commerce from logistics to discovery
05:15 MIN

How AI powers e-commerce from logistics to discovery

Intelligence Everywhere: The Future of Consumer Tech

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
Future ideas for personalized vacation planning
02:19 MIN

Future ideas for personalized vacation planning

Hacking Your Vacation: Using Data for Fun

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
Demo of a unified model and business monitoring dashboard
07:54 MIN

Demo of a unified model and business monitoring dashboard

Deployed ML models need your feedback too

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
Adopting a holistic AI strategy across business functions
05:57 MIN

Adopting a holistic AI strategy across business functions

Fireside Chat with Werner Vogels, VP & CTO, Amazon.com & Daniel Gebler, CTO at Picnic

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
Navigating the machine learning project lifecycle
10:46 MIN

Navigating the machine learning project lifecycle

Intelligent Automation using Machine Learning

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
The challenge of operationalizing production machine learning systems
02:47 MIN

The challenge of operationalizing production machine learning systems

Model Governance and Explainable AI as tools for legal compliance and risk management

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.

Featured Partners

Related Videos

See all videos
Data Science in Retail58:21

Data Science in Retail

Julian Joseph

Deployed ML models need your feedback too37:24

Deployed ML models need your feedback too

David Mosen

Empowering Retail Through Applied Machine Learning22:41

Empowering Retail Through Applied Machine Learning

Christoph Fassbach & Daniel Rohr

Building Products in the era of GenAI59:39

Building Products in the era of GenAI

Julian Joseph

Hybrid AI: Next Generation Natural Language Processing21:01

Hybrid AI: Next Generation Natural Language Processing

Jan Schweiger

What non-automotive Machine Learning projects can learn from automotive Machine Learning projects48:01

What non-automotive Machine Learning projects can learn from automotive Machine Learning projects

Jan Zawadzki

MLOps - What’s the deal behind it?29:24

MLOps - What’s the deal behind it?

Nico Axtmann

DevOps for Machine Learning44:40

DevOps for Machine Learning

Hauke Brammer

Related Articles

View all articles
DD
Dilek Demir
Data Science & more: The Lopez dilemma
Catwalk, Data Science, Hollywood, Google Images, Haute Couture, StackOverflow, Comfort Zone, Dota 2 and Versace – all these topics are connected and influenced by each other. Read here how and why!In 2000 Jennifer Lopez's green Versace dress went vi...
Data Science & more: The Lopez dilemma

From learning to earning

Jobs that call for the skills explored in this talk.