Deployed ML models need your feedback too

Is your deployed ML model slowly failing in production? Learn why feedback loops are the missing piece for monitoring true performance and preventing model decay.

#1about 2 minutes

The role of feedback in the MLOps lifecycle

MLOps extends traditional software engineering by integrating processes from data science and business to ensure deployed models perform correctly.

#2about 2 minutes

Understanding the three pillars of MLOps

MLOps adapts DevOps principles by adding continuous training to continuous integration and delivery, addressing the unique needs of ML models.

#3about 4 minutes

Architecting a mature and automated MLOps pipeline

A mature MLOps architecture automates the entire lifecycle from feature store to prediction service, but requires monitoring to close the loop.

#4about 8 minutes

Exploring the different layers of model monitoring

Effective monitoring covers multiple layers, including infrastructure, data drift, concept drift, model performance, and business KPIs.

#5about 4 minutes

Key characteristics of an effective feedback system

Designing a feedback system requires considering the delay, collection method, and correlation of feedback to predictions.

#6about 4 minutes

Evaluating the state of current monitoring solutions

While many tools exist for monitoring training, live performance monitoring is less mature, with platforms like Google Vertex AI and Seldon Core having limitations.

#7about 8 minutes

Demo of a unified model and business monitoring dashboard

An internal tool demonstrates how to integrate with cloud AI services like AWS Personalize to provide a unified view of model and business metrics.

#8about 5 minutes

Q&A on multi-tenant models and edge deployment

The discussion covers best practices for deploying models to different customers, handling unstructured data, and adapting monitoring concepts for edge devices.