Unified ML Monitoring on Databricks

On-demand

Whether you are monitoring one model – or hundreds of models – at a time, the ability to understand the performance of your model and the supporting infrastructure is key to the long-term success of your Data Science efforts. Across the clouds, there are various solutions when it comes to managing model artifacts, logs, versions, and other information; however, unifying these key data points can be challenging.

In this virtual session, we will demonstrate how you can build a Unified Machine Learning Model Monitoring Solution on Databricks. Following this, a Databricks ML Engineer will walk you through the collection of key model metrics from sources like MLflow and external services such as AzureML or SageMaker. They will also explain how you can leverage Delta Lake as a means of managing this collected model performance information and calculating model drift indicators. Lastly, they will show how it all comes together in SQL Analytics as a Unified ML Monitoring Dashboard, complete with an alerting mechanism that triggers a retraining job when one of the model drift calculations goes below a certain threshold.

At this event, we will create a Unified ML Monitoring Solution, including:

• Collecting model performance data
• Estimating model drift based on collected model data
• Unifying collected model monitoring data into a single dashboard