The presentation introduces Intuit AI Model Monitoring Service (MMS). MMS is an in-house Spark-based solution developed by Intuit AI to provide ongoing monitoring for both data (statistics of model input/output etc.) and model metrics (precision, recall, AUC etc.) of in-production ML models. The project is soon to be open-source. MMS aims to tackle multiple challenges of in-production ML model monitoring:
Qingbo Hu is a machine learning engineer from Intuit AI Engineering team and the acting tech lead for Intuit's Model Monitoring Service project. Before joining Intuit, he worked as a data scientist in LinkedIn. Qingbo earned his Ph.D from University of Illinois at Chicago, majoring in computer science.
Sumanth is a Software Engineer at Intuit, where he focuses on building scalable ML services. He has previously built Big data and ML systems at Verizon and Apple. Sumanth holds a Masters in Computer Science from the University of North Carolina at Charlotte.