As a data scientist at the Digital Innovation Partners, a service department of the Carl Zeiss AG, Jan is working on digital products that make ZEISS customers happy. Recently he focused on the development and deployment of machine learning models for predictive maintenance of ZEISS devices, especially the high-precision measuring machines used for quality assurance in industrial manufacturing. Before joining ZEISS, he was at the Volkswagen Data Lab, where he developed pilot applications for churn prediction, connected car and mobile apps.
We use Apache Spark Structured Streaming on the Databricks Unified Analytics Platform to process live data and Spark MLlib to train models for predicting machine failure. Structured Streaming and MLlib combined in the Zeiss Measuring Capability App allows users to stay on top of all relevant machine information and to know at a glance if a machine is capable of performing reliably. We will demonstrate how Azure Databricks allows us to easily schedule and monitor an increasing number of Spark jobs, continuously adding new features to our app. Session hashtag: #SAISEnt1