Gheorghe Pucea is currently working as a Big Data Engineer at BMW Group’s Autonomous Driving department. He is focusing on building complex big data pipelines for evaluating software components such as road detection. Before joining BMW, Gheorghe worked for several years at companies such as Audi Business Innovation and Teradata gaining experience in building streaming and batch data processing pipelines using Spark. He holds a Bachelor in Software Engineering from Politehnica University Timisoara and a Master’s Degree in Distributed Systems from Technical University of Munich.
Getting cars to drive autonomously is one of the most exciting problems these days. One of the key challenges is making them drive safely, which requires processing large amounts of data. In our talk we would like to focus on only one task of a self-driving car, namely road detection. Road detection is a software component which needs to be safe for being able to keep the car in the current lane. In order to track the progress of such a software component, a well-designed KPI (key performance indicators) evaluation pipeline is required. In this presentation we would like to show you how we incorporate Spark in our pipeline to deal with huge amounts of data and operate under strict scalability constraints for gathering relevant KPIs. Additionally, we would like to mention several lessons learned from using Spark in this environment.