GE Aviation Spark Application – Experience Porting Analytics into PySpark ML Pipelines – Databricks

GE Aviation Spark Application – Experience Porting Analytics into PySpark ML Pipelines

Download Slides

GE is a world leader in the manufacture of commercial jet engines, offering products for many of the best-selling commercial airframes. With more than 33,000 engines in service, GE Aviation has a history of developing analytics for monitoring its commercial engines fleets. In recent years, GE Aviation Digital has developed advanced analytic solutions for engine monitoring, with the target of improving detection and reducing false alerts, when compared to conventional analytic approaches. The advanced analytics are implemented in a real-time monitoring system which notifies GE’s Fleet Support team on a per flight basis. These analytics are developed and validated using large, historical datasets.

Analytic tools such as SQL Server and MATLAB were used until recently, when GE’s data was moved to an Apache Spark environment. Consequently, our advanced analytics are now being migrated to Spark, where there should also be performance gains with bigger data sets. In this talk we will share experiences of converting our advanced algorithms to custom Spark ML pipelines, as well as outlining various case studies.

Session hashtag: #SAISExp12



« back
About Peter Knight

Dr Peter Knight is a senior data scientist with the GE Aviation UK data science team. He has over fifteen years' experience of developing analytics on aviation data sets. He has an Aerospace PhD and 1st class master's degree from the University of Southampton. He has 4 papers and 5 patents to his name. He was awarded the Derek George Astridge Safety in Aerospace Award in 2005 and has also won a number of GE awards including 'Global Customer Champion Award', 'Most Inspirational Employee Award' and open innovation challenge winner.

About Honor Powrie

I have worked in the Aviation industry for more than 25 years, including 20+ years' in Machinery Health Monitoring and Management. Since 2007 I have worked for General Electric where I am a Director of Data and Analytics. I manage GE Aviation's only UK-based Data Science group, which delivers innovative solutions for monitoring and managing GE's extensive and world-renowned commercial engines fleet. I am also a Visiting Professor in the Faculty of Engineering at the University of Southampton and in 2017, I was awarded a Royal Academy of Engineering Visiting Professorship in Data and Analytics, Asset Condition Monitoring and Management.