Eoin O’Flanagan

Lead Data Engineer, Gousto

Eoin O’Flanagan has more than 12 years experience in data and has held a variety of data roles across multiple industries including financial services, social media analytics and retail. He is currently Lead Data Engineer at Gousto, where he is enjoying finally bringing some of his passion for food to his place of work.

Past sessions

Upserting millions of events per day into Delta Lake using Spark Streaming and Autoloader.

Gousto is the leading recipe box company in the UK. Every day we have to keep track of a huge amount of ingredients flowing through our warehouse until they are shipped to customers. In this talk, Gousto's Data Engineers will describe the challenges faced and the solutions found to merge the incoming stream of inventory events into Delta Tables. Come to hear about the bumps along the way, and to discover the tweaks implemented to improve merge performance. Today Gousto has real-time insight into the flow of ingredients through its supply chain, enabling a smarter, more optimised measure of its inventory performance.

Speakers: Eoin O'Flanagan and Andre Sionek