Andre Sionek

Data Engineer, Gousto

Andre Sionek worked for different companies in Brazil before moving to London to work as a Data Engineer at Gousto. His past experiences include financial services, e-commerce and real estate. He regularly teaches data engineering courses and also writes about data for his blog.

Past sessions

Upserting millions of events per day into Delta Lake using Spark Streaming and Autoloader.

Gousto is the leading recipe box company in the UK. Every day we have to keep track of a huge amount of ingredients flowing through our warehouse until they are shipped to customers. In this talk, Gousto's Data Engineers will describe the challenges faced and the solutions found to merge the incoming stream of inventory events into Delta Tables. Come to hear about the bumps along the way, and to discover the tweaks implemented to improve merge performance. Today Gousto has real-time insight into the flow of ingredients through its supply chain, enabling a smarter, more optimised measure of its inventory performance.

Speakers: Eoin O'Flanagan and Andre Sionek