Seamless Flow: Evolving From Batch to Streaming Data Flows Using DLT

84.51 is a retail insights, media and marketing company. We use first-party retail data from 60 million households sourced through a loyalty card program to drive Kroger's customer-centric journey. We chose Delta Live Tables (DLT) to build our clickstream and customer order data flows because DLT provides a clean notebook interface and rich functionality. The declarative nature of the code is easy to write and maintain, and it delivers a robust operational experience. In this session, we will discuss how DLT’s Append Flows API helped us evolve from batch files to streaming over time with ease and without interrupting the data flow in our production lakehouse. We’ll also show how we combined a streaming data source with static historical data and how the Once Flow option allowed us to drop duplicates between the flows efficiently. Finally, we’ll discuss using DLT’s Change Data Capture and the sequence_by option to capture events in a specific order.