Join us for a three-part learning series: Diving Into Delta Lake. This series of tech talks takes you through the internals of Delta Lake, a popular open source technology enabling ACID transactions, time travel, schema enforcement and more on top of your data lakes. The Delta Lake engineering team, including Burak Yavuz, Andrea Neumann, Tathagata “TD” Das and Developer Advocate Denny Lee are your guides for this dive.
Many of the workshops include notebooks and links to slides for you to download.
Just getting started? Check out our Getting Started with Delta Lake tech talk series.
The transaction log is key to understanding Delta Lake because it is the common thread that runs through many of its most important features, including ACID transactions, scalable metadata handling, time travel, and more. In this session, we’ll explore what the Delta Lake transaction log is, how it works at the file level, and how it offers an elegant solution to the problem of multiple concurrent reads and writes.
As business problems and requirements evolve over time, so too does the structure of your data. With Delta Lake, as the data changes, incorporating new dimensions is easy. Users have access to simple semantics to control the schema of their tables, including enforcement and evolution.
In this session, we will dive deeper into how commits, snapshot isolation, and partition and files change when performing deletes, updates, merges, and structured streaming.