SESSION
Effective Lakehouse Streaming with Delta Lake and Friends
OVERVIEW
EXPERIENCE | In Person |
---|---|
TYPE | Breakout |
TRACK | Data Engineering and Streaming |
INDUSTRY | Enterprise Technology, Retail and CPG - Food |
TECHNOLOGIES | Apache Spark, Delta Lake, Orchestration |
SKILL LEVEL | Intermediate |
DURATION | 40 min |
In life, we learn to crawl before we walk and run. This is the natural progression as we train ourselves to operate at different speeds, maintain balance, and learn to listen to and monitor ourselves and our surroundings. The same can be said for any streaming data system, especially for the lakehouse. In this session, we will discover the true power of the streaming lakehouse architecture, how to achieve success at scale, and, more importantly, why Delta Lake is the key to unlocking a consistent data foundation and empowering a "stress-free" data ecosystem. Key lessons:
- Learn practical industry standards for simplifying Data Capture and Ingestion using protocol streams.
- Fall in love with Schema Enforcement and Evolution.
- Dive into the Medallion Architecture and learn why Data Quality matters.
- Discover how Internal and External Data Domains insulate Data Teams and support the Streaming Foundation.
- Unlock Reliable Streaming with Opinionated Workflow Orchestration.
SESSION SPEAKERS
Scott Haines
/Distinguished Software Engineer
Nike
Ashok Singamaneni
/Principal Engineer
Nike