Delivering Portability to Open Data Lakes with Delta Lake UniForm
OVERVIEW
EXPERIENCE | In Person |
---|---|
TYPE | Lightning Talk |
TRACK | Data Lakehouse Architecture |
INDUSTRY | Enterprise Technology, Manufacturing |
TECHNOLOGIES | Data Sharing, Apache Spark, Delta Lake |
SKILL LEVEL | Intermediate |
DURATION | 20 min |
DOWNLOAD SESSION SLIDES |
As data volumes and users rapidly scale, data lakes encounter major challenges around reliability, performance, and governance. Delta Lake UniForm (Universal Format) helps address these pain points on multiple open data lake environments such as Delta Lake, Apache Iceberg, and Apache Hudi. This talk will demonstrate how Delta Lake UniForm enables seamless and unifying access to multiple open data lakes while optimizing workloads. We also deeply dive into key technology behind the UniForm that improves portability, reliability, and query performance. Through live demos, we showcase scaling a cloud-based data lake from terabytes to petabytes while maintaining ACID transactions, audit history, and so on. We deliver Delta Lake UniForm best practices to future-proof their own expanding data lakes. The UniForm capabilities make data lakes more accessible to diverse users.
SESSION SPEAKERS
Tomohiro Tanaka
/Senior Cloud Support Engineer
Amazon Web Services