Harmonizing Data Horizons: A Unified Databricks Data Lake Journey

ProSiebenSat1, a digital group with premier entertainment brands in Germany, Austria, and Switzerland, managed diverse data lakes using various approaches on-premises and in the Cloud. Seeking a holistic solution for a single source of truth that is scalable and compliant, Databricks was pivotal as it serves as an integrated platform for BI and AI use cases. Formerly on Hadoop, Cloud-native services, and diverse ETL tools, efforts beyond migration included harmonizing technology and structure for a unified data lake. Databricks flexibility and scalability were crucial for meeting project timelines. The focus expanded to governance, which was achieved by leveraging Databricks features like Unity Catalog.

This lightning talk delves into our journey, emphasizing Databricks features - Unity Catalog, Unified Governance, and UniForm. It explores complexities in migrating and unifying data from multiple on-premises and cloud-native systems, showcasing strategic technology use for a unified data lake.