Mastering Unity Catalog: Lessons from Zalando's Adoption Journey

Unlocking the full potential of Databricks Unity Catalog can be a game-changer for organizations, enabling features like lineage, fine-grained access control, SQL warehouses, and Databricks Serverless. However, the journey to adoption can become a challenge for larger businesses due to inherent complexities. This session delves into Zalando’s experience, where we confronted and conquered the complexity associated with adopting Unity Catalog as a central element in our self-service data platform. With 800+ users and diverse use cases, managing distributed schemas across 200+ Hive Metastores, and navigating strict legal regulations such as GDPR and antitrust laws, we found innovative solutions to make Unity Catalog an integral part of our Data Mesh architecture. The presentation will showcase the implementation of federated automated governance, compliant internal data sharing, and seamless integration into a data catalog, which elevates engineers’ productivity on the data platform.