Databricks Unity Catalog is the industry’s only unified and open governance solution for data and AI, built into the Databricks Data Intelligence Platform. With Unity Catalog, organizations can seamlessly govern both structured and unstructured data in any format, as well as machine learning models, notebooks, dashboards and files across any cloud or platform. Data scientists, analysts and engineers can securely discover, access and collaborate on trusted data and AI assets across platforms, leveraging AI to boost productivity and unlock the full potential of the lakehouse environment. This unified and open approach to governance promotes interoperability and accelerates data and AI initiatives while simplifying regulatory compliance.
“Databricks Unity Catalog is now an integral part of the PepsiCo Data Foundation, our centralized global system that consolidates over 6 petabytes of data worldwide. It streamlines the onboarding process for more than 1,500 active users and enables unified data discovery for our 30+ digital product teams across the globe, supporting both business intelligence and artificial intelligence applications.”
— Bhaskar Palit, Senior Director, Data and Analytics
Unified visibility into data and AI
Easily discover and classify both structured and unstructured data in any format, including machine learning models, notebooks, dashboards and files across all cloud platforms. Seamlessly manage, govern and query data from external databases and data warehouses like MySQL, PostgreSQL, Amazon Redshift, Snowflake, Azure SQL, Azure Synapse, Google BigQuery and catalogs such as HMS and AWS Glue in one place. Accelerate your data and AI initiatives with a single point of access for data exploration. Improve productivity with intelligent search, discovery and automatically generated data insights and documentation.
Single permission model for data and AI
Simplify access management with a unified interface to define access policies on data and AI assets and consistently apply and audit these policies on any cloud or data platform. Securely access data from other computing platforms using open interfaces, with consistent permissions managed in one place. Enhance security with fine-grained control on rows and columns, while efficiently managing access through low-code attribute-based access policies that scale seamlessly.
AI-powered monitoring and observability
Harness the power of AI to automate monitoring, diagnose errors and uphold data and ML model quality. Benefit from proactive alerts that automatically detect personally identifiable information (PII) data, track model drift, and effectively resolve issues within your data and AI pipelines to maintain accuracy and integrity. Streamline debugging, root cause analysis, and impact assessment with automated column-level data lineage. Gain comprehensive observability into your data and AI with operational intelligence utilizing built-in system tables for billing, auditing, lineage and more.
Open accessibility
Securely access your data and AI assets from any compute engine using open APIs and standard interfaces. Share data and AI assets across clouds, regions and platforms with open source Delta Sharing. Securely collaborate with anyone, anywhere to unlock new revenue streams and drive business value, without relying on proprietary formats, complex ETL processes or costly data replication.
“Unity Catalog allowed us to create a unified view of our data estate, simplifying collaboration across teams within BlackBerry. We now have a standard approach to manage access permissions and audit files or tables in our lake, with the ability to define fine-grained access controls on rows and columns. Automated data lineage helped us see where the data is coming from to pinpoint the source of a potential threat and to understand which research projects or teams are leveraging the data for threat detection.”
“With Unity Catalog, the ability to manage both table and even Amazon S3 access more like a traditional database allows us to have much finer-grained access control than what we had before. We also have more documented lineage for our pipelines and an account-level metastore. All this granularity is why we migrated off Hive metastore.”
“Unity Catalog has improved governance, increased utilization of datasets and made data accessible to external systems and clients in a thoughtful way. It has enabled the company to organize over 150,000 tables and assign permissions effectively. This has made ~70% of the custom cloud infrastructure resources from the previous RBAC architecture obsolete.”
Integrations
Unity Catalog works with your existing data catalogs, data storage systems and governance solutions so you can leverage your existing investments and build a future-proof governance model without expensive migration costs.
