Simplify Access Policy Management With Privilege Inheritance in Unity Catalog

Unity Catalog, now generally available on AWS and Azure, provides a unified governance solution for data, analytics and AI on the lakehouse. Thousands of our customers are now leveraging Unity Catalog for simplified access management and audit controls with a familiar SQL interface, cross-platform data sharing, seamless data observability with automated lineage, and boosting the productivity of their data teams with easy data discovery and auto-generated data insights.

In this blog, we explore how data administrators can leverage privilege inheritance in Unity Catalog to simplify access policy management at scale.

Privilege inheritance in Unity Catalog

Managing access permissions on each object such as a table or view in a database (or schema), especially if you have 100s or 1,000s of data objects in many databases, is painstaking and time-consuming for data administrators. Privilege inheritance makes it easy to setup and manage access privileges to these objects over time. Administrators can assign a group of users access to a database (or schema) and the access rights cascade – or inherit down – to tables contained in the schema. This includes tables that exist now or may be created in the future and provides an easy way to set safe access defaults on catalogs and schemas.

Privilege inheritance becomes an even more powerful tool for data administrators when combined with Unity Catalog's expanded 3-level namespace (<catalog>.<schema>.<table>) because you can leverage inheritance right where it is needed. In the example below, you can grant the SELECT privilege on all tables and views in any schema found in the catalog main to the members of the finance group:

GRANT SELECT ON CATALOG main TO finance;

Or you can grant the privilege at the schema level for a smaller scope of access, as shown in this example:

GRANT SELECT ON SCHEMA main.default TO finance;

The inheritance model provides an easy way to set up default access rules for your data. For example the following commands enable the machine learning team to create tables within a schema and read each other's tables:

CREATE CATALOG ml;
CREATE SCHEMA ml.sandbox;
GRANT USE_CATALOG ON CATALOG ml TO ml_users;
GRANT USE_SCHEMA ON SCHEMA ml.sandbox TO ml_users;
GRANT CREATE TABLE ON SCHEMA ml.sandbox TO ml_users;
GRANT SELECT ON SCHEMA ml.sandbox TO ml_users;

Privileges are still granted by Unity Catalog metastore admins, by the owner of an object, or by the owner of the catalog or schema containing the object. And you can use SQL commands, the Unity Catalog CLI, or the new Data Explorer UI to manage privileges.

New and changed privilege types in Unity Catalog

You'll notice that CREATE and USAGE has been replaced by more specific privilege types. For example, CREATE is replaced by CREATE CATALOG at the metastore level, CREATE SCHEMA at the catalog level, and CREATE TABLE at the schema level. Similarly, USAGE is replaced by USE CATALOG at the catalog level, and USE SCHEMA at the schema level. This provides the flexibility you need to precisely control what users can do at each level in Unity Catalog.

There is also a new ALL PRIVILEGES type that can be assigned at or below the Catalog level to give all present (and future) privilege types to the subject of the grant. For a complete list of privilege types, see the topic Unity Catalog privileges and securable objects (AWS, Azure).

If you created your Unity Catalog metastore during the public preview (before August 25, 2022), you can upgrade to Privilege Model version 1.0. to take advantage of privilege inheritance. Existing workloads will continue to operate as-is until you upgrade your privilege model. Databricks recommends upgrading to Privilege Model version 1.0 to get the benefits of privilege inheritance and new features. To learn more, see the topic Upgrade to privilege inheritance (AWS, Azure).

Getting started with Unity Catalog

To get started with Unity Catalog, see the setup guides (AWS, Azure). To learn more about Unity Catalog privileges please visit the topic Unity Catalog privileges and securable objects (AWS, Azure).

You can also import these notebooks for a walkthrough of some common Unity Catalog administration tasks:

• Unity Catalog Quickstart (SQL)
• Unity Catalog Quickstart (Python)
• Creating an External Table in Unity Catalog (AWS)
• Creating an External Table in Unity Catalog (Azure)

Upcoming roadmap for access policy management in Unity Catalog

• Attribute Based Access Controls: Define access policies based on tags (attributes) of your data assets.
• Row filtering and column masking: Use standard SQL functions to define row filters and column masks, allowing fine-grained access controls on rows and columns.