Announcing the General Availability of Private Link and Customer Managed Keys for Azure Databricks

We are excited to announce that Private Link and using customer-managed keys (CMK) for encryption are now Generally Available (GA) for Azure Databricks! We know that data is your most valuable asset, and the GA of these two key security features will deliver additional control and protection of your data - at rest and in transit - on the Databricks Lakehouse Platform.

Private Link and customer-managed keys are two of the most sought after features for customers in highly regulated industries such as Financial Services and Health and Life Sciences. With general availability, customers can leverage Private Link and customer-managed keys in environments that require a GA guarantee, extending the benefits of the Databricks Lakehouse Platform to even their most sensitive use cases.

This blog will highlight the benefits of using Private Link and CMK for Azure Databricks, including how to get started with these features today.

Secure your data with Azure Private Link

Many customers want the guarantee of private networking to ensure that their users can access data without exposing traffic to a public network. Azure Private Link provides a private network route from one Azure environment to another. Now, Azure Databricks customers can configure Private Link between Databricks users and the control plane and between the control plane and the data plane. Using Private Link on Azure Databricks provides the following benefits:

• End-to-end private networking: With Private Link, you can set up Azure Databricks workspaces that route traffic privately from your users to your data and back again. Routing traffic on private networks substantially reduces the risk of accidental misconfiguration or traffic inspection by very advanced attackers.
• Data exfiltration protection: Private Link endpoints grant access to specific resources, allowing you to tightly control network access. In the event of a security incident within your network, only the mapped resource would be accessible, significantly reducing the attack surface for data exfiltration.
• Meet compliance requirements: With Private Link, you can set up a secure perimeter around your data so that the data is only processed in trusted private networks. This helps you to meet compliance requirements for even your most sensitive workloads.

Protect your data at rest with customer-managed keys

Azure Databricks encrypts customer content at rest by default within our control plane, but some customers may prefer or require the ability to use customer-managed keys for added control. With Microsoft Key Vault for Azure Databricks, customers on Azure can now bring their own encryption keys to protect data in managed services and workspace storage, such as notebooks, secrets, Databricks SQL queries, Databricks SQL query history, and Managed disk volumes.

Using customer-managed keys for Azure Databricks provides the following benefits:

• More control over your data: Because you manage the key needed to decrypt your data, you have overall control over how and when it can be used. If you delete or revoke access to your key, it isn't possible for Azure Databricks (or anyone else) to decrypt that data.
• Greater reassurance in the event of a compromise: Like all of the best security teams in the world, we hope for the best but plan for the worst. In the event of a security compromise, you can simply revoke access to your CMK and, with it, our ongoing access to your data.
• Enforce your own rotation policies: If you use a platform-managed key (PMK), the owner rotates the key per their compliance policy. With a CMK you can rotate the key as per your compliance policy.
• Monitor access: As well as greater control, you have visibility over how and when your key is being used. You can use cloud-native monitoring solutions to track the use of your CMK and detect any unauthorized attempts to access your data.

Getting Started with Private Link and CMK on Azure Databricks

Private Link and customer-managed keys are available on the Premium Tier version of Azure Databricks. For step-by-step instructions on configuring these features for your Azure Databricks workspaces, refer to our documentation (Private Link | CMK).

Please visit our Security and Trust Center for more information about Databricks security practices and features available to customers.