Security Best Practices for AWS on Databricks
The Databrick Lakehouse Platform is the world’s first lakehouse architecture -- an open, unified platform to enable all of your analytics workloads. A lakehouse enables true cross-functional collaboration across data teams of data engineers, data scientists, ML engineers, analysts and more. In this article, we will share a list of cloud security features and capabilities...
Customer-managed Key (CMK) Public Previews for Databricks on Azure and AWS
We’re excited to release the Customer-managed key (CMK) public previews for Azure Databricks and Databricks workspaces on AWS (Amazon Web Services), with full support for production deployments. On Microsoft Azure, you can now use your own key to encrypt the notebooks and queries managed by Azure Databricks; this capability is available in the Premium pricing...
Private Databricks Workspaces With AWS PrivateLink Is in Public Preview
We’re excited to announce that PrivateLink connectivity for Databricks workspaces on AWS (Amazon Web Services) is now in public preview, with full support for production deployments. This release applies to all AWS regions supporting E2 architecture, as part of the Enterprise pricing tier. We have received great feedback from our global customers, including large financial...
Security Cluster Connectivity Is Generally Available on Azure Databricks
This is a collaborative post co-authored by Principal Product Manager Premal Shah, Microsoft, and Principal Enterprise Readiness Manager Abhinav Garg, Databricks We're excited to announce the general availability of Secure Cluster Connectivity (also commonly known as No Public IP) on Azure Databricks. This release applies to Microsoft Azure Public Cloud and Azure Government regions, in...
Automate Azure Databricks Platform Provisioning and Configuration
Table of Contents Introduction Automation options Common workflow Pre-Requisites Create Azure Resource Group and Virtual Network Provision Azure Application / Service Principal Assign Role to Service Principal Configure Postman Environment Provision Azure Databricks Workspace Generate AAD Access Token Deploy Workspace using the ARM template Get workspace URL Generate Access Token for Auth Generate AAD Access...
Enterprise Cloud Service Public Preview on AWS
At Databricks, we have had the opportunity to collaborate with companies that have transformed the way people live. Some of our customers have developed life saving drugs, delivered industry-first user experiences, as well as provided edge-of-the-seat entertainment (so needed during shelter in place). These companies transformed their business by building efficiencies in how they operate,...
Azure Databricks Security Best Practices
Azure Databricks is a Unified Data Analytics Platform that is a part of the Microsoft Azure Cloud. Built upon the foundations of Delta Lake, MLflow, Koalas, Redash and Apache SparkTM, Azure Databricks is a first party PaaS on Microsoft Azure cloud that provides one-click setup, native integrations with other Azure cloud services, interactive workspace, and...
Data Exfiltration Protection with Azure Databricks
In the previous blog, we discussed how to securely access Azure Data Services from Azure Databricks using Virtual Network Service Endpoints or Private Link. Given a baseline of those best practices, in this article we walkthrough detailed steps on how to harden your Azure Databricks deployment from a network security perspective in order to prevent...
Trust but Verify with Databricks
As enterprises modernize their data infrastructure to make data-driven decisions, teams across the organization become consumers of that platform. The data workloads grow exponentially, where cloud data lake becomes the centralized storage for enterprise-wide functions and different tools & technologies are used to gain insights out of it. For cloud security teams, the addition of...
Securely Accessing Azure Data Sources from Azure Databricks
Azure Databricks is a Unified Data Analytics Platform that is a part of the Microsoft Azure Cloud. Built upon the foundations of Delta Lake, MLFlow , Koalas and Apache Spark, Azure Databricks is a first party service on Microsoft Azure cloud that provides one-click setup, native integrations with other Azure services, interactive workspace, and enterprise-grade...
