Introducing Support for gp3, Amazon’s New General Purpose SSD Volume

by

Databricks clusters on AWS now support gp3 volumes, the latest generation of Amazon Elastic Block Storage (EBS) general purpose SSDs. gp3 volumes offer consistent performance, cost savings and the ability to configure the volume’s iops, throughput and volume size separately. Databricks on AWS customers can now easily switch to gp3 for the better price/performance storage...

How Databricks’ Data Team Built a Lakehouse Across 3 Clouds and 50+ Regions

by and

The internal logging infrastructure at Databricks has evolved over the years and we have learned a few lessons along the way about how to maintain a highly available log pipeline across multiple clouds and geographies. This blog will give you some insight as to how we collect and administer real-time metrics using our Lakehouse platform,...

Time Series Data Analytics in Financial Services with Databricks and KX

by and

This is a guest co-authored post. We thank Connor Gervin, partner engineering lead, KX, for his contributions. KX recently announced a partnership with Databricks making it possible to cover all the use cases for high-speed time-series data analytics. Today we’re going to explain the integration options available between both platforms for both streaming and batch...

Announcing the Labelbox Connector for Databricks

by

This is a guest authored post by Nick Lee, partnership integration lead, at Labelbox Large data lakes typically house a combination of structured and unstructured data. Data teams often use Apache Spark™ to analyze structured data, but may struggle to apply the same analysis to unstructured, unlabeled data (specifically in the form of images, video,...

Leverage Unused Compute Capacity for Data + AI With Azure Spot Instances and Azure Databricks

by , and

Azure Databricks support for Microsoft Azure Spot Virtual Machines (Spot VMs) is now generally available. Together, Spot VMs and Azure Databricks help innovative customers like aluminum and energy producer Hydro accelerate data + AI workloads while optimizing costs. By using Spot VMs as workers for Azure Databricks clusters, you can save up to 90%* on...

Security Best Practices for AWS on Databricks

by , and

The Databrick Lakehouse Platform is the world’s first lakehouse architecture -- an open, unified platform to enable all of your analytics workloads. A lakehouse enables true cross-functional collaboration across data teams of data engineers, data scientists, ML engineers, analysts and more. In this article, we will share a list of cloud security features and capabilities...

Customer-managed Key (CMK) Public Previews for Databricks on Azure and AWS

by , and

We’re excited to release the Customer-managed key (CMK) public previews for Azure Databricks and Databricks workspaces on AWS (Amazon Web Services), with full support for production deployments. On Microsoft Azure, you can now use your own key to encrypt the notebooks and queries managed by Azure Databricks; this capability is available in the Premium pricing...

Improved Tableau Databricks Connector With Azure AD Authentication Support

by , and

With the release of Tableau 2021.1, we have added new functionality to the Tableau Databricks Connector that simplifies security administration and streamlines the connection experience for end users. The updated connector lets Tableau users connect to Azure Databricks with a couple of clicks, using Azure Active Directory (Azure AD) credentials and SSO for Tableau Online...

Sign up