Introducing Support for gp3, Amazon’s New General Purpose SSD Volume
Databricks clusters on AWS now support gp3 volumes, the latest generation of Amazon Elastic Block Storage (EBS) general purpose SSDs. gp3 volumes offer consistent performance, cost savings and the ability to configure the volume’s iops, throughput and volume size separately. Databricks on AWS customers can now easily switch to gp3 for the better price/performance storage...
Improving Patient Insights With Textual ETL in the Lakehouse Paradigm
This is a collaborative post from Databricks and Forest Rim Technology. We thank Bill Inmon, Founder and CEO, and Mary Levins, Chief Data Officer, of Forest Rim for their contributions. The amount of healthcare data generated today is unprecedented and rapidly expanding with the growth in digital patient care. Yet much of the data...
How Databricks’ Data Team Built a Lakehouse Across 3 Clouds and 50+ Regions
The internal logging infrastructure at Databricks has evolved over the years and we have learned a few lessons along the way about how to maintain a highly available log pipeline across multiple clouds and geographies. This blog will give you some insight as to how we collect and administer real-time metrics using our Lakehouse platform,...
Time Series Data Analytics in Financial Services with Databricks and KX
This is a guest co-authored post. We thank Connor Gervin, partner engineering lead, KX, for his contributions. KX recently announced a partnership with Databricks making it possible to cover all the use cases for high-speed time-series data analytics. Today we’re going to explain the integration options available between both platforms for both streaming and batch...
Databricks Announces 2021 North America Partner Awards
The Databricks Partner Ecosystem of over 450 partners globally are critical to building and delivering the best data and AI solutions in the world. We are proud to collaborate with our partners and recognize that success of our joint customers is the result of mutual commitment and investments in training, solution development, and our ongoing...
Announcing the Labelbox Connector for Databricks
This is a guest authored post by Nick Lee, partnership integration lead, at Labelbox Large data lakes typically house a combination of structured and unstructured data. Data teams often use Apache Spark™ to analyze structured data, but may struggle to apply the same analysis to unstructured, unlabeled data (specifically in the form of images, video,...
Leverage Unused Compute Capacity for Data + AI With Azure Spot Instances and Azure Databricks
Azure Databricks support for Microsoft Azure Spot Virtual Machines (Spot VMs) is now generally available. Together, Spot VMs and Azure Databricks help innovative customers like aluminum and energy producer Hydro accelerate data + AI workloads while optimizing costs. By using Spot VMs as workers for Azure Databricks clusters, you can save up to 90%* on...
Security Best Practices for AWS on Databricks
The Databrick Lakehouse Platform is the world’s first lakehouse architecture -- an open, unified platform to enable all of your analytics workloads. A lakehouse enables true cross-functional collaboration across data teams of data engineers, data scientists, ML engineers, analysts and more. In this article, we will share a list of cloud security features and capabilities...
Customer-managed Key (CMK) Public Previews for Databricks on Azure and AWS
We’re excited to release the Customer-managed key (CMK) public previews for Azure Databricks and Databricks workspaces on AWS (Amazon Web Services), with full support for production deployments. On Microsoft Azure, you can now use your own key to encrypt the notebooks and queries managed by Azure Databricks; this capability is available in the Premium pricing...
Improved Tableau Databricks Connector With Azure AD Authentication Support
With the release of Tableau 2021.1, we have added new functionality to the Tableau Databricks Connector that simplifies security administration and streamlines the connection experience for end users. The updated connector lets Tableau users connect to Azure Databricks with a couple of clicks, using Azure Active Directory (Azure AD) credentials and SSO for Tableau Online...
