New cost savings option for Azure Databricks with DBU pre-purchase
The rapid adoption of Azure Databricks through our strategic partnership with Microsoft has been remarkable, and it’s proven to be a compelling service for our customers' big data, analytics and machine learning initiatives. To further help our customers save costs and improve budgeting for Azure Databricks, we are pleased to share a new pricing option...
Announcing Databricks Runtime 5.5 and Runtime 5.5 for Machine Learning
Databricks is pleased to announce the release of Databricks Runtime 5.5. This release includes Apache Spark 2.4.3 along with several important improvements and bug fixes as noted in the latest release notes [Azure|AWS]. We recommend all users upgrade to take advantage of this new runtime release. This blog post gives a brief overview of some...
Getting Data Ready for Data Science: On-Demand Webinar and Q&A Now Available
On June 25th, our team hosted a live webinar — Getting Data Ready for Data Science — with Prakash Chockalingam, Product Manager at Databricks. Successful data science relies on solid data engineering to furnish reliable data. Data lakes are a key element of modern data architectures. Although data lakes afford significant flexibility, they also face...
Scaling Genomic Workflows with Spark SQL BGEN and VCF Readers
In the past decade, the amount of available genomic data has exploded as the price of genome sequencing has dropped. Researchers are now able to scan for associations between genetic variation and diseases across cohorts of hundreds of thousands of individuals from projects such as the UK Biobank. These analyses will lead to a deeper...
Announcing Databricks Runtime 5.4
Databricks is pleased to announce the release of Databricks Runtime 5.4. This release includes Apache Spark 2.4.3 along with several important improvements and bug fixes . We recommend all users upgrade to take advantage of this new runtime release. This blog post gives a brief overview of some of the new high value features that...
Databricks Connect: Bringing the capabilities of hosted Apache Spark™ to applications and microservices
In this blog post we introduce Databricks Connect, a new library that allows you to leverage native Apache Spark APIs from any Notebook, IDE, or custom application. Overview Over the last several years, many custom application connectors have been written for Apache Spark. This includes tools like spark-submit, REST job servers, notebook gateways, and so...
Enhanced Hyperparameter Tuning and Optimized AWS Storage with Databricks Runtime 5.4 ML
We are excited to announce the release of Databricks Runtime 5.4 ML (Azure | AWS). This release includes two Public Preview features to improve data science productivity, optimized storage in AWS for developing distributed applications, and a number of Python library upgrades. To get started, you simply select the Databricks Runtime 5.4 ML from the...
Introducing Databricks Runtime 5.4 with Conda (Beta)
We are excited to introduce a new runtime: Databricks Runtime 5.4 with Conda (Beta). This runtime uses Conda to manage Python libraries and environments. Many of our Python users prefer to manage their Python environments and libraries with Conda, which quickly is emerging as a standard. Conda takes a holistic approach to package management by...
Databricks is a Diamond Partner at Snowflake Summit
Here at Databricks, we are excited to participate in the first Snowflake Summit as a Diamond Partner. The event takes place June 3-6 at the Hilton San Francisco Union Square and is another great opportunity to share how Databricks and Snowflake have partnered together to provide: Massively scalable data pipelines. Pipelines running on Databricks can...
Databricks and Informatica Accelerate Development and Complete Data Governance for Intelligent Data Pipelines
The value of analytics and machine learning to organizations is well understood. Our recent CIO survey showed that 90% of organizations are investing in analytics, machine learning and AI. But we’ve also noted that the biggest barrier is getting the right data in the right place and in the right format. So we’ve partnered with...
