Skip to main content
Page 1

Announcing GA of Predictive I/O for Updates: Faster DML Queries, Right Out of the Box

November 2, 2023 by Sirui Sun, Ala Luszczak, Sabir Akhadov, Andreas Chatzistergiou, Carmen Kwan, Ole Sasse, Tom van Bussel, Lars Kroll, Pengfei Xu, Polo-François Poli, Bart Samwel, Himanshu Raja, Christos Stavrakakis, Venki Korukanti, Tathagata Das and Nicolas Poggi in Platform & Products & Announcements
Announcing GA of Predictive I/O for Updates, which harnesses Photon and AI atop Deletion Vectors in order to significantly speed up MERGE, UPDATE and DELETE operations. We've prevented over 15 trillion unnecessary row writes, and we’re just getting started.

Introducing Predictive Optimization: Faster Queries, Cheaper Storage, No Sweat

October 26, 2023 by Sirui Sun, Vijayan Prabhakaran, Naga Raju Bhanoori, Rajesh Parangi Sharabhalingappa, Jintian Liang, Kam Cheung Ting, Vivek Narasimhan, Sunitha Beeram, Himanshu Raja, Rahul Potharaju, Reynold Xin, Matei Zaharia and Susan Pierce in Platform Blog
Predictive Optimization intelligently optimizes your Lakehouse table data layouts for peak performance and cost-efficiency - without you needing to lift a finger.

Delta UniForm: a universal format for lakehouse interoperability

August 14, 2023 by Bilal Obeidat, Sirui Sun, Adam Wasserman, Susan Pierce, Fred Liu, Ryan Johnson and Himanshu Raja in Engineering Blog
Update: BigQuery now offers native support for Delta Lake through BigLake. Check out the documentation for more information. One of the key challenges...

Announcing Delta Lake 3.0 with New Universal Format and Liquid Clustering

June 28, 2023 by Ryan Johnson, Michael Armbrust, Reynold Xin, Denny Lee, Tathagata Das, Bart Samwel, Terry Kim, Sirui Sun, Himanshu Raja, Rahul Potharaju, Juan Yu and Susan Pierce in Engineering Blog
We are excited to announce Delta Lake 3.0, the next major release of the Linux Foundation open source Delta Lake Project, available in...

How Databricks improved query performance by up to 2.2x by automatically optimizing file sizes

May 22, 2023 by Sirui Sun, Himanshu Raja, Vijayan Prabhakaran, Terry Kim, Bart Samwel, Rahul Mahadev, Rajesh Parangi Sharabhalingappa, Rahul Potharaju and Kam Cheung Ting in Platform Blog
Optimizing tables has long been a necessary but complicated task for data engineers. One particularly thorny area has been getting to the optimal...

Announcing the Public Preview of Predictive I/O for Updates

April 25, 2023 by Piyush Revuri, Bart Samwel, Ala Luszczak, Lars Kroll, Polo-François Poli, Frank Munz and Himanshu Raja in Platform Blog
Previously, we’ve shown you how a new technology called Predictive I/O could improve selective reads by up to 35x for CDW customers without...

Faster MERGE Performance With Low-Shuffle MERGE and Photon

October 16, 2022 by Bart Samwel, Piyush Revuri, Himanshu Raja, Justin Breese, Tom van Bussel, Lars Kroll, Sabir Akhadov, Tathagata Das and Prakhar Jain in Platform Blog
At Databricks, one of our key goals is to provide our customers with an industry-best price/performance experience out of the box. From ETL...

Now Generally Available: Introducing Databricks Partner Connect to Discover and Connect Popular Data and AI Tools to the Lakehouse

November 18, 2021 by Hiral Jasani and Himanshu Raja in Platform Blog
Databricks is thrilled to announce Partner Connect, a one-stop portal for customers to quickly discover a broad set of validated data, analytics, and...

Announcing Public Preview of Low Shuffle Merge

September 8, 2021 by Himanshu Raja in Platform Blog
Today, we are excited to announce the public preview of Low Shuffle Merge in Delta Lake , available on AWS, Azure, and Google...