Articles by Sirui Sun - Databricks Blog

Page 1

Announcing GA of Predictive I/O for Updates: Faster DML Queries, Right Out of the Box

November 2, 2023 by Sirui Sun, Ala Luszczak, Sabir Akhadov, Andreas Chatzistergiou, Carmen Kwan, Ole Sasse, Tom van Bussel, Lars Kroll, Pengfei Xu, Polo-François Poli, Bart Samwel, Himanshu Raja, Christos Stavrakakis, Venki Korukanti, Tathagata Das and Nicolas Poggi in Platform & Products & Announcements

Announcing GA of Predictive I/O for Updates, which harnesses Photon and AI atop Deletion Vectors in order to significantly speed up MERGE, UPDATE and DELETE operations. We've prevented over 15 trillion unnecessary row writes, and we’re just getting started.

Introducing Predictive Optimization: Faster Queries, Cheaper Storage, No Sweat

October 26, 2023 by Sirui Sun, Vijayan Prabhakaran, Naga Raju Bhanoori, Rajesh Parangi Sharabhalingappa, Jintian Liang, Kam Cheung Ting, Vivek Narasimhan, Sunitha Beeram, Himanshu Raja, Rahul Potharaju, Reynold Xin, Matei Zaharia and Susan Pierce in Platform Blog

Predictive Optimization intelligently optimizes your Lakehouse table data layouts for peak performance and cost-efficiency - without you needing to lift a finger.

Delta UniForm: a universal format for lakehouse interoperability

August 14, 2023 by Bilal Obeidat, Sirui Sun, Adam Wasserman, Susan Pierce, Fred Liu, Ryan Johnson and Himanshu Raja in Engineering Blog

Update: BigQuery now offers native support for Delta Lake through BigLake. Check out the documentation for more information. One of the key challenges...

Announcing Delta Lake 3.0 with New Universal Format and Liquid Clustering

June 29, 2023 by Ryan Johnson, Michael Armbrust, Reynold Xin, Denny Lee, Tathagata Das, Bart Samwel, Terry Kim, Sirui Sun, Himanshu Raja, Rahul Potharaju, Juan Yu and Susan Pierce in Engineering Blog

We are excited to announce Delta Lake 3.0, the next major release of the Linux Foundation open source Delta Lake Project, available in...

How Databricks improved query performance by up to 2.2x by automatically optimizing file sizes

May 22, 2023 by Sirui Sun, Himanshu Raja, Vijayan Prabhakaran, Terry Kim, Bart Samwel, Rahul Mahadev, Rajesh Parangi Sharabhalingappa, Rahul Potharaju and Kam Cheung Ting in Platform Blog

Optimizing tables has long been a necessary but complicated task for data engineers. One particularly thorny area has been getting to the optimal...