Adaptive Query Execution in Structured StreamingJune 2, 2023 by Steven Chen, MaryAnn Xue and Jungtaek Lim in Engineering Blog In Databricks Runtime, Adaptive Query Execution (AQE) is a performance feature that continuously re-optimizes batch queries using runtime statistics during query execution. Starting...
Faster SQL: Adaptive Query Execution in DatabricksOctober 21, 2020 by MaryAnn Xue and Allison Wang in Engineering Blog Earlier this year, Databricks wrote a blog on the whole new Adaptive Query Execution framework in Spark 3.0 and Databricks Runtime 7.0. The...
Adaptive Query Execution: Speeding Up Spark SQL at RuntimeMay 29, 2020 by Wenchen Fan, Herman van Hövell and MaryAnn Xue in Engineering Blog Read Rise of the Data Lakehouse to explore why lakehouses are the data architecture of the future with the father of the data...
SQL Pivot: Converting Rows to ColumnsNovember 1, 2018 by MaryAnn Xue in Engineering Blog Try this notebook in Databricks Check out the Why the Data Lakehouse is Your Next Data Warehouse ebook to discover the inner workings...