Adaptive Query Execution: Speeding Up Spark SQL at Runtime
Read Rise of the Data Lakehouse to explore why lakehouses are the data architecture of the future with the father of the data...
Schema Evolution in Merge Operations and Operational Metrics in Delta Lake
Get an early preview of O'Reilly's new ebook for the step-by-step guidance you need to start using Delta Lake. Try this notebook to...
Shrink Training Time and Cost Using NVIDIA GPU-Accelerated XGBoost and Apache Spark™ on Databricks
Guest Blog by Niranjan Nataraja and Karthikeyan Rajendran of Nvidia. Niranjan Nataraja is a lead data scientist at Nvidia and specializes in building...
Now on Databricks: A Technical Preview of Databricks Runtime 7 Including a Preview of Apache Spark 3.0
Introducing Databricks Runtime 7.0 Beta We’re excited to announce that the Apache Spark TM 3.0.0-preview2 release is available on Databricks as part of...
Glow 0.3.0 Introduces New Large-Scale Genomic Analysis Features
In October of last year, Databricks and the Regeneron Genetics Center ® partnered together to introduce Project Glow , an open-source analysis tool...
COVID-19 Datasets Now Available on Databricks: How the Data Community Can Help
Initially published April 14th, 2020; updated April 21st, 2020 With the massive disruption of the current COVID-19 pandemic, many data engineers and data...
10 Minutes from pandas to Koalas on Apache Spark
This is a guest community post from Haejoon Lee, a software engineer at Mobigen in South Korea and a Koalas contributor. pandas is...
Trust but Verify with Databricks
As enterprises modernize their data infrastructure to make data-driven decisions, teams across the organization become consumers of that platform. The data workloads grow...
Developing Shiny Applications in Databricks
Join our live webinar hosted by Data Science Central on March 12 to learn more We are excited to announce that you can...
Data Quality Monitoring on Streaming Data Using Spark Streaming and Delta Lake
Try this notebook to reproduce the steps outlined below In the era of accelerating everything, streaming data is no longer an outlier- instead...