The Open Source Delta Lake Project is now hosted by the Linux Foundation.
LEARN MORE >
Learn about Apache Spark, Delta Lake, MLflow, TensorFlow, deep learning, applying software engineering principles to data engineering and machine learning
LEARN MORE >
Spark + AI Summit --- The Virtual Event for Data Teams
NOW ON DEMAND
The Open Source Delta Lake Project is now hosted by the Linux Foundation.
LEARN MORE >
Join us to help data teams solve the world's toughest problems
SEE JOBS >
This is a joint engineering effort between Databricks’ Apache Spark engineering team (Sameer Agarwal and Wenchen Fan) and Huawei’s engineering team (Ron Hu and Zhenhua Wang) Apache Spark 2.2 recently shipped with a state-of-art cost-based optimization framework that collects and leverages a variety of per-column data statistics (e.g., cardinality, number of distinct values, NULL values,...
