Databricks Runtime

Simplify operations and get up to 50x better performance with cloud-optimized Apache Spark™.

Optimized I/O Performance

The Databricks I/O module (DBIO) takes processing speeds to the next level with an optimized AWS S3 access layer — significantly improving the performance of Apache Spark in the cloud.

HIGHER S3 THROUGHPUT

Improves read and write performance of your Spark jobs.

MORE EFFICIENT DECODING

Boosts CPU efficiency when decoding common formats.

DATA SKIPPING

Allows users to leverage statistics on data files to prune files more effectively in query processing.

TRANSACTIONAL WRITES TO S3

Features transactional (atomic) writes (both appends and new writes) to S3.

TRANSPARENT CACHING

Accelerate read speeds through automated caching of data to a node’s local storage.

DATABRICKS RUNTIME OUTPERFORMS OTHER COMPUTE ENGINES

Serverless Infrastructure

Databricks’ serverless and highly elastic cloud service is designed to remove operational complexity while ensuring reliability and cost efficiency at scale.

SHARED POOLS

Enable hundreds of users to share compute resources, enabling best-in-class performance at dramatically lower costs.

AUTO-CONFIGURATION

Easily set up compute resources that auto-configures Spark for SQL and Python workloads.

AUTO-SCALING

Automatically scale compute and local storage independently based on usage to reduce operational complexity and management cost.

RELIABLE FINE-GRAINED SHARING

Enable sharing of pool resources without compromising on reliability through preemption and fault isolation.

FASTER PERFORMANCE DURING CONCURRENT AND HETEROGENOUS LOADS

PRODUCTION-READY

Built and tuned to be highly performant, secure, and reliable out-of-the-box.

HIGHER S3 THROUGHPUT

Improves read and write performance of your Spark jobs.

QUICK RELEASE CYCLES

Rapid releases and early access to new features and bug fixes that are not yet available in open source releases.

AUTOMATION TECHNOLOGY

Simplifies IT operations and streamlines processes so you can focus on the data rather than DevOps.

SUPPORT FROM THE SPARK EXPERTS

Maximize Spark’s potential on Databricks with unparalleled expertise and support.