Horizontally Scalable Relational Databases with Spark - Databricks

Horizontally Scalable Relational Databases with Spark

Download Slides

Scaling out doesn’t have to mean giving up transactions and efficient joins! Relational databases can scale horizontally, and using them as a store for Spark Streaming or batch computations can help cover areas in which Spark is typically weaker. Examples will be drawn from our experience using Citus (https://github.com/citusdata/citus), an open-source extension to Postgres , but lessons learned should be applicable to many databases.