Scalable Partition Handling for Cloud-Native Architecture in Apache Spark 2.1 - The Databricks Blog