What’s New for Apache Spark on Kubernetes in Apache Spark 2.4 Release - The Databricks Blog