Introducing Databricks Optimized Autoscaling on Apache Spark™May 2, 2018 by Prakash Chockalingam, Eric Liang, Tathagata Das and Jean-Yves Stephan in Product Databricks is thrilled to announce our new optimized autoscaling feature. The new Apache Spark™-aware resource manager leverages Spark shuffle and executor statistics to...