We are happy to announce the release of Apache Spark 0.8.1. In addition to performance and stability improvements, this release adds three new features. First, Spark now supports for the newest versions of YARN (2.2+). Second, the standalone cluster manager supports a high-availability mode in which it can tolerate master failures. Third, shuffles have been optimized to create fewer files, improving shuffle performance drastically in some settings.
In conjunction with the Apache Spark 0.8.1 release we are separately releasing Spark In MapReduce (SIMR), which enables seamlessly running Spark on Hadoop MapReduce v1 clusters without requiring the installation of Scala or Spark.
While Apache Spark 0.8.1 is a minor release, it includes these larger features for the benefit of Scala 2.9 users. The next major release of Apache Spark, 0.9.0, will be based on Scala 2.10.
This release was a community effort, featuring contributions from more than 40 developers. For much more information about Apache Spark 0.8.1, head over to the release notes or download Apache Spark 0.8.1 and try it out for yourself.