Today, MapR announced that it will distribute and support the Apache Spark platform as part of the MapR Distribution for Hadoop in partnership with Databricks. We’re thrilled to start on this journey with MapR for a multitude of reasons.
One of our primary goals at Databricks is to drive broad adoption of Spark and ensure everybody who uses it has a fantastic experience. This partnership will enable all of MapR’s enterprise customers, existing and new, to leverage Spark with the backing of the same great enterprise support available for the rest of MapR’s Hadoop Distribution. As Tomer mentioned in his blog post, Spark is one of the most common topics in discussions with MapR’s existing customers and many are even already running it in production!
A core part of Spark’s value proposition is the ability to easily build a unified end-to-end workflow where critical functions are first class citizens that are seamlessly integrated into the platform. An important part of this workflow is the ability to provide SQL-based interactive queries delivered by Shark, which also serves as the gateway for a wealth of traditional and new SQL-based tools to run on top of Spark. At Databricks, we continue to work on innovating across the entire Spark platform, including Shark, and customers now have an enterprise support option for Shark available.
Finally, we see this partnership as continued validation of Spark’s emergence as the leading open source processing engine in the Big Data community. Spark is the most active project in the Hadoop ecosystem in the past year with over 170 contributors, and we’re heartened to see a rapidly growing attendance at community events pointing to entirely new classes of Spark enterprise use cases. The Databricks “Certified on Spark” program has seen incredible interest from application developers who are leveraging Spark to deliver deeper insights, faster to their customers.
At Databricks we are fully committed to open source, and we’re excited to partner with MapR - a company with strong support for open source Big Data projects - to together help drive continued growth and innovation of Spark. Join us at the upcoming Spark Summit, the largest conference focused on Spark, to learn more!