Databricks announces the availability of a major release this week. In addition to several new usability and portability features, this release also provides a preview to the not-yet-released Apache Spark 1.6. Databricks users have an opportunity to experiment with Spark 1.6 features before the official release, easily search through Spark documentation within the Databricks integrated workspace, and share Databricks notebooks between different Databricks instances with a few clicks. In this blog, I’ll provide a brief tour of these exciting additions.
Apache Spark 1.6 Preview
The preview of Spark 1.6 was announced by Patrick Wendell last week. We are excited to make this preview available to allow our users to take advantage of the rapid evolution of the open source project. This preview access to Spark 1.6 is currently available on Databricks.
Having the preview in Databricks means that our users can launch the preview package by simply choosing Version 1.6.0 (Preview) from the Databricks Cluster Manager UI. They can create 1.6 clusters alongside their clusters with older versions, or test existing code on Spark 1.6 to see performance improvements from earlier versions of Spark - currently Databricks also offers versions 1.3, 1.4.1, and 1.5. Being able to run multiple Spark versions allows our users to experiment with cutting edge features while still maintaining the stability of their existing production environments.
Patrick will also give a webinar on Spark 1.6 on December 1st, sign-up today to get the details.
Improved Portability of Databricks Notebooks
A few weeks ago, we announced a new feature that allows users to export notebooks into HTML format. We have extended this functionality to allow the import of previously exported HTML notebooks into any Databricks instance. This gives our users more options to share notebooks and collaborate - including collaboration across different Databricks instances.
Easier Spark Documentation Search
The Databricks integrated search feature has been an easy way for our users to find relevant information. With this release, a user can also easily search through the official Spark documentation to locate the most up-to-date information.
Looking Forward
Being a SaaS platform, Databricks gets updated in rapid iterations to improve the user experience continuously. If you already have a Databricks account, we welcome you to try out these new features and provide feedback. If you are interested in taking Databricks for a spin, contact one of our solution architects or sign up for a trial today.