Introducing Azure Databricks

99% of organizations still struggle to get valuable analytics from their Big Data and achieve the full potential of AI. Today I’m excited to announce a new partnership with Microsoft that represents a major leap forward in achieving Databricks’ mission of making Big Data and AI easy: Databricks will now be offered by Microsoft as an integrated Azure service.

Databricks’ cloud service is built by the team that started the Spark research project at UC Berkeley that later became Apache Spark and is the leading Spark-based analytics platform. This new service, named Microsoft Azure Databricks, provides data science and data engineering teams with a fast, easy and collaborative Spark-based platform on Azure. It gives Azure users a single platform for Big Data processing and Machine Learning.

Azure Databricks is a “first party” Microsoft service, the result of a unique year-long collaboration between the Microsoft and Databricks teams to provide Databricks’ Apache Spark-based analytics service as an integral part of the Microsoft Azure platform. We’ve ensured this offering is natively integrated with Microsoft Azure in a number of ways ranging from a single click start to a unified billing. Azure Databricks leverages Azure’s security and seamlessly integrates with Azure services such as Azure Active Directory, SQL Data Warehouse, and Power BI.

Both teams invested a huge amount of effort in this project throughout 2017 and we’ve all spent many hours on the “Nerd Bird” flights between Seattle and San Francisco. So it’s very exciting to be able to share with the world this jointly-developed Spark analytics cloud service. But more importantly, it is extremely gratifying to bring it to market in the interest of making Big Data analytics and AI much more approachable to the Azure community and to our customers.

To learn more about Azure Databricks, visit our Azure Databricks web page where you can watch the Azure Databricks demo or request access to the Azure Databricks preview.