Databricks and Posit announce new integrations, simplifying Lakehouse access for developers

Today, Databricks and Posit are thrilled to announce a strategic partnership to provide R and Python developers with an simplified development experience when working with Databricks. To meet this goal, we are collaborating with product and engineering teams at Posit to deliver an enhanced integration between Posit Workbench and Databricks, support for Spark Connect in R, and the hosting of Posit Workbench on the Databricks Marketplace. Let's take a closer look:

Enhanced access to your Databricks Workspace from Posit Workbench

Posit Workbench is the standard for enterprise deployments of RStudio IDE, the fan-favorite development experience for R developers everywhere. The new Databricks integration with Posit Workbench will simplify connecting to Databricks Workspaces, accessing data, running code, and interacting with other resources like Databricks Workflows or MLflow models. We are very excited to be able to bring Databricks to wherever you have Posit Workbench installed!

Support for Spark Connect in R via sparklyr

Since Apache Spark 3.4, Spark Connect was introduced to decouple the client-server architecture, allowing IDEs and modern data applications to access Spark interactively with minimal setup. Now, Posit is adding support for Spark Connect in R via sparklyr, making it easier for users to access Spark clusters - including Databricks clusters via Databricks Connect. For many R developers, sparklyr is the preferred method to work with Apache Spark due to its unification with dplyr and the tidyverse ecosystem as well as its integration with the Connections pane in RStudio IDE. sparklyr will continue this tradition and improve parity with Python through wrapping the PySpark API, giving R developers more choices when it comes to writing Spark applications.

Posit Workbench in Databricks Marketplace

Mutual customers often ask us if Posit products can be hosted within their Databricks Workspace. Today, this is limited to RStudio Server or Posit Workbench installed directly on Databricks compute resources in the data plane. Bringing Posit Workbench to Databricks Marketplace allows for a much deeper integration. Marketplace offerings can run directly on a customer's Databricks instance, integrate with their data, and enable users to interact through single sign-on: combining the highest standards of security, privacy, compliance, and scalability with data never having to leave the customer's account. Customers can enjoy their favorite developer tooling right on the shores of their Databricks Lakehouse.

As Databricks and Posit further strengthen their partnership, the data community stands to reap the benefits of these exciting new developments. The sparklyr package, improved integration of Posit Workbench with Databricks, and the addition of Posit Workbench in the Databricks Marketplace demonstrates our commitment to providing comprehensive solutions for our shared customers. We look forward to witnessing the transformative impact of these advancements on our customers' data-driven journeys.

Please sign up here to get notified of product updates.