Since 2014, Tableau and Databricks have partnered to improve the speed and user experience of visualizing massive data sets. This is a natural partnership - Databricks provides a Unified Analytics Platform while Tableau provides a highly popular interactive visualization tool. This year, Tableau won the Databricks Partner Innovation Award for our shared effort in creating a brand-new Databricks Connector for Tableau. Together with Tableau, we are pleased to introduce an optimized Tableau-native approach of connecting to Databricks.
Customers have long been able to connect to Databricks using the Spark SQL Connector which has a number of performance and user experience issues. The new Databricks Connector for Tableau builds on the Spark SQL Connector, adding a number of Databricks-specific improvements:
Tableau-native and 100% compliant
The Databricks Connector for Tableau is built using the new Tableau Connector SDK that is the recommended SDK for providing the best experience to Tableau users. As the first partner to launch on the SDK, we have made the Connector 100% compliant with the Tableau Datasource Verification Tool (TDVT). For users, this translates to fewer edge cases where queries are slow or fail entirely.
Noticeably faster initial connection
The first thing you will notice is how much faster it is to connect to Databricks from Tableau. In our tests, we saw significant speedups over the Spark SQL Connector in initial connection times. We achieved this speedup by tuning networking parameters such as how long the driver waits for asynchronous data to be fetched from Databricks.
Improved SQL Generation
We use the Tableau SDK to control the quality of the queries generated by Tableau, and ensure they are correctly translated to the Databricks SQL dialect. As a result, users can connect their interactive Tableau dashboards for faster and more reliable queries.
Simplified Connection Setup
The connection dialog box of the Spark SQL Connector provided a myriad of connection options, some of which were unnecessary because they were never used with Databricks. The new Databricks Connector for Tableau includes only fields that are necessary to establish a connection, making it simpler to configure and install.
We are focusing first on the Databricks Connector for Tableau, because it is a critical link in the query path. Building upon the current Spark SQL Connector, the Databricks Connector enables users to better handle Tableau queries by reducing latencies and providing performance gains. Through the Tableau SDK, it is possible to achieve an improvement in the performance of Tableau workloads, while simultaneously simplifying the connection process and removing many of the errors that you might see today with the Spark SQL Connector.
Sid Wray and his team have been on a mission to help people see and understand their data. They created the Tableau Connector SDK to empower partners to help in that endeavor. As both companies collaborate closely together on this project, we have proven that this SDK can enable partners like Databricks to build great things.
What's Next?
Databricks is the first partner to build a native connector for Tableau. We are proud of this initiative and the collaboration that is putting the focus on productivity and experience for the end user. Customers will be able to try the Databricks connector natively as part of the Tableau 2019.3 Beta program. Pending positive customer feedback in the Beta, we aim to ship the connector within Tableau sometime during the second half of the year.
Sign up to Tableau's prerelease community, and you will be notified when the Beta is posted.