Databricks Launches SQL Analytics to Enable BI on Data Lakes

First in the industry to realize Lakehouse vision, Databricks delivers up to 9x better price/performance than traditional cloud data warehouses

November 12, 2020

Share this post

San Francisco, CA — November 12, 2020: Databricks, the data and AI company, today announced the launch of SQL Analytics, which for the first time enables data analysts to perform workloads previously meant only for a data warehouse on a data lake. This expands the traditional scope of the data lake from data science and machine learning to include all data workloads including business intelligence (BI) and SQL. Now, organizations can empower data teams across data engineering, data science, and data analytics to work on a single source of truth for data. SQL Analytics realizes Databricks’ vision for a lakehouse architecture that combines data warehousing performance with data lake economics, resulting in up to 9x better price/performance than traditional cloud data warehouses. SQL Analytics is available in public preview effective on November 18. More information is available here.

A lakehouse architecture simplifies data and AI for organizations. In the past, data teams had to maintain proprietary data warehouses for BI workloads and data lakes for data science and machine learning workloads, because no single data platform could meet the performance needs of BI and the flexibility needs of data science. Expensive and complicated to maintain, this coexistence of legacy architectures has created data silos that slow innovation and stifle data team productivity. A lakehouse addresses this by running all workloads through a single architecture.

Shell chose Databricks to be one of the foundational components of its Shell.ai platform. “Shell has been undergoing a digital transformation as part of our ambition to deliver more and cleaner energy solutions. As part of this, we have been investing heavily in our data lake architecture. Our ambition has been to enable our data teams to rapidly query our massive datasets in the simplest possible way. The ability to execute rapid queries on petabyte scale datasets using standard BI tools is a game changer for us. Our co-innovation approach with Databricks has allowed us to influence the product roadmap and we are excited to see this come to market.” Dan Jeavons, GM Data Science

“It is no longer a matter of if organizations will move their data to the cloud, but when. A lakehouse architecture built on a data lake is the ideal data architecture for data-driven organizations and this launch gives our customers a far superior option when it comes to their data strategy,” said Ali Ghodsi, CEO and co-founder of Databricks. “We’ve worked with thousands of customers to understand where they want to take their data strategy, and the answer is overwhelmingly in favor of data lakes. The fact is that they have massive amounts of data in their data lakes and with SQL Analytics, they now can actually query that data by connecting directly to their BI tools like Tableau.”

SQL Analytics is built on Delta Lake, an open format data engine that adds reliability, quality, and security, to a customer’s existing data lake. Customers are able to avoid storing multiple copies of data, as well as locking data up in proprietary formats. To deliver BI-performance on a data lake, SQL Analytics makes use of two unique innovations. First, it provides easy-to-use auto-scaling endpoints that keep query latency consistently low under high user load. Second, it uses Delta Engine, Databricks’ unique polymorphic query execution engine, to complete queries quickly against both large and small data sets. With native connectors for all major BI tools, including Tableau and Microsoft Power BI, customers can easily integrate SQL Analytics into their existing BI workflows to conduct analytics on much fresher, more complete data than ever before. SQL Analytics also provides a SQL-native query and visualization interface to allow analysts, data scientists, and developers without access to traditional BI tools to build dashboards and reports that can be easily shared within their organization.

“Now more than ever, organizations need a data strategy that enables speed and agility to be adaptable,” said Francois Ajenstat, Chief Product Officer at Tableau. “As organizations are rapidly moving their data to the cloud, we’re seeing growing interest in doing analytics on the data lake. The introduction of SQL Analytics delivers an entirely new experience for customers to tap into insights from massive volumes of data with the performance, reliability and scale they need. We're proud to partner with Databricks to bring that opportunity to life.”

The lakehouse architecture is widely supported by Databricks partners including:

BI Partners: Tableau, Power BI, Qlik, Looker, Thoughtspot
Ingest Partners: Fivetran, Fishtown Analytics, Matillion, Talend
Catalog Partners: Collibra, Alation
Consulting Partners: Slalom, Thorogood, Advancing Analytics

“Databricks SQL Analytics is a critical step in the most important trend in the modern data stack: the unification of traditional SQL analytics with machine-learning and data science,” said George Fraser, CEO at Fivetran. “Companies make huge investments in centralizing and curating data, and they should be able to make those investments once and then implement multiple analytical paradigms in a unified environment. The Lakehouse architecture supports that."

This announcement comes on the heels of impressive momentum Databricks has achieved over the past year. The company achieved a $350M+ revenue run rate as of Q3 2020, up from $200M in Q3 2019, and is now among the fastest-growing enterprise software cloud companies on record. It has achieved global growth, doubling its headcount in the UK, Netherlands, Germany, and Sweden, and growing 5x in Australia and India over the last year. Databricks has 1,500 employees worldwide, and thousands of data teams leverage its Unified Data Analytics Platform across all industries and verticals.

Press Contact:
Keyana Corliss
Head of Global Communications
[email protected]

QUOTE SHEET

Atlassian: “At Atlassian, we need to ensure teams can collaborate well across functions to achieve constantly evolving goals,” said Rohan Dhupelia, Data Platform Senior Manager, Atlassian. “A simplified lakehouse architecture would empower us to ingest high volumes of user data and run the analytics necessary to better predict customer needs and improve the experience of our customers. A single, easy-to-use cloud analytics platform allows us to rapidly improve and build new collaboration tools based on actionable insights.”

Shell: “Shell has been undergoing a digital transformation as part of our ambition to deliver more and cleaner energy solutions. As part of this, we have been investing heavily in our data lake architecture. Our ambition has been to enable our data teams to rapidly query our massive datasets in the simplest possible way. The ability to execute rapid queries on petabyte scale datasets using standard BI tools is a game changer for us. Our co-innovation approach with Databricks has allowed us to influence the product roadmap and we are excited to see this come to market.” Dan Jeavons, GM Data Science

Wejo: "At Wejo, we're collecting data from more than 50 million accessible connected cars to build a better driving experience. Databricks and a robust lakehouse architecture will allow us to provide automated analytics to our customers, empowering them to glean insights on nearly 5 trillion data points per month, all in a streaming environment from car to marketplace in seconds." (Daniel Tibble, Head of Data)

Yipitdata: "As a company focused on providing data-driven research to our customers, the massive amount of data in our data lake is our lifeblood. By leveraging Databricks and Delta Lake, we have already been able to democratize data at scale, while lowering the cost of running production workloads by 60%, saving us millions of dollars. We're excited to build on this momentum by leveraging the Databricks lakehouse architecture that will further empower everyone across our organization - from research analysts to data scientists - to interchangeably use the same data, helping us to provide innovative insights to our customers faster than ever before" (Andrew Gross, Director of Engineering)

PARTNER BLOGS