New Performance Improvements in Databricks SQL
Originally announced at Data + AI Summit 2020 Europe, Databricks SQL lets you operate a multi-cloud lakehouse architecture that provides data warehousing performance at data lake economics. Our vision is to give data analysts a simple yet delightful tool for obtaining and sharing insights from their lakehouse using a purpose-built SQL UI and world-class support...
Frequently Asked Questions About the Data Lakehouse
Question Index What is a Data Lakehouse? How is a Data Lakehouse different from a Data Warehouse? How is the Data Lakehouse different from a Data Lake? How easy is it for data analysts to use a Data Lakehouse? How do Data Lakehouse systems compare in performance and cost to data warehouses? What data governance...
How We Achieved High-bandwidth Connectivity With BI Tools
Business Intelligence (BI) tools such as Tableau and Microsoft Power BI are notoriously slow at extracting large query results from traditional data warehouses because they typically fetch the data in a single thread through a SQL endpoint that becomes a data transfer bottleneck. Data analysts can connect their BI tools to Databricks SQL endpoints to...
Introducing Apache Spark™ 3.1
We are excited to announce the availability of Apache Spark 3.1 on Databricks as part of Databricks Runtime 8.0. We want to thank the Apache Spark™ community for all their valuable contributions to the Spark 3.1 release. Continuing with the objectives to make Spark faster, easier and smarter, Spark 3.1 extends its scope with the...
Spark + AI Summit Europe is Expanding and Getting a New Name: Data + AI Summit Europe
Back in 2013, we held the first Spark Summit — a gathering of the Apache Spark™ community with leading contributors and production users sharing their wisdom. Since the first event, Spark’s success has accelerated the evolution of data science, data engineering and analytics. As the data community has expanded, we’ve evolved the content and the...
Welcoming Redash to Databricks
This morning at Spark and AI Summit, we announced that Databricks has acquired Redash, the company behind the popular open source project of the same name. With this acquisition, Redash joins Apache Spark, Delta Lake, and MLflow to create a larger and more thriving open source system to give data teams best-in-class tools. I would...
Introducing Apache Spark 3.0
We’re excited to announce that the Apache SparkTM 3.0.0 release is available on Databricks as part of our new Databricks Runtime 7.0. The 3.0.0 release includes over 3,400 patches and is the culmination of tremendous contributions from the open-source community, bringing major advances in Python and SQL capabilities and a focus on ease of use...
Evolving the Databricks brand
Some brands start out as, well, brands. A lot of work goes into the concept and painting the picture before the business is ever launched. Databricks is different. It always has been and always will be an engineering-led company. Databricks’ model for innovation is inspired by the open-source community. This is where our roots run...
What is a Lakehouse?
Over the past few years at Databricks, we've seen a new data management architecture that emerged independently across many customers and use cases: the lakehouse. In this post we describe this new architecture and its advantages over previous approaches. Data warehouses have a long history in decision support and business intelligence applications. Since its inception...
Solving the World’s Toughest Problems with the Growing Open Source Ecosystem and Databricks
We started Databricks in 2013 in a tiny little office in Berkeley with the belief that data has the potential to solve the world’s toughest problems. We entered 2020 as a global organization with over 1000 employees and a customer base spanning from two-person startups to Fortune 10s. In this blog post, let’s take a...
