Company Blog

Data + AI Summit Europe will take place 17-19 November and is now open for registration!  Formerly known as the Spark + AI Summit, this free and virtual event is expanded to bring together the world’s leading experts on data analysis, data engineering, data science, machine learning and artificial intelligence (AI) to explore the convergence...

This blog was collaboratively written with Databricks partner Avanade. A special thanks to Dael Williamson, Avanade CTO, for his contributions. Financial Institutions today are still struggling to keep up with the emerging risks and threats facing their business. Managing risk, especially within the banking sector, has increased in complexity over the past several years. First,...

Earlier this year, Databricks wrote a blog that outlined how more and more enterprises are adopting the lakehouse pattern. The blog created a massive amount of interest from technology enthusiasts. While lots of people praised it as the next-generation data architecture, some people thought the Lakehouse is the same thing as the data lake. Recently,...

Back in 2013, we held the first Spark Summit — a gathering of the Apache Spark™ community with leading contributors and production users sharing their wisdom. Since the first event, Spark’s success has accelerated the evolution of data science, data engineering and analytics. As the data community has expanded, we’ve evolved the content and the...

Now generally available an upgraded platform architecture that adds customer-managed IP access lists, customer managed-VPC, Account API, multiple workspaces per account, cluster level policies, IAM credential passthrough, and more. We are excited to announce the general availability of a major upgrade to the Databricks Unified Data Analytics Platform on Amazon Web Services  (AWS) that adds...

In rapidly changing environments, Azure Databricks enables organizations to spot new trends, respond to unexpected challenges and predict new opportunities. Organizations are leveraging machine learning and artificial intelligence (AI) to derive insight and value from their data and to improve the accuracy of forecasts and predictions. Data teams are using Delta Lake to accelerate ETL...

As the leader of the State and Local Government business at Databricks, I get to see what governments all over the U.S. are doing to address the Novel Coronavirus and COVID-19 crisis. I am continually inspired by the work of public servants as they go about their business to save lives and address this crisis....

Databricks is committed to fostering an environment that promotes equality and inclusion. For Women’s Equality Day on August 26, we invited Coco Brown, founder and CEO of Athena Alliance — a digital platform dedicated to revolutionizing leadership — to present a talk about women in leadership and the path to the boardroom. This talk highlighted...

The Databricks team has been working hard to recreate content and enhance the experience as we transition all our events to a virtual experience. And we’ve learned a thing or two about what you want to learn: integrating Databricks with AWS services. So we’re excited to present our Cloud Data Lake Dev Day Workshop in...

If you examine the agenda for any of the Spark Summits in the past five years, you will notice that there is no shortage of talks on how best to architect a data lake in the cloud using Apache Spark™ as the ETL and query engine and Apache Parquet as the preferred file format.  There...