Company Blog › Product

“Everything should be as simple as it can be, but not simpler” - Albert Einstein Generally, a simple data architecture is preferable to a complex one. Code complexity increases points of failure, requires more compute to run jobs, adds latency, and increases the need for support. As a result, data pipeline performance degrades over time,...

Today, at the Data + AI Summit Europe 2020, we shared some exciting updates on the next generation Data Science Workspace - a collaborative environment for modern data teams - originally unveiled at Spark + AI Summit 2020. The power of data and artificial intelligence is already disrupting many industries, yet we've only scratched the...

Introduction Data has shown that police in the United States kill civilians at a rate far higher than police in other wealthy countries.1 In 2019, law enforcement in the U.S. killed 33.5 civilians per 10 million people, compared to 1.3 in Germany and 0.5 in the U.K.2 This use of deadly force falls disproportionately on...

Today, we announced the new SQL Analytics service to provide Databricks customers with a first-class experience for performing BI and SQL workloads directly on the data lake. This launch brings to life a new experience within Databricks that data analysts and data engineers are going to love. The service provides a dedicated SQL-native workspace, built-in...

In the last two years since its release, MLflow has seen a rapid adoption among enterprises and the data science community. With over 2M downloads, 260 contributors, and 100+ organizations contributing, the momentum seems to grow each year. We have put together a short list of picks–keynotes, tutorials, and sessions–for MLflow: how the community and...

In this first of two blogs, we want to talk about WHY an organization might want to look at a lakehouse architecture (based on Delta Lake) for their data analytics pipelines instead of the standard patterns of lifting and shifting their Enterprise Data Warehouse (EDW) from on-prem or in the cloud. We will follow this...

Databricks and Microsoft Power BI customers will be delighted to know that an enhanced Azure Databricks PowerBI connector is now natively integrated into Power BI Desktop (2.85.681.0 and above ) and Power BI Service! The native connector lets users connect to Databricks from PowerBI Desktop with a couple of clicks, using Azure Active Directory (Azure...

You are a security practitioner, a data scientist or a security data engineer; you’ve seen the Large Scale Threat Detection and Response talk with Databricks . But you're wondering, “how can I try Databricks in my own security operations?” In this blog post, you will learn how to detect a remote access trojan using passive...

Earlier this year, Databricks wrote a blog that outlined how more and more enterprises are adopting the lakehouse pattern. The blog created a massive amount of interest from technology enthusiasts. While lots of people praised it as the next-generation data architecture, some people thought the Lakehouse is the same thing as the data lake. Recently,...

Now generally available an upgraded platform architecture that adds customer-managed IP access lists, customer managed-VPC, Account API, multiple workspaces per account, cluster level policies, IAM credential passthrough, and more. We are excited to announce the general availability of a major upgrade to the Databricks Unified Data Analytics Platform on Amazon Web Services  (AWS) that adds...