Easy Ingestion to Lakehouse With COPY INTO
A new data management architecture known as the data lakehouse emerged independently across many organizations and use cases to support AI and BI...
Supercharging H3 for Geospatial Analytics
On the heels of the initial release of H3 support in Databricks Runtime (DBR ), we are happy to share ground-breaking performance improvements...
Databricks Power BI Connector Now Supports Native Query
This is a collaborative post from Databricks and Microsoft. We thank Mahesh Prakriya (Director in Intelligence Platform, Microsoft) and Bob Zhang (Sr. Technical...
Building Geospatial Data Products
Geospatial data has been driving innovation for centuries, through use of maps, cartography and more recently through digital content. For example, the oldest...
Accelerating SIEM Migrations With the SPL to PySpark Transpiler
In this blog post, we introduce transpiler , a Databricks Labs open-source project that automates the translation of Splunk Search Processing Language (SPL)...
Scalable Kubernetes Upgrade Using Operators
At Databricks, we run our compute infrastructure on AWS, Azure, and GCP. We orchestrate containerized services using Kubernetes clusters. We develop and manage...
Build Reliable and Cost Effective Streaming Data Pipelines With Delta Live Tables’ Enhanced Autoscaling
This year we announced the general availability of Delta Live Tables (DLT) , the first ETL framework to use a simple, declarative approach...
Spatial Analytics at Any Scale With H3 and Photon
H3's global grid indexing system is driving new patterns for spatial analytics across a variety of geospatial use-cases. Recently, Databricks added built-in support...
Memory Profiling in PySpark
There are many factors in a PySpark program's performance. PySpark supports various profiling tools to expose tight loops of your program and allow...
Introducing Ingestion Time Clustering with Databricks SQL and Databricks Runtime 11.2
Databricks customers are processing over an exabyte of data every day on the Databricks Lakehouse platform using Delta Lake , a significant amount...