Supernovas, Black Holes and Streaming Data
Overview This blog post is a follow-up to the session From Supernovas to LLMs at Data + AI Summit 2024, where I demonstrated...
Page 1
Project Lightspeed Update - Advancing Apache Spark Structured Streaming
In this blog post, we will review the advancements in Spark Structured Streaming since we announced Project Lightspeed a year ago, from performance...
Announcing the Public Preview of Predictive I/O for Updates
Previously, we’ve shown you how a new technology called Predictive I/O could improve selective reads by up to 35x for CDW customers without...
Real-Time Insights: The Top Three Reasons Why Customers Love Data Streaming with Databricks
The world operates in real-time The ability to make real-time decisions in today's fast paced world is more critical than ever before. Today's...
Why We Migrated From Apache Airflow to Databricks Workflows at YipitData
This is a collaborative post from Databricks and YipitData. We thank Engineering Manager Hillevi Crognale at YipitData for her contributions. YipitData is the...
Databricks at Current 2022
Current 2022 , organized by Confluent, is the first-ever data streaming industry event – and it's coming up soon! No matter where you...
Low-latency Streaming Data Pipelines with Delta Live Tables and Apache Kafka
Delta Live Tables (DLT) is the first ETL framework that uses a simple declarative approach for creating reliable data pipelines and fully manages...
Introducing Databricks Workflows
Today we are excited to introduce Databricks Workflows , the fully-managed orchestration service that is deeply integrated with the Databricks Lakehouse Platform. Workflows...
How We Built Databricks on Google Kubernetes Engine (GKE)
Our release of Databricks on Google Cloud Platform (GCP) was a major milestone toward a unified data, analytics and AI platform that is...