Introducing Easier Change Data Capture in Apache Spark™ Structured Streaming
This blog describes the new change feed and snapshot capabilities in Apache Spark™ Structured Streaming’s State Reader API. The State Reader API enables...
Page 1
A Deep Dive into the Latest Performance Improvements of Stateful Pipelines in Apache Spark Structured Streaming
This post is the second part of our two-part series on the latest performance improvements of stateful pipelines. The first part of this...
Performance Improvements for Stateful Pipelines in Apache Spark Structured Streaming
Introduction Apache Spark™ Structured Streaming is a popular open-source stream processing platform that provides scalability and fault tolerance, built on top of the...
Announcing Support for Enhanced Fan-Out for Kinesis on Databricks
Historically, conventional wisdom held that stream processing is the domain of strictly low latency, operational use cases like sub-second inventory updates on an...