Skip to main content
Page 1

A Deep Dive into the Latest Performance Improvements of Stateful Pipelines in Apache Spark Structured Streaming

February 28, 2024 by Mojgan Mazouchi, Mrityunjay Kumar, Anish Shrigondekar and Karthikeyan Ramasamy in Engineering Blog
This post is the second part of our two-part series on the latest performance improvements of stateful pipelines. The first part of this...

Performance Improvements for Stateful Pipelines in Apache Spark Structured Streaming

February 27, 2024 by Mojgan Mazouchi, Mrityunjay Kumar, Anish Shrigondekar and Karthikeyan Ramasamy in Engineering Blog
Introduction Apache Spark™ Structured Streaming is a popular open-source stream processing platform that provides scalability and fault tolerance, built on top of the...