Introducing Low-latency Continuous Processing Mode in Structured Streaming in Apache Spark 2.3

by , , and

Structured Streaming in Apache Spark 2.0 decoupled micro-batch processing from its high-level APIs for a couple of reasons. First, it made developer’s experience with the APIs simpler: the APIs did not have to account for micro-batches. Second, it allowed developers to treat a stream as an infinite table to which they could issue queries as...

Lakehouse Architecture from vision to reality - register