Structured Streaming with Lakehouse

Learn how to quickly build, deploy, and maintain an enterprise Lakehouse platform by combining Delta Lake and Spark Structured Streaming on Databricks. This course highlights the use of cutting-edge Databricks features that simplify scaling streaming analytics.

At the end of this training, you’ll be able to:

  • Describe the basic programming model used by Structured Streaming
  • Use Databricks AutoLoader to write robust ingestion logic for JSON data that is robust to schema changes
  • Propagate streaming inserts, updates, and deletes through a series of Delta tables using Delta Change Stream
  • Join streaming and static data to generate near real-time analytic views

Prerequisites: 

  • 6+ months experience working with the Spark DataFrame API is recommended
  • Intermediate programming experience
  • Conceptual familiarity with the Delta Lake multi-hop architecture (technical experience preferred)

 

Role: Data Engineer

Duration: Full day

Labs: Yes