Structured Streaming with Lakehouse

May 25, 2021 09:00 AM (PT)

Learn how to quickly build, deploy, and maintain an enterprise Lakehouse platform by combining Delta Lake and Spark Structured Streaming on Databricks. This course highlights the use of cutting-edge Databricks features that simplify scaling streaming analytics.

At the end of this training, you’ll be able to:

  • Describe the basic programming model used by Structured Streaming
  • Use Databricks AutoLoader to write robust ingestion logic for JSON data that is robust to schema changes
  • Propagate streaming inserts, updates, and deletes through a series of Delta tables using Delta Change Stream
  • Join streaming and static data to generate near real-time analytic views

Prerequisites:

  • 6+ months experience working with the Spark DataFrame API is recommended
  • Intermediate programming experience
  • Conceptual familiarity with the Delta Lake multi-hop architecture (technical experience preferred)


Role: Data Engineer

Duration: Full day

Labs: Yes