Data Engineering with Lakehouse

May 25, 2021 09:00 AM (PT)

Review data architecture concepts during this introduction to the Lakehouse paradigm and an in-depth look at Delta Lake features and functionality. Learn to build end-to-end OLAP data pipelines using Delta Lake.

Emphasis will be placed on using data engineering best practices within Databricks and exploring considerations around:

  • Normalization
  • Change data capture
  •  Slow changing dimensions
  • Regulatory compliance
  •  End user data through aggregate tables and SQL Analytics

Prerequisites: 

  • Intermediate to advanced programming in Python/Scala
  • Beginning experience using the Spark DataFrames API
  • Intermediate to advanced SQL skills
  • Awareness of general data engineering concepts
  • Understanding of the core features and use cases of Delta Lake

 

Role: Data Engineer

Duration: Full day

Labs: Yes