Databricks Streaming and Delta Live Tables

The Databricks Streaming and Delta Live Tables (SDLT) course is designed to prepare students for the Databricks Certified Professional Data Engineer certification exam. The content for this course consists of the Professional-level modules of the Data Engineer Learning Path and can be delivered as instructor-led training (ILT)

Languages Available: English | 日本語 | Português BR | 한국어

Skill Level

Associate

Duration

Prerequisites

Ability to perform basic code development tasks using the Databricks Data Engineering and Data Science workspace (create clusters, run code in notebooks, use basic notebook operations, import repos from git, etc.)
Intermediate programming experience with PySpark

* Extract data from a variety of file formats and data sources

* Apply a number of common transformations to clean data

* Reshape and manipulate complex data using advanced built-in functions

Intermediate programming experience with Delta Lake (create tables, perform complete and incremental updates, compact files, restore previous versions, etc.)
Beginner experience configuring and scheduling data pipelines using the Delta Live Tables (DLT) UI
Beginner experience defining Delta Live Tables pipelines using PySpark

* Ingest and process data using Auto Loader and PySpark syntax

* Process Change Data Capture feeds with APPLY CHANGES INTO syntax

* Review pipeline event logs and results to troubleshoot DLT syntax

Outline

Incremental Processing with Spark Structured Streaming

* Streaming Data Concepts

* Introduction to Apache Spark™ Structured Streaming

* Reading from a Streaming Query

* Aggregations, Time Windows, Watermarks

* Windowed Aggregation with Watermark

Streaming ETL Patterns with DLT

* Data Ingestion Patterns

* Auto Load to Bronze

* Stream from Multiplex Bronze

* Quality Enforcement Patterns

* Quality Enforcement

Upcoming Public Classes

Date	Time	Language	Price
Date	Time	Language	Price	Jan 10	09 AM - 01 PM (America/New_York)	English	$750.00
Feb 03	09 AM - 01 PM (Asia/Kolkata)	English	$750.00
Feb 05	01 PM - 05 PM (America/New_York)	English	$750.00
Feb 07	01 PM - 05 PM (Europe/London)	English	$750.00
Mar 31	09 AM - 01 PM (Asia/Kolkata)	English	$750.00
Apr 02	01 PM - 05 PM (America/New_York)	English	$750.00
Apr 03	01 PM - 05 PM (Europe/London)	English	$750.00

Public Class Registration

If your company has purchased success credits or has a learning subscription, please fill out the Training Request form. Otherwise, you can register below.

Private Class Request

If your company is interested in private training, please submit a request.

Request Private Training

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Machine Learning Practitioner

Machine Learning at Scale

In this course, you will gain theoretical and practical knowledge of Apache Spark’s architecture and its application to machine learning workloads within Databricks. You will learn when to use Spark for data preparation, model training, and deployment, while also gaining hands-on experience with Spark ML and pandas APIs on Spark. This course will introduce you to advanced concepts like hyperparameter tuning and scaling Optuna with Spark. This course will use features and concepts introduced in the associate course such as MLflow and Unity Catalog for comprehensive model packaging and governance.

Data Analytics and Warehousing with Databricks

This content provides a comprehensive introduction to Databricks SQL. It is designed to support individuals seeking the Associate Data Analysis of Databricks SQL certification. Participants will learn about ingesting data, writing queries, producing visualizations and dashboards, and connecting Databricks SQL to additional tools using Partner Connect.

Languages Available: English | 日本語 | Português BR

Advanced Data Engineering with Databricks

In this course, students will build upon their existing knowledge of Apache Spark, Structured Streaming, and Delta Lake to unlock the full potential of the data lakehouse by utilizing the suite of tools provided by Databricks. This course places a heavy emphasis on designs favoring incremental data processing, enabling systems optimized to continuously ingest and analyze ever-growing data. By designing workloads that leverage built-in platform optimizations, data engineers can reduce the burden of code maintenance and on-call emergencies, and quickly adapt production code to new demands with minimal refactoring or downtime.

The topics in this course should be mastered prior to attempting the Databricks Certified Data Engineer Professional exam.

Languages Available: English | 日本語 | Português BR | 한국어