Databricks Streaming and Delta Live Tables

This course provides a comprehensive understanding of Spark Structured Streaming and Delta Lake, including computation models, configuration for streaming read, and maintaining data quality in a streaming environment.

Note: This course is part of the 'Advanced Data Engineering with Databricks' course series.

Skill Level

Professional

Duration

Prerequisites

The content was developed for participants with these skills/knowledge/abilities:

- Ability to perform basic code development tasks using the Databricks Data Engineering and Data Science workspace (create clusters, run code in notebooks, use basic notebook operations, import repos from git, etc.)
- Intermediate programming experience with PySpark
  - Extract data from a variety of file formats and data sources
  - Apply a number of common transformations to clean data
  - Reshape and manipulate complex data using advanced built-in functions
- Intermediate programming experience with Delta Lake (create tables, perform complete and incremental updates, compact files, restore previous versions, etc.)
- Beginner experience configuring and scheduling data pipelines using the Delta Live Tables (DLT) UI
- Beginner experience defining Delta Live Tables pipelines using PySpark

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Machine Learning Practitioner

Machine Learning at Scale

In this course, you will gain theoretical and practical knowledge of Apache Spark’s architecture and its application to machine learning workloads within Databricks. You will learn when to use Spark for data preparation, model training, and deployment, while also gaining hands-on experience with Spark ML and pandas APIs on Spark. This course will introduce you to advanced concepts like hyperparameter tuning and scaling Optuna with Spark. This course will use features and concepts introduced in the associate course, such as MLflow and Unity Catalog, for comprehensive model packaging and governance.

Note: This course is the first in the series of Advanced Machine Learning.

Paid & Subscription

Lab

Professional

Platform Administrator

Get Started with Databricks Platform Administration

In this course, you will learn the basics of platform administration on the Databricks Data Intelligence Platform. It offers a comprehensive overview of the Unity Catalog, a vital component for effective data governance within Databricks environments. Divided into five modules, it begins with a detailed introduction to Databricks infrastructure and its data intelligence platform, including an in-depth walkthrough of the Databricks Workspace. You will explore data governance principles within Unity Catalog, covering its key concepts, architecture, and roles. The course further emphasizes managing Unity Catalog metastores and compute resources, including clusters and SQL warehouses. Finally, you'll master data access control by learning about privileges, fine-grained access, and how to govern data objects. By the end, you will be equipped with essential skills to administer the Unity Catalog to implement effective data governance, optimize compute resources, and enforce robust data security strategies. With the purchase of a Databricks Labs subscription, the course also closes out with a comprehensive lab exercise to practice what you’ve learned in a live Databricks Workspace environment.

Get Started with SQL Analytics and BI on Databricks

Please note: This course was formerly titled 'Get Started with Databricks for Data Analysis,' but the course content remains unchanged.

In this course, you will learn basic skills that will allow you to use the Databricks Data Intelligence Platform to perform a simple data analytics workflow and support data warehousing endeavors. You will be given a tour of the workspace and be shown how to work with data objects in Databricks such as catalogs, schemas, tables, compute clusters, notebooks, and dashboards. You will then follow a basic data analytics workflow to perform tasks such as manipulating data using Databricks SQL, leveraging Delta Lake version logs to time travel, creating dashboards within the platform, and creating Genie Spaces for data exploration using natural language prompts. You will also learn how Databricks supports data warehousing needs through the use of Databricks SQL, Delta Live Tables, and Unity Catalog. With the purchase of a Databricks Labs subscription, the course also closes out with a comprehensive lab exercise to practice what you’ve learned in a live Databricks Workspace environment.

Languages Available: English | Português BR | 한국어

Paid & Subscription

Lab

Onboarding