Data Engineering with Databricks

Important Notice:: This course will be retired on December - 31, 2024. It has been updated and is now offered as 4 modular courses:

1- Data Ingestion with Delta Lake

2- Deploy Workloads with Databricks Workflows

3- Build Data Pipelines with Delta Live Tables

4- Data Management and Governance with Unity Catalog

Databricks recommends that you complete the course by December 31. If you are a Databricks Academy Lab Subscriber, you have the option to enroll in the self-paced versions of these modules available in the Databricks Academy Labs Catalog in Customer Academy.

This course prepares data professionals to leverage the Databricks Lakehouse Platform to productionalize ETL pipelines. Students will use Delta Live Tables to define and schedule pipelines that incrementally process new data from a variety of data sources into the Lakehouse. Students will also orchestrate tasks with Databricks Workflows and promote code with Databricks Repos.

Skill Level

Associate

Duration

12h

Prerequisites

Prerequisities for both versions of the course (Spark SQL and PySpark):

Beginner familiarity with cloud computing concepts (virtual machines, object storage, etc.)
Production experience working with data warehouses and data lakes
Familiarity with basic SQL concepts (select, filter, group by, join, etc)

Additional prerequisites for the Python version of this course (PySpark):

Beginner programming experience with Python (syntax, conditions, loops, functions)
Beginner programming experience with the Spark DataFrame API:
Configure DataFrameReader and DataFrameWriter to read and write data
Express query transformations using DataFrame methods and Column expressions
Navigate the Spark documentation to identify built-in functions for various transformations and data types

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Machine Learning Practitioner

Machine Learning at Scale

In this course, you will gain theoretical and practical knowledge of Apache Spark’s architecture and its application to machine learning workloads within Databricks. You will learn when to use Spark for data preparation, model training, and deployment, while also gaining hands-on experience with Spark ML and pandas APIs on Spark. This course will introduce you to advanced concepts like hyperparameter tuning and scaling Optuna with Spark. This course will use features and concepts introduced in the associate course, such as MLflow and Unity Catalog, for comprehensive model packaging and governance.

Note: This course is the first in the series of Advanced Machine Learning.

Paid & Subscription

Lab

Professional

Platform Administrator

Get Started with Databricks Platform Administration

In this course, you will learn the basics of platform administration on the Databricks Data Intelligence Platform. It offers a comprehensive overview of the Unity Catalog, a vital component for effective data governance within Databricks environments. Divided into five modules, it begins with a detailed introduction to Databricks infrastructure and its data intelligence platform, including an in-depth walkthrough of the Databricks Workspace. You will explore data governance principles within Unity Catalog, covering its key concepts, architecture, and roles. The course further emphasizes managing Unity Catalog metastores and compute resources, including clusters and SQL warehouses. Finally, you'll master data access control by learning about privileges, fine-grained access, and how to govern data objects. By the end, you will be equipped with essential skills to administer the Unity Catalog to implement effective data governance, optimize compute resources, and enforce robust data security strategies. With the purchase of a Databricks Labs subscription, the course also closes out with a comprehensive lab exercise to practice what you’ve learned in a live Databricks Workspace environment.

Get Started with SQL Analytics and BI on Databricks

Please note: This course was formerly titled 'Get Started with Databricks for Data Analysis,' but the course content remains unchanged.

In this course, you will learn basic skills that will allow you to use the Databricks Data Intelligence Platform to perform a simple data analytics workflow and support data warehousing endeavors. You will be given a tour of the workspace and be shown how to work with data objects in Databricks such as catalogs, schemas, tables, compute clusters, notebooks, and dashboards. You will then follow a basic data analytics workflow to perform tasks such as manipulating data using Databricks SQL, leveraging Delta Lake version logs to time travel, creating dashboards within the platform, and creating Genie Spaces for data exploration using natural language prompts. You will also learn how Databricks supports data warehousing needs through the use of Databricks SQL, Delta Live Tables, and Unity Catalog. With the purchase of a Databricks Labs subscription, the course also closes out with a comprehensive lab exercise to practice what you’ve learned in a live Databricks Workspace environment.

Languages Available: English | Português BR | 한국어

Paid & Subscription

Lab

Onboarding