Machine Learning at Scale

In this course, you will gain theoretical and practical knowledge of Apache Spark’s architecture and its application to machine learning workloads within Databricks. You will learn when to use Spark for data preparation, model training, and deployment, while also gaining hands-on experience with Spark ML and pandas APIs on Spark. This course will introduce you to advanced concepts like hyperparameter tuning and scaling Optuna with Spark. This course will use features and concepts introduced in the associate course such as MLflow and Unity Catalog for comprehensive model packaging and governance.

Skill Level

Professional

Duration

Prerequisites

The content was developed for participants with these skills/knowledge/abilities:

A beginner-level understanding of Python.
Basic understanding of DS/ML concepts (e.g. classification and regression models), common model metrics (e.g. F1-score), and Python libraries (e.g. scikit-learn and XGBoost).

Outline

Machine Learning Development with Spark

A Brief Overview of Spark Architecture for Machine Learning
Introduction to Spark ML for Model Development
Model Tracking and Packaging with MLflow and Unity Catalog on Databricks
Model Development with Spark

Model Tuning with Optuna on Spark

Overview of Hyperparameter Tuning
Introduction to Optuna on Spark
Model Tuning with Optuna

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Data Analyst

Data Analytics and Warehousing with Databricks

This content provides a comprehensive introduction to Databricks SQL. It is designed to support individuals seeking the Associate Data Analysis of Databricks SQL certification. Participants will learn about ingesting data, writing queries, producing visualizations and dashboards, and connecting Databricks SQL to additional tools using Partner Connect.

Languages Available: English | 日本語 | Português BR

Advanced Data Engineering with Databricks

In this course, students will build upon their existing knowledge of Apache Spark, Structured Streaming, and Delta Lake to unlock the full potential of the data lakehouse by utilizing the suite of tools provided by Databricks. This course places a heavy emphasis on designs favoring incremental data processing, enabling systems optimized to continuously ingest and analyze ever-growing data. By designing workloads that leverage built-in platform optimizations, data engineers can reduce the burden of code maintenance and on-call emergencies, and quickly adapt production code to new demands with minimal refactoring or downtime.

The topics in this course should be mastered prior to attempting the Databricks Certified Data Engineer Professional exam.

Languages Available: English | 日本語 | Português BR | 한국어

Introduction to Python for Data Science and Data Engineering

This course is intended for complete beginners to Python to provide the basics of programmatically interacting with data. The course begins with a basic introduction to programming expressions, variables, and data types. It then progresses into conditional and control statements followed by an introduction to methods and functions. You will learn the basics of data structures, classes, and various string and utility functions. Lastly, you will gain experience using the pandas library for data analysis and visualization as well as the fundamentals of cloud computing. Throughout the course, you will gain hands-on practice through lab exercises with additional resources to deepen your knowledge of programming after the class.