Machine Learning at Scale

In this course, you will gain theoretical and practical knowledge of Apache Spark’s architecture and its application to machine learning workloads within Databricks. You will learn when to use Spark for data preparation, model training, and deployment, while also gaining hands-on experience with Spark ML and pandas APIs on Spark. This course will introduce you to advanced concepts like hyperparameter tuning and scaling Optuna with Spark. This course will use features and concepts introduced in the associate course, such as MLflow and Unity Catalog, for comprehensive model packaging and governance.

Note: This course is the first in the series of Advanced Machine Learning.

Skill Level

Professional

Duration

Prerequisites

The content was developed for participants with these skills/knowledge/abilities:

A beginner-level understanding of Python.
Basic understanding of DS/ML concepts (e.g. classification and regression models), common model metrics (e.g. F1-score), and Python libraries (e.g. scikit-learn and XGBoost).

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Generative AI Engineer

Generative AI Fundamentals

Welcome to Generative AI Fundamentals. This course provides an introduction to how organizations can understand and utilize generative artificial intelligence (AI) models. First, we'll start off with a quick introduction to generative AI - we'll discuss what it is and pay special attention to large language models, also known as LLMs. Then, we’ll move into how organizations can find success with generative AI - we’ll take a deeper dive into what LLM applications are, discuss how Lakehouse AI can help you succeed, and discuss essential considerations for adopting AI in general. Finally, we'll tackle important aspects to consider when evaluating the potential risks and challenges associated with using/adopting generative AI.

Languages Available: English | 日本語 | Português BR | 한국어

Free

Introductory

Generative AI Engineer

Generative AI Application Development

The course is designed to provide you the practical experience in building advanced LLM applications using multi-stage reasoning LLM chains and agents. First, you’ll learn how to decompose a problem into its components and select the most suitable model for each step to enhance business use cases. Following this, we’ll show you how to construct a multi-stage reasoning chain utilizing LangChain and HuggingFace transformers. Finally, you’ll be introduced to agents and will design an autonomous agent using generative models on Databricks.

Note: This is the second course in the 'Generative AI Engineering with Databricks’ series.

Free

Associate

Machine Learning Practitioner

Machine Learning Model Deployment

This course is designed to introduce three primary machine learning deployment strategies and illustrate the implementation of each strategy on Databricks. Following an exploration of the fundamentals of model deployment, the course delves into batch inference, offering hands-on demonstrations and labs for utilizing a model in batch inference scenarios, along with considerations for performance optimization. The second part of the course comprehensively covers pipeline deployment, while the final segment focuses on real-time deployment.

Note: This is the third course in the 'Machine Learning with Databricks’ series.

Free

Associate