delivery

track

type

level

industry

Advanced Machine Learning Operations

AI/BI for Data Analysts

In this course, you will learn how to self-serve business insights from your company’s Databricks Data Intelligence Platform using AI/BI. After a tour of the fundamental components of the platform, you’ll learn how to interact with pre-created AI/BI Dashboards to explore your company’s data through existing charts and visualizations. You’ll also learn how to use AI/BI Genie to go beyond dashboards by asking follow-up questions in natural language to self-serve new insights, create visualizations, and share them with your colleagues. Pre-requisites: A working understanding of your organization’s business and key performance indicators. Labs: NoCertification Path: N/A

AI/BI for Self-Service Analytics

We are challenged with delivering TV viewership data in a way that satisfies customers needs and data privacy regulations. After POC’ing two other solutions, we went with Databricks Clean Rooms as it was the only one that could programmatically enforce k-anonymization constraints and differential privacy. While other solutions created bottlenecks through approval processes and rigid templates blocking our customers from writing arbitrary SQL queries, Clean Rooms allows them to explore the data while implicitly filtering out the data that does not meet the approval criteria. Clean Rooms has eliminated the need to replicate data, simplifying data management, improving efficiency and streamlining workflows. The implementation of Clean Rooms has allowed us to monetize our data and improve our internal processes. Clean Rooms is able to handle complex and diverse workloads without compromising on functionality, performance or security. It's built not only for our needs today but also for future use cases we plan to deploy.

Software Engineering Manager

Richard Sharrott

Sr Solutions Engineer

Johanna Hughes

Allowing Arbitrary SQL Queries of TV Viewership Data With Privacy Compliance Through Clean Rooms

As organizations increasingly leverage sensitive data for AI applications, generating high quality synthetic data with mathematical guarantees of privacy has become crucial. This talk explores the use of Gretel Navigator to generate differentially private synthetic data that maintains high fidelity to the source data and high utility on downstream tasks across heterogeneous datasets. Our analysis covers a framework for privacy-preserving synthetic data generation with two use cases: patient events and e-commerce reviews. We reveal nuanced strategies for: calibrating privacy parameters ε and δ for mixed-modal data, leveraging both record-level and user-level differential privacy depending on which entity in the dataset requires protection, maintaining statistical properties and high utility on downstream classification tasks under stringent privacy constraints (e.g., <0.05 difference in AUC when using DP), and quantifying resilience to membership inference and attribute inference attacks.

Lipika Ramaswamy

Beyond the Privacy-Utility Tradeoff: Differential Privacy in Tabular Data Synthesis

In this course, you’ll learn how to define and schedule data pipelines that incrementally ingest and process data through multiple tables on the Data Intelligence Platform, using LakeFlow Pipelines (previously Delta Live Tables) in Spark SQL and Python. We’ll cover topics like how to get started with LakeFlow Pipelines, how LakeFlow Pipelines tracks data dependencies in data pipelines, how to configure and run data pipelines using the LakeFlow Pipelines UI, how to use Python or Spark SQL to define data pipelines that ingest and process data through multiple tables on the Data Intelligence Platform, using Auto Loader and LakeFlow Pipelines, how to use APPLY CHANGES INTO syntax to process Change Data Capture feeds, and how to review event logs and data artifacts created by pipelines and troubleshoot syntax. Pre-requisites: Beginner familiarity with the Databricks Data Intelligence Platform (selecting clusters, navigating the Workspace, executing notebooks), cloud computing concepts (virtual machines, object storage, etc.), production experience working with data warehouses and data lakes, intermediate experience with basic SQL concepts (select, filter, groupby, join, etc), beginner programming experience with Python (syntax, conditions, loops, functions), beginner programming experience with the Spark DataFrame API (Configure DataFrameReader and DataFrameWriter to read and write data, Express query transformations using DataFrame methods and Column expressions, etc.) Labs: YesCertification Path: Databricks Certified Data Engineer Associate

Build Data Pipelines with LakeFlow Pipelines (previously Delta Live Tables)

At Zillow, we have accelerated the volume and quality of our dashboards by leveraging a modern SDLC with version control and CI/CD. In the past three months, we have released 32 production-grade dashboards with 2,000 monthly views across 200 unique users, quadrupling our total viewership while cutting error rates in half over that span. In this session, we will provide an overview of how we utilize Databricks asset bundles and GitLab CI/CD to create performant dashboards that can be confidently used for mission-critical operations. As a concrete example, we'll then explore how Zillow's Data Platform team used this approach to automate our on-call support analysis, leveraging our dashboard development strategy alongside Databricks LLM offerings to create a comprehensive view that provides actionable performance metrics alongside AI-generated insights and action items from the hundreds of requests that make up our support workload.

Senior Software Development Engineer

Caleb Priester

Building Dashboards as a Production-Grade Data Product

In the dynamic world of sports betting, precision and adaptability are key. Sports traders must navigate risk management, limitations of data feeds, and much more to prevent small model miscalculations from causing significant losses. To ensure accurate real-time pricing of hundreds of interdependent markets, traders provide key inputs such as player skill-level adjustments, whilst maintaining precise correlations. Black-box models aren’t enough— constant feedback loops drive informed, accurate decisions. Join DraftKings as we showcase how we expose real-time metrics from our simulation engine, to empower traders with deeper insights into how their inputs shape the model. Using Spark Structured Streaming, Kafka, and Databricks dashboards, we transform raw simulation outputs into actionable data. This transparency into our engines enables fine-grained control over pricing― leading to more accurate odds, a more efficient sportsbook, and an elevated customer experience.

Senior Data science Engineer

Aaron Hope

Senior Data Science Engineer

Ethan Summers

Building Real-Time Sport Model Insights with Spark Structured Streaming

For most companies, building compound AI systems remains aspirational. LLMs are powerful, but imperfect, and their non-deterministic nature makes steering them to high accuracy a challenge. In this session, we’ll demonstrate how to build compound AI systems using SLMs and highly accurate mini-agents that can be integrated into agentic workflows. You'll learn about breakthrough techniques, including: memory RAG, an embedding algorithm that reduces hallucinations using embed-time compute to generate contextual embeddings, improving indexing and retrieval, and memory tuning, a finetuning algorithm that reduces hallucinations using a Mixture of Memory Experts (MoME) to specialize models with proprietary data. We’ll also share real-world examples (text-to-SQL, factual reasoning, function calling, code analysis and more) across various industries. With these building blocks, we’ll demonstrate how to create high accuracy mini-agents that can be composed into larger AI systems.

CEO & Cofounder

Sharon Zhou

Composing High-Accuracy AI Systems With SLMs and Mini-Agents

The convergence of cloud, data and AI is revolutionizing the pharmaceutical industry, creating a powerful ecosystem that drives innovation, efficiency and precision across the entire value chain. At Gilead, teams harness these technologies on a unified platform, accelerating drug discovery, streamlining clinical trials and enabling smarter manufacturing processes. Our approach leverages Databricks and data mesh to manage vast amounts of heterogeneous data, eliminating silos and enhancing collaboration. This integration facilitates personalized medicine, targeted marketing and optimized supply chains. This session will showcase use cases of traditional AI/ML and Generative AI, demonstrating how this cloud-based, AI-powered platform is transforming drug discovery, development, manufacturing and commercialization. Join us to explore the future of pharmaceutical innovation, where the synergy of cloud, data and AI is unlocking new possibilities for a healthier world.

Head of Cloud, Data, & Analytics

Murali Vridhachalam

Convergence of Cloud, Data and AI Ignites Innovation at Gilead

Modern data organizations have moved beyond big data analytics to also incorporate advanced AI/ML data workloads. These workflows often involve multimodal datasets containing documents, images, long-form text, embeddings, URLs and more. Unity Catalog is an ideal solution for organizing and governing this data at scale. When paired with the Daft open source data engine, you can build a truly multimodal, AI-ready data lakehouse. In this session, we’ll explore how Daft integrates with Unity Catalog’s core features (such as volumes and functions) to enable efficient, AI-driven data lakehouses. You will learn how to ingest and process multimodal data (images, text and videos), run AI/ML transformations and feature extractions at scale, and maintain full control and visibility over your data with Unity Catalog’s fine-grained governance.

Co-Founder

Jay Chia

Daft and Unity Catalog: A Multimodal/AI-Native Lakehouse

In this course, you’ll learn how to have efficient data ingestion with LakeFlow Connect and manage that data with Databricks. We’ll cover topics such as ingestion with built-in connectors for popular SaaS applications, databases and file sources, as well as ingestion from cloud object storage, and batch and streaming ingestion. LakeFlow Connect is fully integrated with the Data Intelligence Platform including unified governance, observability, and Delta Lake for the foundation of a data lakehouse architecture. We'll cover the new connector components, setting up the pipeline, validating the source and mapping to the destination for each type of connector. We'll also cover how to ingest data with Batch to Streaming ingestion into Delta tables, using the UI with Auto Loader, automating ETL with Lakeflow Pipelines (previously Delta Live Tables) or using the API. Pre-requisites: Beginner familiarity with the Databricks Data Intelligence Platform (selecting clusters, navigating the Workspace, executing notebooks), cloud computing concepts (virtual machines, object storage, etc.), production experience working with data warehouses and data lakes, intermediate experience with basic SQL concepts (select, filter, groupby, join, etc), beginner programming experience with Python (syntax, conditions, loops, functions), beginner programming experience with the Spark DataFrame API (Configure DataFrameReader and DataFrameWriter to read and write data, Express query transformations using DataFrame methods and Column expressions, etc.) Labs: YesCertification Path: Databricks Certified Data Engineer Associate

Data Ingestion with LakeFlow Connect

In this course, you'll learn concepts and perform labs that showcase workflows using Unity Catalog - Databricks' unified and open governance solution for data and AI. We'll start off with a brief introduction to Unity Catalog, discuss fundamental data governance concepts, and then dive into a variety of topics including using Unity Catalog for data access control, managing external storage and tables, data segregation, and more. Pre-requisites: Beginner familiarity with the Databricks Data Intelligence Platform (selecting clusters, navigating the Workspace, executing notebooks), cloud computing concepts (virtual machines, object storage, etc.), production experience working with data warehouses and data lakes, intermediate experience with basic SQL concepts (select, filter, groupby, join, etc), beginner programming experience with Python (syntax, conditions, loops, functions), beginner programming experience with the Spark DataFrame API (Configure DataFrameReader and DataFrameWriter to read and write data, Express query transformations using DataFrame methods and Column expressions, etc.). Labs: YesCertification Path: Databricks Certified Data Engineer Associate

Data Management and Governance with UC

Data Modeling Strategies

In this course, you’ll learn the fundamentals of preparing data for machine learning using Databricks. We’ll cover topics like exploring, cleaning, and organizing data tailored for traditional machine learning applications. We’ll also cover data visualization, feature engineering, and optimal feature storage strategies. Pre-requisites: Familiarity with Databricks workspace, notebooks, as well as Unity Catalog. An intermediate level knowledge of Python (scikit-learn, Matplotlib), Pandas, and PySpark. As well as with concepts of exploratory data analysis, feature engineering, standardization, and imputation methods). Labs: YesCertification Path: Databricks Certified Machine Learning Associate

Data Preparation for Machine Learning

In this course, you’ll learn how to apply patterns to securely store and delete personal information for data governance and compliance on the Data Intelligence Platform. We’ll cover topics like storing sensitive data appropriately to simplify granting access and processing deletes, processing deletes to ensure compliance with the right to be forgotten, performing data masking, and configuring fine-grained access control to configure appropriate privileges to sensitive data. Pre-requisites: Ability to perform basic code development tasks using the Databricks workspace (create clusters, run code in notebooks, use basic notebook operations, import repos from git, etc), intermediate programming experience with SQL and PySpark (extract data from a variety of file formats and data sources, apply a number of common transformations to clean data, reshape and manipulate complex data using advanced built-in functions), intermediate programming experience with Delta Lake (create tables, perform complete and incremental updates, compact files, restore previous versions etc.). Beginner experience with Delta Live Tables and streaming workloads. Labs: YesCertification Path: Databricks Certified Data Engineer Professional

Databricks Data Privacy

Databricks Devops for Pipelines

In this course, you’ll learn how to optimize workloads and physical layout with Spark and Delta Lake and and analyze the Spark UI to assess performance and debug applications. We’ll cover topics like streaming, liquid clustering, data skipping, caching, photons, and more. Pre-requisites: Ability to perform basic code development tasks using the Databricks workspace (create clusters, run code in notebooks, use basic notebook operations, import repos from git, etc), intermediate programming experience with SQL and PySpark (extract data from a variety of file formats and data sources, apply a number of common transformations to clean data, reshape and manipulate complex data using advanced built-in functions), intermediate programming experience with Delta Lake (create tables, perform complete and incremental updates, compact files, restore previous versions etc.) Labs: YesCertification Path: Databricks Certified Data Engineer Professional

Databricks Performance Optimization

In this course, you’ll learn how to define and schedule data pipelines that incrementally ingest and process data through multiple tables on the Data Intelligence Platform, using LakeFlow Pipelines (previously Delta Live Tables)in Spark SQL and Python. We’ll cover topics like how to get started with LakeFlow Pipelines, how LakeFlow Pipelines tracks data dependencies in data pipelines, how to configure and run data pipelines using the LakeFlow Pipelines UI, how to use Python or Spark SQL to define data pipelines that ingest and process data through multiple tables on the Data Intelligence Platform, using Auto Loader and LakeFlow Pipelines, how to use APPLY CHANGES INTO syntax to process Change Data Capture feeds, and how to review event logs and data artifacts created by pipelines and troubleshoot syntax. Pre-requisites: Beginner familiarity with the Databricks Data Intelligence Platform (selecting clusters, navigating the Workspace, executing notebooks), cloud computing concepts (virtual machines, object storage, etc.), production experience working with data warehouses and data lakes, intermediate experience with basic SQL concepts (select, filter, groupby, join, etc), beginner programming experience with Python (syntax, conditions, loops, functions), beginner programming experience with the Spark DataFrame API (Configure DataFrameReader and DataFrameWriter to read and write data, Express query transformations using DataFrame methods and Column expressions, etc.) Labs: YesCertification Path: Databricks Certified Data Engineer Associate

Databricks Streaming and LakeFlow Pipelines (previously Delta Live Tables)

In this course, you’ll learn how to orchestrate data pipelines with LakeFlow Jobs (previously Databricks Workflows) and schedule dashboard updates to keep analytics up-to-date. We’ll cover topics like getting started with LakeFlow Jobs, how to use Databricks SQL for on-demand queries, and how to configure and schedule dashboards and alerts to reflect updates to production data pipelines. Pre-requisites: Beginner familiarity with the Databricks Data Intelligence Platform (selecting clusters, navigating the Workspace, executing notebooks), cloud computing concepts (virtual machines, object storage, etc.), production experience working with data warehouses and data lakes, intermediate experience with basic SQL concepts (select, filter, groupby, join, etc), beginner programming experience with Python (syntax, conditions, loops, functions), beginner programming experience with the Spark DataFrame API (Configure DataFrameReader and DataFrameWriter to read and write data, Express query transformations using DataFrame methods and Column expressions, etc.) Labs: YesCertification Path: Databricks Certified Data Engineer Associate

Deploy Workloads with Lakeflow Jobs (previously Databricks Workflows)

This session will showcase Bosch’s journey in consolidating supply chain information using the Databricks platform. It will dive into how Databricks not only acts as the central data lakehouse but also integrates seamlessly with transformative components such as dbt and Large Language Models (LLMs). The talk will highlight best practices, architectural considerations, and the value of an interoperable platform in driving actionable insights and operational excellence across complex supply chain processes. Key Topics and Sections<ul>	<li>Introduction & Business Context</li>	<li>Brief Overview of Bosch’s Supply Chain Challenges and the Need for a Consolidated Data Platform.</li>	<li>Strategic Importance of Data-Driven Decision-Making in a Global Supply Chain Environment.</li>	<li>Databricks as the Core Data Platform</li>	<li>Integrating dbt for Transformation</li>	<li>Leveraging LLM Models for Enhanced Insights</li></ul>

Development Lead

Satish Karunakaran

Project Lead Logistics Innovations

Marc-Alexander Frey

End-to-End Interoperable Data Platform: How Bosch Leverages Databricks Supply Chain Consolidation

Mastercard is a global technology company whose role is anchored in trust. It supports 3.4 billion cards and over 143 billion transactions annually. To address customers’ increasing data volume and complex privacy needs, Mastercard has developed a novel service atop Databricks’ Clean Rooms and broader Data Intelligence Platform. This service combines several Databricks components with Mastercard’s IP, providing an evolved method for data-driven insights and value-added services while ensuring a unique standalone turnkey service. The result is a secure environment where multiple parties can collaborate on sensitive data without directly accessing each other’s information. After this session, attendees will understand how Mastercard used its expertise in privacy-enhancing technologies to create collaboration tools powered by Databricks’ Clean Rooms, AI/BI, Apps, Unity Catalog, Workflows and DatabricksIQ — as well as how to take advantage of this new privacy-enhancing service directly.

Lead Solutions Architect

Spencer Cook

VP Data Strategy

John Derrico

Evolving Data Insights With Privacy at Mastercard

As language models have advanced, they have moved beyond code completion and are beginning to tackle software engineering tasks in a more autonomous, agentic way. However, evaluating agentic capabilities is challenging. To address this, we first introduce SWE-bench, a benchmark built from real GitHub issues that has become the standard for assessing AI’s ability to resolve complex software tasks in large codebases. We will discuss the current state of the field, the limitations of today’s models, and how far we still are from truly autonomous AI developers. Next, we will explore the fundamentals of agents based on hands-on demonstrations with SWE-agent, a simple yet powerful agent framework designed for software engineering but adaptable to a variety of domains. By the end of this session, you will have a clear understanding of the current frontier of agentic AI in software engineering, the challenges ahead and how you can experiment with AI agents in your own workflows.

Kilian Lieret

From Code Completion to Autonomous Software Engineering Agents

This course provides participants with information and practical experience in building advanced LLM (Large Language Model) applications using multi-stage reasoning LLM chains and agents. In the initial section, participants will learn how to decompose a problem into its components and select the most suitable model for each step to enhance business use cases. Following this, participants will construct a multi-stage reasoning chain utilizing LangChain and HuggingFace transformers. Finally, participants will be introduced to agents and will design an autonomous agent using generative models on Databricks. Pre-requisites: Solid understanding of natural language processing (NLP) concepts, familiarity with prompt engineering and prompt engineering best practices, experience with the Databricks Data Intelligence Platform, experience with retrieval-augmented generation (RAG) techniques including data preparation, building RAG architectures, and concepts like embeddings, vectors, and vector databases Labs: YesCertification Path: Databricks Certified Generative AI Engineer Associate

Gen AI Application Development

This course introduces learners to deploying, operationalizing, and monitoring generative artificial intelligence (AI) applications. First, learners will develop knowledge and skills in deploying generative AI applications using tools like Model Serving. Next, the course will discuss operationalizing generative AI applications following modern LLMOps best practices and recommended architectures. Finally, learners will be introduced to the idea of monitoring generative AI applications and their components using Lakehouse Monitoring. Pre-requisites: Familiarity with prompt engineering and retrieval-augmented generation (RAG) techniques, including data preparation, embeddings, vectors, and vector databases. A foundational knowledge of Databricks Data Intelligence Platform tools for evaluation and governance (particularly Unity Catalog). Labs: YesCertification Path: Databricks Certified Generative AI Engineer Associate

Gen AI Deployment and Monitoring

This course introduces learners to evaluating and governing GenAI (generative artificial intelligence) systems. First, learners will explore the meaning behind and motivation for building evaluation and governance/security systems. Next, the course will connect evaluation and governance systems to the Databricks Data Intelligence Platform. Third, learners will be introduced to a variety of evaluation techniques for specific components and types of applications. Finally, the course will conclude with an analysis of evaluating entire AI systems with respect to performance and cost. Pre-requisites: Familiarity with prompt engineering, and experience with the Databricks Data Intelligence Platform. Additionally, knowledge of retrieval-augmented generation (RAG) techniques including data preparation, embeddings, vectors, and vector databases Labs: YesCertification Path: Databricks Certified Generative AI Engineer Associate

Gen AI Evaluation and Governance

This course is designed to introduce participants to contextual GenAI (generative artificial intelligence) solutions using the retrieval-augmented generation (RAG) method. Firstly, participants will be introduced to the RAG architecture and the significance of contextual information using Mosaic AI Playground. Next, the course will demonstrate how to prepare data for GenAI solutions and connect this process with building an RAG architecture. Finally, participants will explore concepts related to context embedding, vectors, vector databases, and the utilization of the Mosaic AI Vector Search product. Pre-requisites: Familiarity with embeddings, prompt engineering best practices, and experience with the Databricks Data Intelligence Platform Labs: YesCertification Path: Databricks Certified Generative AI Engineer Associate

Gen AI Solution Development

Databricks Unity Catalog and AI/BI Genie are helping us make our data AI ready. Unity Catalog provides a comprehensive governance framework, enabling our team  to implement secure data organization, access control, and metadata management. Genie spaces help us leverage this foundation to create domain-specific AI environments, allowing business users to interact with data using natural-language queries. We were able to stand up a LLM-powered conversational interface for our business users on our Gold standard leasing data products using AI/BI Genie which also enabled our business to add domain knowledge in instructions, add certified queries, and influence the context to the LLM― all using the same interface, enabling business users with self-service of AI on qualified datasets. UC and AI/BI Genie significantly reduced the complexity of building AI for BI and helped integrate GenAI into our text-to-SQL use cases possible, accessible, and quantifiable.

Senior Data Scientist

Arvind Krishnamoorthy

Sr Business Intelligence Mgr

Brian Schober

Getting Data AI Ready: How Good Governance Practices Led to Accurate NLQ Through Genie Rooms

This talk takes you through the Nationwide Security and Infrastructure data team's journey of migrating from HMS to UC. Discover how HMS federation simplified our transition to UC, allowing for an incremental migration that minimized disruption to data consumers while optimizing our data layout. We’ll share the key technical decisions, challenges faced and lessons learned along the way. The migration process wasn’t without its hurdles, so we’ll walk you through our detailed, step-by-step approach covering planning, execution and validation. We will also showcase the benefits realized, such as improved data governance, more efficient data access and enhanced operational performance. Join us to gain practical insights into executing complex data migrations with a focus on security, flexibility and long-term scalability.

Praveena Edward Panneer Selvam

How HMS Federation Powered Nationwide’s Seamless and Efficient Unity Catalog Migration

A big challenge in LLM development and synthetic data generation is ensuring data quality and diversity. While data incorporating varied perspectives and reasoning traces consistently improves model performance, procuring such data remains impossible for most enterprises. Human-annotated data struggles to scale, while purely LLM-based generation often suffers from distribution clipping and low entropy. In a novel compound AI approach, we combine LLMs with probabilistic graphical models and other tools to generate synthetic personas grounded in real demographic statistics. The approach allows us to address major limitations in bias, licensing and persona skew of existing methods. We release the first open source dataset aligned with real-world distributions and show how enterprises can leverage it with its Gretel Navigator extensions to bring diversity and quality to model training on the Databricks Platform, all while addressing model collapse and data provenance concerns head-on.

Chief Scientist

Yev Meyer

Staff Applied Scientist

Dane Corneil

Improve AI Training With the First Synthetic Personas Dataset Aligned to Real-World Distributions

Retail data is expanding at an unprecedented rate, demanding a scalable, cost-efficient, and near real-time architecture. At Unilever, we transformed our data management approach by leveraging Databricks DLT, achieving approximately $500K in cost savings while accelerating computation speeds by 200–500%. By adopting a streaming-driven architecture, we built a system where data flows continuously across processing layers, enabling real-time updates with minimal latency. DLT’s serverless simplicity replaced complex-dependency management, reducing maintenance overhead, and improving pipeline reliability. DLT Direct Publishing further enhanced data segmentation, concurrency, and governance, ensuring efficient and scalable data operations while simplifying workflows. This transformation empowers Unilever to manage data with greater efficiency, scalability, and reduced costs, creating a future-ready infrastructure that evolves with the needs of our retail partners and customers.

Senior Data Science Manager

Evan Cherney

Innovating Retail Data: Unilever’s Transformation with Databricks DLT

Intro to Data Warehousing with Databricks

Large Language Models (LLMs) excel at understanding messy, real-world data, but integrating them into production systems remains challenging. Prompts can be unruly to write, vary by model and can be difficult to manage in the large context of a pipeline. In this session, we'll demonstrate incorporating LLMs into a geospatial conflation pipeline, using DSPy. We'll discuss how DSPy works under the covers and highlight the benefits it provides pipeline creators and managers.

Drew Breunig

Let the LLM Write the Prompts: An Intro to DSPy in Compound AI Pipelines

The course intends to equip professional-level machine learning practitioners with knowledge and hands-on experience in utilizing Apache Spark™ for machine learning purposes, including model fine-tuning. Additionally, the course covers using the Pandas library for scalable machine learning tasks. The initial section of the course focuses on comprehending the fundamentals of Apache Spark™ along with its machine learning capabilities. Subsequently, the second section delves into fine-tuning models using the hyperopt library. The final segment involves learning the implementation of the Pandas API within Apache Spark™, encompassing guidance on Pandas UDFs (User-Defined Functions) and the Functions API for model inference. Pre-requisites: Familiarity with Databricks workspace and notebooks; knowledge of machine learning model development and deployment with MLflow (e.g. basic understanding of DS/ML concepts, common model metrics and python libraries as well as a basic understanding of scaling workloads with Spark) Labs: YesCertification Path: Databricks Certified Machine Learning Professional

Machine Learning at Scale

Machine Learning at Scale (M AM)

This course is designed to introduce three primary machine learning deployment strategies and illustrate the implementation of each strategy on Databricks. Following an exploration of the fundamentals of model deployment, the course delves into batch inference, offering hands-on demonstrations and labs for utilizing a model in batch inference scenarios, along with considerations for performance optimization. The second part of the course comprehensively covers pipeline deployment, while the final segment focuses on real-time deployment. Participants will engage in hands-on demonstrations and labs, deploying models with Model Serving and utilizing the serving endpoint for real-time inference. Pre-requisites: Familiarity with Databricks workspace and notebooks, familiarity with Delta Lake and Lakehouse, intermediate level knowledge of Python (e.g. common Python libraries for DS/ML like Scikit-Learn, awareness of model deployment strategies) Labs: YesCertification Path: Databricks Certified Machine Learning Associate

Machine Learning Model Deployment

In this course, you’ll learn how to develop traditional machine learning models on Databricks. We’ll cover topics like using popular ML libraries, executing common tasks efficiently with AutoML and MLflow, harnessing Databricks' capabilities to track model training, leveraging feature stores for model development, and implementing hyperparameter tuning. Additionally, the course covers AutoML for rapid and low-code model training, ensuring that participants gain practical, real-world skills for streamlined and effective machine learning model development in the Databricks environment. Pre-requisites: Familiarity with Databricks workspace and notebooks, familiarity with Delta Lake and Lakehouse, intermediate level knowledge of Python (e.g. common Python libraries for DS/ML like Scikit-Learn, fundamental ML algorithms like regression and classification, model evaluation with common metrics) Labs: YesCertification Path: Databricks Certified Machine Learning Associate

Machine Learning Model Development

This course will guide participants through a comprehensive exploration of machine learning model operations, focusing on MLOps and model lifecycle management. The initial segment covers essential MLOps components and best practices, providing participants with a strong foundation for effectively operationalizing machine learning models. In the latter part of the course, we will delve into the basics of the model lifecycle, demonstrating how to navigate it seamlessly using the Model Registry in conjunction with the Unity Catalog for efficient model management. By the course's conclusion, participants will have gained practical insights and a well-rounded understanding of MLOps principles, equipped with the skills needed to navigate the intricate landscape of machine learning model operations. Pre-requisites: Familiarity with Databricks workspace and notebooks, familiarity with Delta Lake and Lakehouse, intermediate level knowledge of Python (e.g. understanding of basic MLOps concepts and practices as well as infrastructure and importance of monitoring MLOps solutions) Labs: YesCertification Path: Databricks Certified Machine Learning Associate

Machine Learning Operations

Each LLM has unique strengths and weaknesses, and there is no one-size-fits-all solution. Companies strive to balance cost reduction with maximizing the value of their use cases by considering various factors such as latency, multi-modality, API costs, user need, and prompt complexity. Model routing helps in optimizing performance and cost along with enhanced scalability and user satisfaction.  Overview of cost-effective models training using AI gateway logs, user feedback, prompt, and model features to design an intelligent model-routing AI agent. Covers different strategies for model routing, deployment in Mosaic AI, re-training, and evaluation through A/B testing and end-to-end Databricks workflows. Additionally, it will delve into the details of training data collection, feature engineering, prompt formatting, custom loss functions, architectural modifications, addressing cold-start problems, query embedding generation and clustering through VectorDB, and RL policy-based exploration.

Senior Machine Learning Engineer

Aditya Gautam

Optimize Cost and User Value Through Model Routing AI Agent

Accurate charge time estimation is key to vehicle performance and user experience. We developed a scalable ML model that enhances real-time charge predictions in vehicle controls. Traditional rule-based methods struggle with dynamic factors like environment, vehicle state, and charging conditions. Our adaptive ML solution improves accuracy by 10%. We use Unity Catalog for data governance, Delta Tables for storage, and Liquid Clustering for data layout. Job schedulers manage data processing, while AutoML accelerates model selection. MLflow streamlines tracking, versioning, and deployment. A dedicated serving endpoint enables A/B testing and real-time insights. As our data ecosystem grew, scalability became critical. Our flexible ML framework was integrated into vehicle control systems within months. With live accuracy tracking and software-driven blending, we support 50,000+ daily charge sessions, improving energy management and user experience.

Mohammed Farag

Sihang Chen

Optimizing EV Charging Experience: Machine Learning for Accurate Charge Time Estimation

PepsiCo, given its scale, has numerous teams leveraging different tools and engines to access data and perform analytics and AI. To streamline governance across this diverse ecosystem, PepsiCo unifies its data and AI assets under an open and enterprise-grade governance framework with Unity Catalog. In this session, we'll explore real-world examples of how PepsiCo extends Unity Catalog’s governance to all its data and AI assets, enabling secure collaboration even for teams outside Databricks. Learn how PepsiCo architects permissions using service principals and service accounts to authenticate with Unity Catalog, building a multi-engine architecture with seamless and open governance. Attendees will gain practical insights into designing a scalable, flexible data platform that unifies governance across all teams while embracing openness and interoperability.

Lead Specialist Solutions Architect

Dipankar Kushari

Data Architect Senior Manager

Sudipta Das

Powering Secure and Scalable Data Governance at PepsiCo With Unity Catalog Open APIs

Join us to explore how Standard Chartered Bank's (SCB) groundbreaking strategy is reshaping the future of the cybersecurity landscape by replacing traditional SIEM with a cutting-edge Databricks solution, achieving remarkable business outcomes:<ul>	<li>68% cost reduction</li>	<li>98% improvement in query resolution time — 380 sec to 8 sec</li>	<li>Significant enhancements in threat detection and response metrics</li>	<li>Substantial increase in ML-driven use cases</li></ul> This talk unveils SCB's journey to a distributed, multi-cloud lakehouse architecture that unlocks unprecedented performance and commercial optimization. Explore why a unified data and AI platform is becoming the cornerstone of next-generation, self-managed SIEM solutions for forward-thinking organizations in this era of AI-powered banking transformation.

Global Head, Cybersecurity Operations

Lavy Stokhamer

Revolutionizing Cybersecurity: SCB's Journey to a Self-Managed SIEM

Assortment and space analytics optimizes product selection and shelf allocation to boost sales, improve inventory management and enhance customer experience. However, challenges like evolving demand, data accuracy and operational alignment hinder success. Older approaches struggled due to siloed tools, slow performance and poor governance. Databricks unified platform resolved these issues, enabling seamless data integration, high-performance analytics and governed sharing. The innovative AI/BI Genie interface empowered self-service analytics, driving non-technical user adoption. This solution helped Walmart cut time to value by 90% and saved $5.6M annually in FTE hours leading to increased productivity. Looking ahead, AI agents will let store managers and merchants execute decisions via conversational interfaces, streamlining operations and enhancing accessibility. This transformation positions retailers to thrive in a competitive, customer-centric market.

Senior Manager, Assortment & Space

Alexandro Arreola-Garcia

Self-Service Assortment and Space Analytics at Walmart Scale

Unity Catalog (UC) enables governance and security for all data and AI assets within an enterprise’s data lake and is necessary to unlock the full potential of Databricks as a true Data Intelligence Platform.  Unfortunately, UC migrations are non-trivial; especially for enterprises that have been using Databricks for more than five years, i.e., 7-Eleven.  System Integrators (SIs) offer accelerators, guides, and services to support UC migrations; however, cloud infrastructure changes, anti-patterns within code, and data sprawl can significantly complicate UC migrations.  There is no “shortcut” to success when planning and executing a complex UC migration.  In this session, we will share how UCX by Databricks Labs, a UC Migration Assistant, allowed 7-Eleven to reorient their UC migration by leveraging assessments and workflows, etc., to assess, characterize, and ultimately plan a tenable approach for their UC migration.

Sr Director of Data, AI/ML, and R&D

Shahmeer Mirza

Delivery Solutions Architect

Prashanth Subrahmanyam

Story of a Unity Catalog (UC) Migration:  Using UCX at 7-Eleven to Reorient a Complex UC Migration

Eli Lilly and Company, a leading bio-pharma company, is revolutionizing manufacturing with next-gen fully digital sites. Lilly and Tredence have partnered to establish a Databricks-powered Global Manufacturing Data Fabric (GMDF), laying the groundwork for transformative data products used by various personas at sites and globally. By integrating data from various manufacturing systems into a unified data model, GMDF has delivered actionable insights across several use cases such as batch release by exception, predictive maintenance, anomaly detection, process optimization and more. Our serverless architecture leverages Databricks Auto Loader for real-time data streaming, PySpark for automation and Unity Catalog for governance, ensuring seamless data processing and optimization. This platform is the foundation for data driven processes, self-service analytics, AI and more. This session will provide details on the data architecture and strategy and share a few use cases delivered.

Senior Cloud Architect - Data & Edge

Saunak Debroy

Head of Delivery - HLS

Abhijay Datta

Transforming Bio-Pharma Manufacturing: Eli Lilly's Data-Driven Journey With Databricks

Mining is fuel-intensive, with haul trucks and mobile equipment consuming vast resources. While factors like terrain and haul distances are fixed, inefficiencies in routing, idling, maintenance, and operator performance present optimization opportunities. With rising pressure to decarbonize and cut costs, AI-driven insights and predictive models offer a game-changing solution. Join us to explore how a mining company leveraged the AVEVA-Databricks partnership and CONNECT to integrate IT and OT data, breaking down silos for real-time haul-truck analytics. By connecting operations data with cloud-based AI models, they achieved a scalable infrastructure that improved efficiency, asset health, and sustainability. Key Takeaways:<ul>	<li>Integrate operations and enterprise data to enable scalable, predictive analytics</li>	<li>Optimize operations with real-time AI insights</li>	<li>Reduce emissions and improve efficiency</li>	<li>Securely share data via Delta Share</li>	<li>Discover how AI transforms mining operations</li></ul>

Bry Dillon

Transforming Mining With AI: Integrating Operational Data for Predictive Insights and Optimization

/* SAVE ON TRAINING */ REGISTER TODAY AND GET 50% OFF TRAINING WITH CODE TRAIN507TMI

entity.node.canonical

Agenda

Pricing

Travel

<nolink>

Event Info

Summit On Demand

Event Terms

Code of Conduct

Privacy Notice

More

Explore sessions

Agenda at a glance

sessions

Register

Sessions

Agenda — Data + AI Summit 2025 | Databricks

title

Data + AI Summit — the premier event for the global data, analytics and AI community. Register now for keynotes, training and 700+ sessions.

description

https://microsites.databricks.com/dais-summit-2025/agenda

Explore the comprehensive agenda for the Data + AI Summit by Databricks. Plan your conference experience with sessions, workshops, and keynotes led by industry experts.

https://microsites.databricks.com/sites/default/files/2025-02/dais25-3-og-1200x628.png

Preview the Sessions

Agenda at a glance

Sessions

Show filters

Preview the
Sessions