Getting Started with Apache Spark DataFrames


This hands-on self-paced training course targets Analysts and Data Scientists getting started using Databricks to analyze big data with Apache Spark™ DataFrames. The course ends with a capstone project demonstrating Exploratory Data Analysis with Spark DataFrames on Databricks.


3-6 hours, 75% hands-on

Format: Self-paced

The course is a series of six self-paced lessons plus a final capstone project performing Exploratory Data Analysis using Spark DataFrames on Databricks. Each lesson includes hands-on exercises.

Supported platforms include Databricks Community Edition, Azure Databricks and Amazon.

Learning Objectives

During this course learners

  • Use the interactive Databricks notebook environment.
  • Examine external data sets.
  • Query existing data sets using Spark DataFrames.
  • Visualize query results and data using the built-in Databricks visualization features.
  • Perform exploratory data analysis using Spark DataFrames.
  • Learn to translate SQL statements to DataFrame syntax.


  • Getting Started and Accessing the Course
  • Querying Files with DataFrames
  • Aggregations and JOINs
  • Uploading and Accessing Data
  • Querying JSON & Hierarchical Data with DataFrames
  • Querying Data Lakes with DataFrames
  • Capstone Project: Exploratory Data Analysis

Target Audience

  • Primary Audience: Data Scientists and Engineers
  • Secondary Audience: Data Analysts


  • Programming in Scala or Python required.

Lab Requirements

  • Chrome or Firefox browser. Internet Explorer, Edge, and Safari are not supported.