Data Analysis with Databricks

This course explores software engineering best practices and DevOps principles, specifically designed for data engineers working with Databricks. Participants will build a strong foundation in key topics such as code quality, version control, documentation, and testing. The course emphasizes DevOps, covering core components, benefits, and the role of continuous integration and delivery (CI/CD) in optimizing data engineering workflows.You will learn how to apply modularity principles in PySpark to create reusable components and structure code efficiently. Hands-on experience includes designing and implementing unit tests for PySpark functions using the pytest framework, followed by integration testing for Databricks data pipelines with DLT and Workflows to ensure reliability.The course also covers essential Git operations within Databricks, including using Databricks Git Folders to integrate continuous integration practices. Finally, you will take a high level look at various deployment methods for Databricks assets, such as REST API, CLI, SDK, and Databricks Asset Bundles (DABs), providing you with the knowledge of techniques to deploy and manage your pipelines.By the end of the course, you will be proficient in software engineering and DevOps best practices, enabling you to build scalable, maintainable, and efficient data engineering solutions.

This comprehensive course provides a practical guide to developing traditional machine learning models on Databricks, emphasizing hands-on demonstrations and workflows using popular ML libraries. This course focuses on executing common tasks efficiently with AutoML and MLflow. Participants will delve into key topics, including regression and classification models, harnessing Databricks' capabilities to track model training, leveraging feature stores for model development, and implementing hyperparameter tuning. Additionally, the course covers AutoML for rapid and low-code model training, ensuring that participants gain practical, real-world skills for streamlined and effective machine learning model development in the Databricks environment.

Languages Available: English | 日本語 | Português BR | 한국어

This course is intended for complete beginners to Python to provide the basics of programmatically interacting with data. The course begins with a basic introduction to programming expressions, variables, and data types. It then progresses into conditional and control statements followed by an introduction to methods and functions. You will learn the basics of data structures, classes, and various string and utility functions. Lastly, you will gain experience using the pandas library for data analysis and visualization as well as the fundamentals of cloud computing. Throughout the course, you will gain hands-on practice through lab exercises with additional resources to deepen your knowledge of programming after the class.