Applying DevOps to Databricks can be a daunting task. In this talk this will be broken down into bite size chunks. Common DevOps subject areas will be covered, including CI/CD (Continuous Integration/Continuous Deployment), IAC (Infrastructure as Code) and Build Agents.
We will explore how to apply DevOps to Databricks (in Azure), primarily using Azure DevOps tooling. As a lot of Spark/Databricks users are Python users, will will focus on the Databricks Rest API (using Python) to perform our tasks.
Anna is a veteran software & data engineer, with over 15 years of experience. She’s tackled projects from real-time analytics with Scala & Kafka, building out Data Lakes with spark and applying engineering to Data Science. Anna is a senior consultant with Advancing Analytics, helping shape & evolve their data engineering practice. She has a real passion for data and strives to bring the worlds of Software Development and Data Science closer together. Other areas of interest include UX, Agile methodologies, and helping to organize/run local Code Clubs.