SESSION

Accelerating Adoption of Datalake for Streaming/ML Use Cases

Accept Cookies to Play Video

OVERVIEW

EXPERIENCEIn Person
TYPEBreakout
TRACKData Lakehouse Architecture
INDUSTRYRetail and CPG - Food
TECHNOLOGIESApache Spark, Delta Lake, Developer Experience
SKILL LEVELIntermediate
DURATION40 min
DOWNLOAD SESSION SLIDES

The objective of this presentation is to showcase how Doordash and Databricks teams came together to accelerate the pace of Databricks adoption for ML and streaming usecases and how this  helped accelerate the adoption of Databricks for workloads that perform more optimally with Delta and Spark compute. However, adoption, in general, can be time-consuming and operationally intensive. Operational efficiency and enablement through automation is a critical factor in making the adoption of any new platform successful.

The focus of this topic will primarily be around tooling and Databricks native features that can help audiences who are on a similar journey. Databricks Accelerators that will be covered in this discussion include Databricks Migration Diagnostics Tool, Delta Lake object builder, Apache Spark™ SQL translator, Delta Lake validation and reconciliation tool, and Databricks Airflow dag migration tool.

SESSION SPEAKERS

Harsha Reddy

/Engineering Manager
Doordash

Aydar Akhmetzyanov

/Software Engineer
Doordash