How Apache Spark and AI Powers UberEats - Databricks

How Apache Spark and AI Powers UberEats

The overall relevance and health of the UberEATS marketplace is critical in order to make and maintain it an everyday product for Uber’s users. In this session, Uber will share a few key design choices it made, such as how Apache Spark and AI are leveraged as an integral part of their production system to improve both the relevance and reliability of their recommender system and services. They will first dive into a few concrete use cases and lessons learned from building AI algorithms with Spark to improve the relevance of UberEats, such as how an multi-objective optimization framework is deployed with the recommender system to find a tradeoff between different business metrics. In addition, maintaining the marketplace’s health is imperative for Uber to provide reliable service. So, in the second part of their talk, they will discuss a dynamic pricing framework that is designed to balance the demand and supply in real-time, in which Spark Streaming allows them to generate real-time features for their geospatial-temporal demand and supply forecasting models and proactively make pricing decisions to optimize market efficiency. Session hashtag: #SFds4

About Chen Jin

Chen has been a software engineer at Uber since 2015, where she has built several active services (such as streamio, eater-surge service). Prior to joining Uber, she worked on BigTable product at Palantir, an interactive large-scale data analytical platform used by several influential clients then. Before that, she spent 4 years in PhD program at Northwestern University and focused her researching on scaling data mining in big data.

About Xian Xing Zhang

Xian Xing has been a data scientist at Uber since 2016, where he is leading the development of recommender system for UberEats. Prior to joining Uber, he worked for LinkedIn was a designer of the recommender systems that power LinkedIn's jobs homepage and member profile page, was the leading contributor to the PhotonML library. Xian Xing also holds a PhD degree and a Master's degree in Computer Engineering and Statistics, respectively, both from Duke University.