Building an Implicit Recommendation Engine with Spark - Databricks

Building an Implicit Recommendation Engine with Spark

Download Slides

Many of today’s most engaging – and commercially important – applications provide personalised experiences to users. Collaborative filtering algorithms capture the commonality between users and enable applications to make personalised recommendations quickly and efficiently. The Alternating Least Squares (ALS) algorithm is still deemed the industry standard in collaborative filtering. In this talk Sophie will show you how to implement ALS using Apache Spark to build your own recommendation engine.

Sophie will show that, by splitting the recommendation engine into multiple cooperating services, it is possible to reduce the system’s complexity and produce a robust collaborative filtering platform with support for continuous model training. In this presentation you will learn how to build a recommendation system for the case where recorded data is explicitly given as a rating, as well as for the case where the data is less succinct. You will walk away from this talk with the knowledge and tools needed to implement your own recommendation system using collaborative filtering and microservices.

Session hashtag: #SAISDS12

About Sophie Watson

Sophie is a software engineer at Red Hat, where she works in an emerging technology group. She has a background in Mathematics and has recently completed a PhD in Bayesian statistics, in which she developed algorithms to estimate intractable quantities quickly and accurately. Since joining Red Hat in 2017, Sophie has focused on applying her data science and statistics skills to solving business problems and informing next-generation infrastructure for intelligent application development.