Daniel Hen

Data Scientist, Fyber

Three years of experience as a Data Scientist, previously worked in Software Engineering.

Past sessions

In this talk, we will present how we used Spark, Databricks, Airflow and MLflow to process big data, and build a pipeline of both ML(XGBoost) and statistical models that maximizes our revenues in one of our core products, called the "Offer Wall". The "Offer wall" is a mobile phone product that is integrated with existing apps, suggesting users to perform tasks in exchange for in-app currency. The problem gets even more interesting when considering the fact that some of the tasks users do take 15 minutes and some may take up to 2 to weeks, forcing us to make revenue determining decisions in an uncertain space all of the time. The solution we developed utilizes Databricks and Spark's strengths and diversity in machine learning, big data, MLflow and Airflow integrations, allowing us to deliver a production-grade solution with short development time between experiments.

Speakers: Michael Winer and Daniel Hen