Kostas Andrikopoulos is a big data architect at Kaizen Gaming, where he leads the development of the high performance in house real time data platform that processes hundreds of millions events daily. With more than 15 years experience in software development and distributed systems he holds a Master of Digital Networking and Telecommunications from the University Piraeus. Kostas is passionate about Functional Programming, Distributed Systems, Data Lakes, Streaming, Apache Spark and Machine Learning.
November 18, 2020 04:00 PM PT
In the online gaming industry we receive a vast amount of transactions that need to be handled in real time. Our customers get to choose from hundreds or even thousand options, and providing a seamless experience is crucial in our industry. Recommendation systems can be the answer in such cases but require handling loads of data and need to utilize large amounts of processing power. Towards this goal, in the last two years we have taken down the road of machine learning and AI in order to transform our customer's daily experience and upgrade our internal services.
In this long journey we have used the Databricks on Azure Cloud to distribute our workloads and get the processing power flexibility that is needed along with the stack that empowered us to move forward. By using MLflow we are able to track experiments and model deployment, by using Spark Streaming and Kafka we moved from batch processing to Streaming and finally by using Delta Lake we were able to bring reliability in our Data Lake and assure data quality. In our talk we will share our transformation steps, the significant challenges we faced and insights gained from this process.
Speakers: Stefanos Doltsinis and Kostas Andrikopoulos