Anand is a Global Director of Industry Solutions at Databricks currently leading the Migration and Modernization practice, aiding customers to modernize their on-premises Data and Analytics platforms to a modern Cloud based architecture that includes Databricks.
Anand has over 24 years of Industry and leadership experience in Cloud, Big Data Analytics and multiple industry verticals, bringing new technologies and products to market while being focused on use-cases and business impact from technology.
This tech talk deals with how we leveraged Spark Streaming and Spark Machine Learning models to build & operationalize real-time credit card approvals for a banking major. We plan to cover ML capabilities in Spark and how a typical ML pipeline looks like. We are going to talk about the domain and the use case of how a major credit card provider is using spark to calculate card eligibility in real-time. We're also going to share the challenges faced by the current system and how spark is a good fit to solve these kinds of problems. We will then take a deep dive on the different tools that were used to design the solution and the architecture of the system. Here, we will also be sharing of how a spark based workflow was created to address various aspects like reading from Kafka, parsing, data enrichment, model selection, model scoring, rule execution to conclude the recommended output. Finally, we're also going to talk about the key challenges, learning and recommendations when building such a system and taking it to production. Session hashtag: #Ent6SAIS