The Pursuit of Happiness: Building a Scalable Pipeline Using Apache Spark and NLP to Measure Customer Service Quality

Download Slides

How do we get better than good enough? Leveraging NLP techniques, we can determine the general sentiment of a sentence, phrase, or a paragraph of text. We can mine the world of social data to get a sense of what is being said. But, how do you get control of the factors that create happiness? How do you become proactive in making end-users happy? Chatbots, human chats, and conversations are the means we are using to express our ideas to each other. NLP is great for helping us process and understand this data but can fall short.

In our session, we will explore how to expand NLP/sentiment analysis to investigate the intense interactions that can occur between humans and humans or humans and robots. We will show how to pinpoint the things that work to improve quality and how to use those data points to measure the effectiveness of chatbots. Learn how we have applied popular NLP frameworks such as NLTK, Stanford CoreNLP and John Snow Labs NLP to financial customer service data. Explore techniques to analyze conversations for actionable insights. Leave with an understanding of how to influence your customers’ happiness.


Try Databricks
See More Spark + AI Summit in San Francisco 2019 Videos

« back
Brooke Wenig
About Brooke Wenig

Brooke Wenig is a Machine Learning Practice Lead at Databricks. She leads a team of data scientists who develop large-scale machine learning pipelines for customers, as well as teach courses on distributed machine learning best practices. She is a co-author of Learning Spark, 2nd Edition, co-instructor of the Distributed Computing with Spark SQL Coursera course, and co-host of the Data Brew podcast. She received an MS in Computer Science from UCLA with a focus on distributed machine learning. She speaks Mandarin Chinese fluently and enjoys cycling. [daisna21-speakers]

About Jameson Woodfin

Jameson is a Data Engineer at FIS, where he works towards modernizing its data pipeline and building its Advanced Data Engine that processes data from millions of devices. His journey into big data and data science began during his studies at UC Santa Barbara where he was a founding member the school’s data science club, Data Science at UCSB. Jameson is most passionate about projects that involve real-time data and machine learning. When Jameson isn’t learning or tinkering, he is likely trying out a new board game or exploring San Francisco’s music scene.