Headaches and Breakthroughs in Building Continuous Applications - Databricks

Headaches and Breakthroughs in Building Continuous Applications

Download Slides

At SpotX, we have built and maintained a portfolio of Spark Streaming applications — all of which process records in the millions per minute. From pure data ingestion, to ETL, to real-time reporting, to live customer-facing products and features, continuous applications are in our DNA. Come along with us as we outline our journey from square one to present in the world of Spark Streaming. We’ll detail what we’ve learned about efficient processing and monitoring, reliability and stability, and long term support of a streaming app. Come learn from our mistakes, and leave with some handy settings and designs you can implement in your own streaming apps.

 

Try Databricks
See More Spark + AI Summit in San Francisco 2019 Videos


« back
About Jack Chapa

Jack Chapa is a Senior Big Data Engineer at SpotX, playing a critical role in the big data pipelines that power the company. He's contributed to reliable 24/7 streaming applications, ETL pipelines, and infrastructure designs. Also, he loves Scala and Spark!

About Landon Robinson

Landon is a Senior Big Data Engineer at SpotX, a video ad serving platform that reaches 100s of millions of people a day. He engineers pipelines that process dozens of terabytes daily in Spark, and enjoys building solutions to difficult-to-scale problems.