Jack Chapa is a Senior Big Data Engineer at SpotX, playing a critical role in the big data pipelines that power the company. He’s contributed to reliable 24/7 streaming applications, ETL pipelines, and infrastructure designs. Also, he loves Scala and Spark!
April 24, 2019 05:00 PM PT
At SpotX, we have built and maintained a portfolio of Spark Streaming applications -- all of which process records in the millions per minute. From pure data ingestion, to ETL, to real-time reporting, to live customer-facing products and features, continuous applications are in our DNA. Come along with us as we outline our journey from square one to present in the world of Spark Streaming. We'll detail what we've learned about efficient processing and monitoring, reliability and stability, and long term support of a streaming app. Come learn from our mistakes, and leave with some handy settings and designs you can implement in your own streaming apps.