How to Boost 100x Performance for Real World Application w/ Apache Spark

Download Slides

With the bloom of Apache spark, various big data applications shift to Spark pool to pursue better user experience. However the initial performance doesn’t always meet expectation. In this talk, we will share our experience on working with several top China internet companies to build their next generation big data engine on Spark – including graph analysis, interactive, batch OLAP/BI and real-time analytics. With careful tuning, Spark brought x5-100 speedup versus their original Map Reduce implements. We even accumulated certain experience to further improve the user experience from building real-world Spark application in production environment. We expect this talk will be very useful for people who want to deploy their own spark application and also spark developers who are interested to learn some real case challenges.