Crossing The Ocean for Spark Summit EU

This is a guest post from Movile. Eiti Kimura and Flavio Clésio share their highlights and what they're looking forward to the most at Spark Summit EU in Dublin, Ireland.

About the Authors:
Eiti Kimura has over 15 years of experience working with software development. He holds a Master’s Degree in Electrical Engineering. He has vast experience with back-end systems for carrier billing services. He currently works as IT Coordinator at Movile Brazil.

Flavio Clesio is a specialist in machine learning and revenue assurance at Movile, where he helps build core intelligent applications to maximize revenue opportunities and automation in decision making. He holds a Master’s degree in computational intelligence applied in financial markets. Currently Working as Head of Machine Learning at Movile Brazil.

Spark Summit is nearly here and we are excited to cross the ocean and head to Dublin in Ireland. It is a new world for us, tropical country guys - new weather (not so tropical), city, conference, people, and culture. It promises to be a very rich experience.

Networking with the community and attending  presentations to learn new information and ideas is always the best part of going to a conference.

USE CODE "DATABRICKS" TO SAVE 15%

Here are some of the talks that we are looking forward to attending:

At Movile, as we're taking major steps to migrate our ETL processes from traditional RDBMS to Apache Spark, we’ll be learning from Brandon Carl at Facebook who will provide a presentation that grabbed our attention called “Lessons learned developing and managing massive (300 TB+) Apache Spark pipelines in production”.

We're living an era where hardware is cheaper than ever before, and the right question is not about whether we can get hardware to deliver good machine learning models, but what is the best model to put in production to deliver the business. For that reason, we'll attend the session by Marcin Kulka and Michał Kaczmarczyk from 9LivesData called “No More Cumbersomeness: Automatic Predictive Modeling on Apache Spark”.

In our opinion Ramya Raghavendra from IBM’s talk called “Improving Traffic Prediction Using Weather Data” will teach us how to solve a really hard problem using Apache Spark as we pivot from the engineering and Machine Learning perspective.

At Movile we're dealing with several projects using Apache Spark and Redis to solve a myriad of problems, and we believe that Dvir Volk from Redis Labs will provide us with some interesting knowledge about how to put together the best setup in their talk called “Getting Ready to use Redis with Apache Spark”.

For everyone that is interested in streaming (and who isn’t these days?) the presentation by Tathagata Das from Databricks called “Easy, Scalable, Fault-Tolerant stream processing with structured streaming in Apache Spark” is  an excellent choice. He will show us how to read Kafka and parse JSON payloads in less than 10 lines.

Have you ever faced a situation with poor performance? We have! And that is why this presentation caught our attention. Luca Canali will talk about the methods and tools for troubleshooting Spark workloads at scale: "Apache Spark Performance Troubleshooting At Scale, Challenges, Tools, And Methodologies".

We hope you liked our highlights. And last but not least, Flavio and I will be talking about how we have saved more than $3 million US  in revenue leakages with an intelligent monitoring solution with machine learning using Apache Spark MLlib: "Preventing Revenue Leakage And Monitoring Distributed Systems With Machine Learning".

We hope to see you in Dublin!