Michael McCune

Software Developer, Red Hat

Michael McCune is a software developer in Red Hat’s emerging technology group. He is an active contributor to several radanalytics.io projects, as well as being a core reviewer for the OpenStack API Working Group. Since joining Red Hat three years ago, he has been developing and deploying applications for cloud platforms. Prior to his career at Red Hat, Michael developed Linux based software for embedded global positioning systems.


Past sessions

Users demand tailored, dynamic, and constantly refined experiences: they expect intelligent applications that will learn from data and improve with longevity and popularity. Application intelligence can take many forms, including anomaly detection, recommendations, natural-language understanding, and speech and image recognition.

All of these capabilities need to be put into production and managed alongside conventional application components. This session will cover a developer's journey learning Spark and using it to develop a containerized, cloud native application with analysis and visualization components. More specifically, these topics will be covered: exploratory analysis in a Jupyter notebook running against an ephemeral Spark cluster using PySpark for loading and analyzing data from external data sources like PostgreSQL transforming your notebook into a cloud-native application deploying your application in containers on Kubernetes PySpark API functionality that you didn't know you needed.

Session hashtag: #SAISExp8

Summit Europe 2017 Fire in the Sky: An Introduction to Monitoring Apache Spark in the Cloud

October 25, 2017 05:00 PM PT

Writing intelligent cloud native applications is hard enough when things go well, but what happens when there are performance and debugging issues that arise during production? Inspecting the logs is a good start, but what if the logs don't show the whole picture? Now you have to go deeper, examining the live performance metrics that are generated by Spark, or even deploying specialized microservices to monitor and act upon that data. Spark provides several built-in sinks for exposing metrics data about the internal state of its executors and drivers, but getting at that information when your cluster is in the cloud can be a time consuming and arduous process. In this presentation, Michael McCune will walk through the options available for gaining access to the metrics data even when a Spark cluster lives in a cloud native containerized environment. Attendees will see demonstrations of techniques that will help them to integrate a full-fledged metrics story into their deployments. Michael will also discuss the pain points and challenges around publishing this data outside of the cloud and explain how to overcome them. In this talk you will learn about: Deploying metrics sinks as microservices, Common configuration options, and Accessing metrics data through a variety of mechanisms.
Session hashtag: #EUdev11