Intelligent Applications with Apache Spark and Kubernetes from Prototype to Production - Databricks

Intelligent Applications with Apache Spark and Kubernetes from Prototype to Production

Users demand tailored, dynamic, and constantly refined experiences: they expect intelligent applications that will learn from data and improve with longevity and popularity. Application intelligence can take many forms, including anomaly detection, recommendations, natural-language understanding, and speech and image recognition.

All of these capabilities need to be put into production and managed alongside conventional application components. This session will cover a developer’s journey learning Spark and using it to develop a containerized, cloud native application with analysis and visualization components. More specifically, these topics will be covered: exploratory analysis in a Jupyter notebook running against an ephemeral Spark cluster using PySpark for loading and analyzing data from external data sources like PostgreSQL transforming your notebook into a cloud-native application deploying your application in containers on Kubernetes PySpark API functionality that you didn’t know you needed.

Session hashtag: #SAISExp8



« back
About Rebecca Simmonds

Rebecca Simmonds is a senior software engineer at Red Hat. Here she is part of an emerging technology group, which comprises of both data scientists and developers. She completed a PhD at Newcastle University, in which she developed a platform for scalable, geospatial and temporal analysis of the Twitter data. After this she moved to a small startup company as a Java developer creating solutions to improve performance for a CV analyser. She has a keen interest in architecture design and data analysis, which she is furthering at Red Hat with OpenShift and ML research.

About Michael McCune

Michael McCune is a software developer in Red Hat's emerging technology group. He is an active contributor to several radanalytics.io projects, as well as being a core reviewer for the OpenStack API Working Group. Since joining Red Hat three years ago, he has been developing and deploying applications for cloud platforms. Prior to his career at Red Hat, Michael developed Linux based software for embedded global positioning systems.