Kostas is a Software Engineer and contributor to Apache Spark. Previous to that, he contributed to the extensibility effort on Cloudera Manager. Before Cloudera, Kostas did a 6 year stint at Amazon working across various teams including platform infrastructure. Kostas has a Bachelors of Mathematics in Computer Science from the University of Waterloo.
With Spark being used for more and more production workloads with stringent security requirements, fully locking down Spark applications has become critical. In this talk you will learn the different aspects of securing your spark application. First, we will describe how Spark utilizes kerberos for application authentication. Next, we will discuss the protection of sensitive data through on disk and on the wire encryption. Finally, when integrating with external datastore, authorization becomes important to control who can access what data. We will conclude with discussing open challenges and future work to improve overall security.