Carl Steinbach

Senior Software Engineer, LinkedIn

Carl Steinbach is a Senior Staff Software Engineer at LinkedIn where he leads the Grid Platform Team. In 2014 he launched the Dr. Elephant project with the goal of improving user productivity and the overall efficiency of LinkedIn’s cluster infrastructure.


Dr. Elephant for Monitoring and Tuning Apache Spark Jobs on Hadoop

Dr. Elephant helps improve Spark and Hadoop developer productivity and increase cluster efficiency by making clear recommendations on how to tune workloads and configurations. Originally developed by LinkedIn, Dr. Elephant is now in use at multiple sites. This session will explore how Dr. Elephant works, the data it collects from Spark environments and the customizable heuristics that generate tuning recommendations. Learn how Dr. Elephant can be used to improve production cluster operations, help developers avoid common issues, and green light applications for use on production clusters. Session hashtag: #SFdev18