Sina Sheikholeslami

Doctoral Student, KTH Royal Institute of Technology

Sina Sheikholeslami is a Doctoral student in the Distributed Computing Group of KTH Royal Institute of Technology, and his main research interests lie in the intersection of distributed systems, machine learning, and data-intensive computing.

Past sessions

Summit Europe 2020 Parallel Ablation Studies for Machine Learning with Maggy on Apache Spark

November 18, 2020 04:00 PM PT

Ablation studies have become best practice in machine learning research as they provide insights into the relative contribution of the different architectural and regularization components to the performance of models. An ablation study consists of several trials, where one trial could be, e.g., removing the last convolutional layer of a CNN model, retraining the model, and observing the resulting performance. However, as machine learning architectures become ever deeper and data sizes keep growing, there is an explosion in the number of different architecture combinations that need to be evaluated to understand their relative performance. In this talk, we introduce a new Spark-based framework for design and parallel execution of ablation studies on machine learning frameworks.

Our framework provides a declarative way to define ablation experiments for deep learning model architectures and training datasets, in a way that eliminates the need for maintaining redundant copies of code for an ablation study. Furthermore, our framework enables parallel execution of ablation trials without requiring the developers to modify their code, which leads to shorter study times and better resource utilization. To this end, we build our ablation study framework on Maggy, an open-source framework for asynchronous parallel execution of trials on Apache Spark. In this talk, we introduce a new framework for design and parallel execution of ablation studies on machine learning frameworks. Our framework provides researchers and practitioners with a declarative way to define ablation experiments for deep learning model architectures and training datasets, in a way that eliminates the need for maintaining redundant copies of code for an ablation study. Furthermore, our framework enables parallel execution of ablation trials without requiring the developers to modify their code, which leads to shorter study times and better resource utilization.

Speakers: Sina Sheikholeslami and Jim Dowling