Deploying Accelerators At Datacenter Scale Using Spark - Databricks

Deploying Accelerators At Datacenter Scale Using Spark

Download Slides

Accelerators, such as GPUs and FPGAs, have been gaining popularity in commodity clusters due to their high computing and energy efficiency. However, deploying accelerators at a large scale still requires significant effort when using existing approaches. In this talk, we present Blaze, an accelerator-aware runtime system that enables rapid warehouse-scale accelerator deployment for the Hadoop/Spark ecosystem. In particular, Blaze provides accelerator management for data-intensive scalable computing (DISC) systems in a cluster with heterogeneous accelerator platforms, including GPUs and FPGAs. Blaze also provides APIs for system architects to design cluster-wide ready-to-use accelerators, and APIs for DISC system developers to use accelerators. Blaze demonstrates improved DISC system throughput and accelerator utilization on a cluster with commodity FPGA and GPU cards.

About Di Wu

Di Wu is a 5th-year PhD student at UCLA, working with professor Jason Cong. Previous to UCLA, he got his bachelor degree of Electrical Engineering from Tsinghua University, China. He did an internship at Intel Lab from Jun. 2014 to Sep. 2014, working on Spark integration with the Intel HARP FPGA system. His research interests include large-scale deployment of hardware acceleration systems including GPU and FPGA, and domain-specific acceleration for large-scale machine learning and cognitive computations.

About Muhuan Huang

Muhuan Huang is a 6th-year PhD student at UCLA, working with professor Jason Cong. Previous to UCLA, she got BS degree in Electrical Engineering from Xi'an Jiaotong University, China in 2010. At UCLA, Muhuan has been working extensively on various projects that relate to resource and data management in accelerator-rich architectures both at cluster-level and chip-level. Her research interests include cloud computing platforms and infrastructure, and computation acceleration for data processing domains. Muhuan did a summer internship at Hewlett-Packard in 2013 and a summer internship at Google in 2015.