Tensors Are All You Need: Faster Inference with Hummingbird

May 26, 2021 03:50 PM (PT)

Download Slides

The ever-increasing interest around deep learning and neural networks has led to a vast increase in processing frameworks like TensorFlow and PyTorch. These libraries are built around the idea of a computational graph that models the dataflow of individual units. Because tensors are their basic computational unit, these frameworks can run efficiently on hardware accelerators (e.g. GPUs).Traditional machine learning (ML) such as linear regressions and decision trees in scikit-learn cannot currently be run on GPUs, missing out on the potential accelerations that deep learning and neural networks enjoy.


In this talk, we’ll show how you can use Hummingbird to achieve 1000x speedup in inferencing on GPUs by converting your traditional ML models to tensor-based models (PyTorch andTVM). https://github.com/microsoft/hummingbird


This talk is for intermediate audiences that use traditional machine learning and want to speedup the time it takes to perform inference with these models. After watching the talk, the audience should be able to use ~5 lines of code to convert their traditional models to tensor-based models to be able to try them out on GPUs.



  • Introduction of what ML inference is (and why it’s different than training) 
  • Motivation: Tensor-based DNN frameworks allow inference on GPU, but “traditional” ML frameworks do not 
  • Why “traditional” ML methods are important 
  • Introduction of what Hummingbirddoes and main benefits 
  • Deep dive on how traditional ML models are built 
  • Brief intro onhow Hummingbird converter works 
  • Example of how Hummingbird can convert a tree model into a tensor-based model 
  • Other models 
  • Demo 
  • Status 
  • Q&A 
In this session watch:
Karla Saur, Senior Research Software Development Engineer, Microsoft
Matteo Interlandi, Scientist, Microsoft


Karla Saur

Karla Saur is a Senior Research Software Development Engineer in the Gray Systems Lab (GSL) atMicrosoft. She finished her PhD in Computer Science at the University of Maryland, College Park in2015. Af...
Read more

Matteo Interlandi

Matteo Interlandi is a Senior Scientist in the Gray Systems Lab (GSL) at Microsoft, working on scalable Machine Learning systems. Before Microsoft, he was a Postdoctoral Scholar in the CS Department a...
Read more