This 2-day course provides a theoretical and hands-on practical introduction to deep learning with modern neural network tools and architectures. You will build and experiment with many neural network patterns and — more importantly — you will gain a solid understanding and intuition for working with deep learning.
After this class, you will be able to…
- Discuss which problems are well suited to deep learning
- Build, test, evaluate, and modify neural networks for solving common classes of problems
- Understand tradeoffs in data, training cost, deployment, security, and infrastructure for production systems
- Make informed choices about tooling, architecture, and pipelines for deep learning projects
- Background, Machine Learning Review, and Intro
- Set up and review general categories of problems, approaches, and evaluation metrics for machine learning.
- Breakthrough Problems for Deep Learning, Alternate Approaches
- Deep learning is based on 60-year-old ideas: why is it taking off now? What kind of problems can businesses solve with deep learning? Some of the most famous achievements in deep learning are around image and audio processing — but we’ll see how your business use deep learning to solve “traditional” business domain predictive analytics problems.
- Perceptrons and Multilayer-Perceptron Feed-Forward Networks
- Build a “classic” neural network; learn how it trains, and understand why it doesn’t work very well. In several hands-on steps, we’ll “modernize” our network to get a feel for the breakthroughs of the last 40+ years, resulting in a model that performs exceptionally well and fast on a challenging business-style machine learning problem.
- Deep Multilayer Networks & Neural Net Training
- What is a “deep neural network”? What do the additional “hidden layers” mean for our model, its understanding of the world, training, and fitting to a dataset? We’ll also cover key elements of the backpropagation algorithm for training networks and the common high-performing gradient-descent variants.
- Information and Cross Entropy
- Basic terminology and concepts for deep learning which may be unfamiliar: probability distributions, conditional probabilities, information measures and cross entropy
- Convolutional Networks and Image Processing
- Giving the network “hints” about the structure of the data can allow it to form more powerful abstractions faster. We’ll learn exactly how convolutional networks operate and experiment with several of them, while reviewing the state of the art networks that have appeared in recent research, from LeNet to AlexNet, Inception, and ResNet
- Time, Sequences, and Recurrent Networks
- Building on our intuition so far, we’ll model a way for neural networks to learn how to better understand the relations of data over time (or sequences). We’ll train models and look at the state of the art approaches used by companies like Microsoft and Baidu for language processing. We’ll also explore pathological/catastrophic training failures, and learn from them to improve our intuition of the training process.
- Generative Networks
- Explore the theory behind “generative models” — models which can create data records (e.g., images) which are statistically likely or similar to real training data. We’ll learn and build two different kinds of generative networks: variational autoencoders (VAEs) and generative adversarial networks (GANs)
- Agents and Reinforcement Learning
- In reinforcement learning, we train an agent to function in some environment, such as playing a video game or navigating a robot in an environment. In these environments, we don’t have “correct answers” for each move, that we can use to train a network. We’ll look at ways of mapping rewards and punishments from the environment into a mathematical model that a neural network can learn.
- Operational Considerations for Development and Deployment
- What are the options for businesses that want to deploy deep learning in production? Most companies are not focused on university-level research, the way Google and Facebook are — what are the best ways for “regular” companies to leverage deep learning? Once a company is happy with a model, how can it be deployed for real-time business use on the web, on mobile, or on embedded devices? What are security and reliability challenges associated with deep learning? How can deep learning integrate with best-of-breed ETL and Analytics platforms such as Apache Spark?