Lab 1: Intro and Setup - Full Stack Deep Learning - Spring 2021
1 like20,704 views
This document outlines the labs for a course on full stack deep learning. Lab 1 introduces the problem of recognizing text from handwritten paragraphs. It involves setting up the computing environment and reviewing code to train a model on MNIST data. Future labs will develop models using CNNs, RNNs, Transformers and other techniques for line and paragraph recognition. The full pipeline will include data management, model deployment, and monitoring in a production environment.
![[246]reasoning, attention and memory toward differentiable reasoning machines](https://cdn.slidesharecdn.com/ss_thumbnails/246reasoningattentionandmemory-towarddifferentiablereasoningmachines-171017054258-thumbnail.jpg?width=560&fit=bounds)
More Related Content
What's hot (20)
Similar to Lab 1: Intro and Setup - Full Stack Deep Learning - Spring 2021 (20)
More from Sergey Karayev (9)
Recently uploaded (20)
Lab 1: Intro and Setup - Full Stack Deep Learning - Spring 2021
- 1. Full Stack Deep Learning - UC Berkeley Spring 2021 - Sergey Karayev, Josh Tobin, Pieter Abbeel Lab 1: Introduction and Setup
- 2. Full Stack Deep Learning - UC Berkeley Spring 2021 Labs • We will incrementally develop a complete deep learning codebase for understanding the content of handwritten paragraphs. • Best framework: PyTorch + PyTorch-Lightning • Best methods: CNNs + RNNs + Transformers • Best experiment management: Weights & Biases • Best way to deploy on the web: Docker + AWS Lambda • Best way to monitor the model in production 2
- 3. Full Stack Deep Learning - UC Berkeley Spring 2021 Outline of the labs 3 Lab 1: Introduction • Lab 1: Introduction. Formulate problem, structure codebase, train an MLP on MNIST data. • Lab 2: CNNs. Introduce EMNIST, generate synthetic handwritten lines, and train CNNs. • Lab 3: RNNs. Using CNN + LSTM with CTC loss for line text recognition. • Lab 4: Transformers. Using Transformers for line text recognition. • Lab 5: Experiment Management. Real handwriting data, Weights & Biases, and hyperparameter sweeps. • Lab 6: Line Detection. Train and evaluate line detection model (or paragraph recognition). • Lab 7: Data Management. Label our own handwriting data and properly store it. • Lab 8: Continuous Integration. Add continuous linting and testing of our code. • Lab 9: Deployment. Run as a REST API locally, then in Docker, then put in production using AWS Lambda. • Lab 10: Monitoring. Set up monitoring that alerts us when the incoming data distribution changes.
- 4. Full Stack Deep Learning - UC Berkeley Spring 2021 • Lab 1: Introduction. Formulate problem, structure codebase, train an MLP on MNIST data. • Lab 2: CNNs. Introduce EMNIST, generate synthetic handwritten lines, and train CNNs. • Lab 3: RNNs. Using CNN + LSTM with CTC loss for line text recognition. • Lab 4: Transformers. Using Transformers for line text recognition. • Lab 5: Experiment Management. Real handwriting data, Weights & Biases, and hyperparameter sweeps. • Lab 6: Line Detection. Train and evaluate line detection model (or paragraph recognition). • Lab 7: Data Management. Label our own handwriting data and properly store it. • Lab 8: Continuous Integration. Add continuous linting and testing of our code. • Lab 9: Deployment. Run as a REST API locally, then in Docker, then put in production using AWS Lambda. • Lab 10: Monitoring. Set up monitoring that alerts us when the incoming data distribution changes. Outline of the labs 4
- 5. Full Stack Deep Learning - UC Berkeley Spring 2021 Lab 1 Goals • Understand the problem and path to solution • Set up our computing environment • Review codebase and train on MNIST 5 Lab 1: Introduction
- 6. Full Stack Deep Learning - UC Berkeley Spring 2021 Lab 1 Goals • Understand the problem and path to solution • Set up our computing environment • Review codebase and train on MNIST 6 Lab 1: Introduction
- 7. Full Stack Deep Learning - UC Berkeley Spring 2021 7 Submit Text Recognizer Pro Full stack deep learning is a weekend program for people who already have some deep learning experience but want to learn about the rest of the “stack” …
- 8. Full Stack Deep Learning - UC Berkeley Spring 2021 8 Model training Web Backend Submit Text Recognizer Pro Full stack deep learning is a weekend program for people who already have some deep learning experience but want to learn about the rest of the “stack” … Compiled Prediction Model Lab 1: Introduction
- 9. Full Stack Deep Learning - UC Berkeley Spring 2021 9 Model training Web Backend Submit Text Recognizer Pro Full stack deep learning is a weekend program for people who already have some deep learning experience but want to learn about the rest of the “stack” … Compiled Prediction Model ‘POST' request {‘image’: b'%xe3L xa6r<x1bxdcx0c x11:xfbxd9xeb …} 9 Lab 1: Introduction
- 10. Full Stack Deep Learning - UC Berkeley Spring 2021 10 Model training Web Backend Submit Text Recognizer Pro Full stack deep learning is a weekend program for people who already have some deep learning experience but want to learn about the rest of the “stack” … Compiled Prediction Model ‘POST' request {‘image’: b'%xe3L xa6r<x1bxdcx0c x11:xfbxd9xeb …} Decode image 10 Lab 1: Introduction
- 11. Full Stack Deep Learning - UC Berkeley Spring 2021 11 Model training Web Backend Submit Text Recognizer Pro Full stack deep learning is a weekend program for people who already have some deep learning experience but want to learn about the rest of the “stack” … Compiled Prediction Model ‘POST' request {‘image’: b'%xe3L xa6r<x1bxdcx0c x11:xfbxd9xeb …} Decode image LineDetector 11 Lab 1: Introduction
- 12. Full Stack Deep Learning - UC Berkeley Spring 2021 12 Model training Web Backend Submit Text Recognizer Pro Full stack deep learning is a weekend program for people who already have some deep learning experience but want to learn about the rest of the “stack” … Compiled Prediction Model ‘POST' request {‘image’: b'%xe3L xa6r<x1bxdcx0c x11:xfbxd9xeb …} Decode image LineDetector LineTextRecognizer Full stack deep learning is a 12 Lab 1: Introduction
- 13. Full Stack Deep Learning - UC Berkeley Spring 2021 13 Model training Web Backend Submit Text Recognizer Pro Full stack deep learning is a weekend program for people who already have some deep learning experience but want to learn about the rest of the “stack” … Compiled Prediction Model ‘POST' request Decode image Encode response {‘pred’: ‘Full Stack Deep Learning’…, ‘conf’: 0.01} LineDetector LineTextRecognizer Full stack deep learning is a 13 {‘image’: b'%xe3L xa6r<x1bxdcx0c x11:xfbxd9xeb …} Lab 1: Introduction
- 14. Full Stack Deep Learning - UC Berkeley Spring 2021 14 Model training Web Backend Submit Text Recognizer Pro Full stack deep learning is a weekend program for people who already have some deep learning experience but want to learn about the rest of the “stack” … Compiled Prediction Model ‘POST' request Decode image Encode response {‘pred’: ‘Full Stack Deep Learning’…, ‘conf’: 0.01} LineDetector LineTextRecognizer Full stack deep learning is a Dataset • Text pages • Line labels • Line images • Decoded text 14 {‘image’: b'%xe3L xa6r<x1bxdcx0c x11:xfbxd9xeb …} Lab 1: Introduction
- 15. Full Stack Deep Learning - UC Berkeley Spring 2021 15 Model training Web Backend Submit Text Recognizer Pro Full stack deep learning is a weekend program for people who already have some deep learning experience but want to learn about the rest of the “stack” … Compiled Prediction Model ‘POST' request Decode image Encode response {‘pred’: ‘Full Stack Deep Learning’…, ‘conf’: 0.01} LineDetector LineTextRecognizer Full stack deep learning is a Dataset • Text pages • Line labels • Line images • Decoded text Training code 15 {‘image’: b'%xe3L xa6r<x1bxdcx0c x11:xfbxd9xeb …} Lab 1: Introduction
- 16. Full Stack Deep Learning - UC Berkeley Spring 2021 16 Model training Web Backend Submit Text Recognizer Pro Full stack deep learning is a weekend program for people who already have some deep learning experience but want to learn about the rest of the “stack” … Compiled Prediction Model ‘POST' request Decode image Encode response {‘pred’: ‘Full Stack Deep Learning’…, ‘conf’: 0.01} LineDetector LineTextRecognizer Full stack deep learning is a Dataset Training code Model weights • Text pages • Line labels • Line images • Decoded text *.h5 16 {‘image’: b'%xe3L xa6r<x1bxdcx0c x11:xfbxd9xeb …} Lab 1: Introduction
- 17. Full Stack Deep Learning - UC Berkeley Spring 2021 Questions? 17
- 18. Full Stack Deep Learning - UC Berkeley Spring 2021 Lab 1 Goals • Understand the problem and path to solution • Set up our computing environment • Review codebase and train on MNIST 18 Lab 1: Introduction
- 19. Full Stack Deep Learning - UC Berkeley Spring 2021 Set up computing environment • Readme • Colab setup video 19
- 20. Full Stack Deep Learning - UC Berkeley Spring 2021 Questions? 20
- 21. Full Stack Deep Learning - UC Berkeley Spring 2021 Lab 1 Goals • Understand the problem and path to solution • Set up our computing environment • Review codebase and train on MNIST 21 Lab 1: Introduction
- 22. Full Stack Deep Learning - UC Berkeley Spring 2021 Review Codebase and Train on MNIST • Readme • Video walkthrough 22
- 23. Full Stack Deep Learning - UC Berkeley Spring 2021 Questions? 23