Deep Learning for Developers (December 2017)
6 likes834 views
The document introduces deep learning and its applications, particularly using Apache MXNet and Amazon SageMaker. It covers key concepts like neural networks, training processes, and validation, alongside practical demonstrations and resources. Additionally, it discusses the performance and features of MXNet as an open-source deep learning library.
Deep Learning for Developers (December 2017)
- 1. Deep Learning for Developers Julien Simon, AI Evangelist, EMEA @julsimon
- 2. What to expect • An introduction to Deep Learning • An introduction to Apache MXNet • Demos using Jupyter notebooks on Amazon SageMaker • Resources
- 3. • Artificial Intelligence: design software applications which exhibit human-like behavior, e.g. speech, natural language processing, reasoning or intuition • Machine Learning: teach machines to learn without being explicitly programmed • Deep Learning: using neural networks, teach machines to learn from complex data where features cannot be explicitly expressed
- 4. Amazon Echo Amazon Alexa is based on Deep Learning
- 5. Amazon AI is based on Deep Learning
- 6. https://www.oreilly.com/ideas/self-driving-trucks-enter-the-fast-lane-using-deep-learning Last June, tuSimple drove an autonomous truck for 200 miles from Yuma, AZ to San Diego, CA
- 8. Activation functions The neuron ! xi ∗ wi & '() = u ”Multiply and Accumulate” Source: Wikipedia
- 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. x = x11, x12, …. x1I x21, x22, …. x2I … … … xm1, xm2, …. xmI I features m samples y = y1 y2 … ym m labels, N2 outputs 0,0,1,0,0,…,0 1,0,0,0,0,…,0 … 0,0,0,0,1,…,0 One-hot encoding Neural networks
- 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. x = x11, x12, …. x1I x21, x22, …. x2I … … … xm1, xm2, …. xmI I features m samples y = y1 y2 … ym m labels, N2 categories Total number of predictions Accuracy = Number of correct predictions 0,0,1,0,0,…,0 1,0,0,0,0,…,0 … 0,0,0,0,1,…,0 One-hot encoding Neural networks
- 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Neural networks Initially, the network will not predict correctly f(X1) = Y’1 A loss function measures the difference between the real label Y1 and the predicted label Y’1 error = loss(Y1, Y’1) For a batch of samples: ! loss(Yi, Y’i) ,-./0 2345 3() = batch error The purpose of the training process is to minimize error by gradually adjusting weights
- 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Training Training data set Training Trained neural network Batch size Learning rate Number of epochs Hyper parameters Backpropagation
- 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Validation Validation data set Trained neural network Validation accuracy Prediction at the end of each epoch Save the model at the end of each epoch
- 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Early stopping Training accuracy Loss function Accuracy 100% Epochs Validation accuracy Loss Best checkpoint OVERFITTING
- 15. Apache MXNet
- 16. Apache MXNet: Open Source library for Deep Learning Programmable Portable High Performance Near linear scaling across hundreds of GPUs Highly efficient models for mobile and IoT Simple syntax, multiple languages Most Open Best On AWS Optimized for Deep Learning on AWS Accepted into the Apache Incubator MXNet 1.0 released on December 4th
- 19. Text Detection and Recognition https://github.com/Bartzi/stn-ocr
- 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The Apache MXNet API • Storing and accessing data in multi-dimensional arrays àNDArray API • Building models (layers, weights, activation functions) à Symbol API • Serving data during training and validation à Iterators • Training and using models à Module API
- 24. Demos https://github.com/juliensimon/dlnotebooks 1) Synthetic data set 2) Classify images with pre-trained models 3) Learn MNIST with a Multi-Layer Perceptron 4) Learn MNIST with the LeNet CNN 5) Predict handmade MNIST samples 6) Train and host a MNIST model with Amazon SageMaker
- 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Resources https://aws.amazon.com/machine-learning https://aws.amazon.com/blogs/ai https://mxnet.incubator.apache.org https://github.com/apache/incubator-mxnet https://github.com/gluon-api https://github.com/awslabs/sockeye An overview of Amazon SageMaker https://www.youtube.com/watch?v=ym7NEYEx9x4 https://medium.com/@julsimon
- 26. Thank you! Julien Simon, AI Evangelist, EMEA @julsimon