Optimize your Machine Learning workloads (April 2019)

Editor's Notes

• #2: This talk is for builders I want to build a new app or website and I want it to work on every platform I want easily leverage AWS from my existing web or mobile apps and I don’t want to rewrite everything I want to learn about cool new development tools like React, GraphQL, CLIs, and serverless technologies I want to focus less on ops and configuration and more on my product If you recognize yourself in one of the previous sentences, you’re in the right room. TODO for the presenter : adjust the banner to the conference you will speak to.
• #3: Once your model is trained and tuned, SageMaker makes it easy to deploy in production so you can start generating predictions on new data (a process called inference). Amazon SageMaker deploys your model on an auto-scaling cluster of Amazon EC2 instances that are spread across multiple availability zones to deliver both high performance and high availability. It also includes built-in A/B testing capabilities to help you test your model and experiment with different versions to achieve the best results.   For maximum versatility, we designed Amazon SageMaker in three modules – Build, Train, and Deploy – that can be used together or independently as part of any existing ML workflow you might already have in place.
• #5: 1/ The new Amazon EC2 P3dn instance 2/ With four-times the networking bandwidth and twice the GPU memory of the largest P3 instance, P3dn is ideal for large scale distributed training. No one else has anything close. 3/ P3dn.24xlarge instances offer 96vCPUs of Intel Skylake processors to reduce preprocessing time of data required for machine learning training. 3/ The enhanced networking of the P3n instance allows GPUs to be used more efficiently in multi-node configurations so training jobs complete faster. 4/ Finally, the extra GPU memory allows developers to easily handle more advanced machine learning models such as holding and processing multiple batches of 4k images for image classification and object detection systems
• #9: 1/ Let’s take a look at a common computer vision model for image classification, a deep neural network called ResNet-50 <rez net fifty>, trained on hundreds of thousands of images. 2/ The fastest time to train this model, by a team in Mountain View, was 30 minutes, using a specially built training algorithm which was optimized just for this single neural network, and for specialized hardware which is only available in beta (and not available to most developers). These improvements are locked away from most models, and out of reach of the vast majority of developers. 4/ With the improvements we made in TF, we reduced training time by over 50% to just 14 minutes. This is the fastest time for training ResNet using TensorFlow, anywhere. 5/ But even more importantly, our optimizations can be applied to multiple different models - including convolutional neural networks (images), and recurrent neural networks (language, recommendation). 6/ It also runs on P3 instances, which are globally available to all developers in 14 regions. 7/ Available in SageMaker and the Deep Learning AMI
• #17: These performance and accuracy trade offs are felt most acutely at the edge. 1/ IoT applications are usually running on devices, out there in the real world. This means that the accuracy of models can be felt quickly, and immediately. Consumer IoT applications have a high expectation of accuracy - such as Alexa detecting the wake word reliably - the accuracy of that model really matters to the overall experience. In industrial IoT, devices are often responsible for monitoring and maintaining and core manufacturing processes, or safety. The accuracy of a model here is critical. 2/ Applications running on IoT devices at the edge are commonly very sensitive to latency; it’s part of the reason why customers are running the workload there in the first place, because they can’t afford the round trip to the cloud and back. So any increase in that latency can have a meaningful impact on the success of the device itself. 3/ IoT applications are often incredibly resource constrained, in a way which is much more acute than in the cloud. The devices are smaller, and have less memory and processing power, which is a real problem for machine learning models. 4/ In many cases, IoT applications need to run on very diverse hardware platforms, with a dizzying myriad of processor architectures. To get any sort of performance, developers have to optimize 5/ Finally, one of the key benefits of machine learning can get lost; the ability to continually improve the model. IoT applications are great data generators, and once that data is “ground-truthed”, it can be used to build more sophisticated models. However, if the effort to optimize those improved models for the constraints and diverse hardware at the edge is high, then it’s less likely to happen, and developers are leaving money on the table. A real missed opportunity. 6/ We don’t think that customers should have to choose between accuracy and performance. It’s a false choice, with a high cost. So I’m excited to announce a new feature of SageMaker…
• #25: TODO for the presenter : adjust the banner to the conference you will speak to.