Adding Image and Video Analysis to your Applications (May 2018)

Editor's Notes

• #8: Let’s take a look at what Rekognition Image can do and how it has evolved over the past year Last re:Invent we launched Rekognition with Face detection, Face recognition and Object and Scene Detection In April, we added Unsafe Image detection. . In June, we launched Celebrity Recognition. In November, we launched Text In Image. And we also updated Face Detection, added real-time Face recognition, and updated our Moderation models …and we are continually improving based on customer feedback.
• #13: You can use the ‘MinConfidence’ parameter in your API requests to balance detection of content (recall) vs the accuracy of detection (precision). 
• #15: You can use the ‘MinConfidence’ parameter in your API requests to balance detection of content (recall) vs the accuracy of detection (precision). 
• #17: SageMaker is going to make it much easier for everyday developers to build machine-learning models. But, people and developers are still really interested in learning more about how they can use machine learning. They want to do it, so they're reading all kinds of literature, and there are some code samples they can play around with. But, for any of us who've had to learn something new that has any kind of complexity, there's no substitute for hands-on training and application. And so we thought about: What can we do that would allow our builders and our developers to get this hands-on training? Our teams worked on this problem and developed AWS DeepLens, which is the world's first wireless deep-learning-enabled video-camera for developers.
• #19: Current solutions mostly rely on having humans review and catalog the most important footage. It is expensive, slow, and error-prone, but it also leaves potential value on the table with the remaining uncatalogued video. Some people have tried applying traditional image analysis solutions to videos by sampling individual frames, but this loses important context. For example, one frame may detect a person standing, and another person lying down. However, the analysis would miss the context that this is the same person who has had a sudden fall. This solution also doesn’t support analysis of real-time video streaming. Existing solutions for extracting data on streaming video is expensive and does not scale. For example, Home IoT cameras providers are seeking a high quality analytics solution that provides output at low latency and does not trigger false alerts.
• #23: We heard a need from customers they would like to have a deep learning-based Video analysis service Introducing Amazon Rekognition Video: a deep learning-based video recognition and analysis service for live or stored video that offers: Object and Activity Detection: With Rekognition Video, you can detect thousands of objects and activities accurately, and extract motion-based context from a video. For example, your application can quickly analyze a consumer video of a baby's first walk, and generate labels like "baby”, "crawling”, "falling”, and "hugging” with timestamps and confidence scores. This allows you to generate search indexes for consumer video archives. Person Tracking: With Rekognition Video motion-based video analysis capabilities, you can track people through the video even when their faces are not visible, or as the whole person might go in and out of the scene. This makes investigation and monitoring of individuals easy and accurate. Face Recognition: Rekognition Video allows you to identify persons from a large existing collection of faces. This allows you to build applications that locate a Person of Interest in real time from a live stream or in batch from Amazon S3. Streaming mode by native integration with Amazon Kinesis Video Streams: In streaming mode, the face detection and recognition APIs natively integrate with video stream from Kinesis Video Streams to provide output at low latency. Kinesis Video Streams enables developers to transmit thousands of live feeds and associated metadata to persist them on Amazon S3 and Glacier. Unsafe Video Detection: Rekognition Video enables easy filtering of video for explicit and suggestive content, providing fine-grained detection of labels associated with inappropriate / NSFW content at a video frame level. Celebrity Recognition: With Rekognition Video, you can detect and recognize celebrities in a video and track in which video frames each of them appears. This allows you to index and search digital video libraries for celebrities based on your particular interest. Feature capabilities summary: Contextual insight - motion and real-time Fully managed, scalable, and easy-to-use video analysis service Deep learning-based -> Continuously improving Integrated with Amazon S3, Amazon Kinesis Video Streams, AWS Lambda – get started with video analysis right out-of-the-box
• #25: A mobile app uploads new videos to S3 An S3 notification triggers a Lambda function which calls Rekognition Video’s APIs with an S3 url DetectLabels analyzes the video and returns labels for objects and scenes detected in the video CelebrityRecognition returns celebrities recognized in the videos This output is persisted as metadata into DynamoDB to ensure durability Lambda writes the metadata to Elasticsearch The application provides a smart search index (powered by Elasticsearch), and serves the selected videos directly from S3
• #27: Live streaming for immediate response Accurate analysis of a massive amount of data for investigative response Recognize a person across a collection of millions at low latency and high accuracy in streaming mode, across hundreds of cameras. Track a person of interest by outputting time segments when the person appeared in the video. Create alerts by detecting objects and activities of interest, such as car, license plate, running etc.
• #31: So far, we've discussed the bottom and middle layers of the machine learning stack – first we talked about the frameworks and the deep learning AMI for expert practitioners. Then, SageMaker and DeepLens in the middle layer to bring ML capabilities to all developers. Now, at the top of the stack, we serve developers and companies who want to add solution-oriented intelligence to their applications through an API call rather than developing and training their own models. These are services that exhibit artificial intelligence that emulates a human’s cognitive skills. Last year, we announced three services in this area: Amazon Rekognition (image analysis), Amazon Polly (text-to-speech), and Amazon Lex (conversational applications).