Develop and optimize CV/DL applications with Intel OpenVINO toolkit
Download as PPTX, PDF2 likes533 views
High level introduction of OpenVINO toolkit features that make it best tool for CV/DL deployment @ Intel platforms
Ad
More Related Content
What's hot (20)
Similar to Develop and optimize CV/DL applications with Intel OpenVINO toolkit (20)
Ad
Recently uploaded (20)
Ad
Develop and optimize CV/DL applications with Intel OpenVINO toolkit
- 2. Intel ConfidentialIntel Confidential Brief OpenVINO™ Introduction • OpenVINO ™ is • set of tools and libraries for CV/DL application developers • high performance, low footprint solution for deployment • API for unified access to CV/DL capabilities of Intel platforms • OpenVINO ™ is not • tool for data scientists • solution for training of deep learning models
- 3. Intel ConfidentialIntel Confidential 3 OpenVINO™ Benefits • Powerful combination of highly optimized Classical CV and DL primitives • Allows to run inference on Intel CPU, GPU, VPU and FPGA • Best performing solution on all Intel architectures • ~2x faster than fastest TensorFlow and MxNet on CPU • Smallest execution footprint (lowest memory consumption) • 2x smaller than MXNet, 4x smaller than TensorFlow • Minimum # of dependencies (no dependencies on training frameworks) • Multiple OS support (Linux and Windows)
- 4. Intel ConfidentialIntel Confidential 4 OpenVINO™ Specifics • Highly optimized implementations of DL primitives • Most efficient on each Intel platform • Focus on inference only • Aggressive layer fusion at the inference step • Including HW accelerated steps tuned for inference • Efficient activation memory reuse • Often close to bare minimum
- 5. Intel ConfidentialIntel Confidential 5 DL Workflow Caffe MXNet TensorFlow Caffe2 PyTorch Serialized trained DL model ONNX MKLDNN Plugin clDNN Plugin FPGA Plugin Myriad Plugin Inference Engine Deploy Application Model Optimizer IR .xml .bin Step 1: Import model from Framework format to Framework independent representation Step 2: Update application to use Inference Engine API Eliminate unnecessary layers, lossless fusion where possible Remove framework dependency Accuracy against original model ensured
- 6. Intel ConfidentialIntel Confidential 6 Customization Capabilities • OpenVINO™ provides good coverage of DL primitives out of the box • Constantly growing list of primitives to support new DL topologies • Frequent releases, substantial additions • Not a problem if something is missing! • Good extension mechanism for adding new primitives • Possible to add proprietary layers, more optimized layers, etc. • Both in Model Optimizer and Inference Engine (import and run)
- 7. Intel ConfidentialIntel Confidential 7 Application Design Workflow Desig n Verify logic Debug Fix Best done on CPU: - Easier to verify - Simpler debugging procedures CHANGE TARGET Check scalability Fix pipeline System testing Best done on Actual target (e.g. VPU): - Exact performance - Correct timings Accuracy and functionality across targets
- 8. Intel ConfidentialIntel Confidential 8 Heterogeneous Execution • When a certain primitive is not supported on a target • Custom proprietary primitive or inefficient HW for a task • Heterogeneous execution ensures full topology execution • Automatic data transfer between targets whenever needed • Work splitting and scheduling • No need to do any manual network manipulations! FPGA CPU
- 9. Intel ConfidentialIntel Confidential 9 Power of Parallel Execution Asynchronous API provides capabilities for: • Running main thread in parallel with ongoing inference (CPU/GPU) • Hiding data transfer latency for accelerators (VPU, FPGA) • Filling accelerators with work Transfer 1 Inference 1 Transfer 2 Transfer 1 Transfer 2 Inference 2 Inference 1 Inference 2 Result 1 Result 1 Result 2 Result 2 Sync API Async API
- 10. Intel ConfidentialIntel Confidential 10 Efficient Frame Preprocessing DECODE Layout Transform Resize to network Detection network Resize to network Object Analysis Network Crop Preprocessing DL Inference • Preprocessing is typically done via manual coding or using libraries • OpenCV is most popular • Fastest OpenCV build is available in OpenVINO™ • Deep Learning Inference Engine encapsulates basic preprocessing capability • Automatic frame resize based on input frame and network size • Automatic layout conversion and cropping • In a nutshell –> Just provide frame and it will be suitable for inference automatically
- 11. Intel ConfidentialIntel Confidential 11 INT8 Quantization for CPU • INT8 provides additional acceleration using AVX-512 • Not all CPU targets support it • Very minor quality/accuracy loss • No retraining is required • No code update is needed IR Dataset Analysis tool Updated IR
- 12. Intel ConfidentialIntel Confidential 12 OpenVINO™ Model Zoo • OpenVINO provides pre-trained DL models for deployment • Lightweight, low compute, real time on Intel platforms • Cover popular CV use cases • Face analysis, Security use-cases (person, vehicle, bicycle detection) • Transportation analytics (road segmentation, vehicle/pedestrian detection)
- 13. Intel ConfidentialIntel Confidential 13 Extensive Set of Samples • Actual examples of applications, not just API demonstration • Switching between targets, work distribution • Multiple models in pipeline with preprocessing • Open Models Zoo demonstration
- 14. 14