![HUAWEI TECHNOLOGIES CO., LTD. 6
CPU is not fast/effective/… enough
Heterogeneous is the future
Nvidia GPU-compute, Intel Skylake Xeon+FPGA, AWS P2/F1 Instance
use the right processor, in the right place, at the right time
Why Accelerator?
Container
a.k.a. Docker
[baidu, HotChips 2016]](https://image.slidesharecdn.com/makeacceleratorpluggableforcontainerengine-170620145928/85/Make-Accelerator-Pluggable-for-Container-Engine-6-320.jpg)
![[20200720]cloud native develoment - Nelson Lin](https://cdn.slidesharecdn.com/ss_thumbnails/20200720cloudnativedeveloment-200824031228-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (20)
Viewers also liked (19)
Ad
Similar to Make Accelerator Pluggable for Container Engine (20)
Ad
More from LinuxCon ContainerCon CloudOpen China (8)
Recently uploaded (20)
Make Accelerator Pluggable for Container Engine
- 1. www.huawei.com HUAWEI TECHNOLOGIES CO., LTD. Make Accelerator Pluggable for Container Engine Author/ Email: Jiuyue Ma / [email protected] Version: V1.0 (20150504)
- 2. HUAWEI TECHNOLOGIES CO., LTD. 2 Today’s Topic Container Engine Hardware Accelerators e.g. GPU、QAT、FPGA OLTP/OLAP BigData Processing Neural Network CloudRAN Pluggable Accelerator Support for Container Engine
- 3. HUAWEI TECHNOLOGIES CO., LTD. 3 Agenda Use accelerator in containers and How? Key problems to deal with Identify accelerator requirements Prepare accelerator runtimes Manage accelerator resources Further development
- 4. HUAWEI TECHNOLOGIES CO., LTD. 4 Why Container?
- 5. HUAWEI TECHNOLOGIES CO., LTD. 5 Why Container? Why NOT Container? * Raphael Da Silva. OPEN SOURCE TECHNOLOGY: Docker Containers on IBM Bluemix. 2015.
- 6. HUAWEI TECHNOLOGIES CO., LTD. 6 CPU is not fast/effective/… enough Heterogeneous is the future Nvidia GPU-compute, Intel Skylake Xeon+FPGA, AWS P2/F1 Instance use the right processor, in the right place, at the right time Why Accelerator? Container a.k.a. Docker [baidu, HotChips 2016]
- 7. HUAWEI TECHNOLOGIES CO., LTD. 7 Google it ! “Use XXX in docker” NVIDIA’s GPU-Docker Solution DPDK FPGA GPU
- 8. HUAWEI TECHNOLOGIES CO., LTD. 8 Use GPU in Docker (from nvidia-docker) Wrapped Command docker run --volume-driver=nvidia-docker --volume=nvidia_driver_361.48:/usr/local/nvidia:ro ③ nvidia-docker cli-wrapper $ ldconfig -p | grep -E 'nvidia|cuda' libnvidia-ml.so (libc6,x86-64) => /usr/lib/nvidia-361/libnvidia-ml.so libnvidia-glcore.so.361.48 (libc6,x86-64) => /usr/lib/nvidia-361/lib… libnvidia-compiler.so.361.48 (libc6,x86-64) => /usr/lib/nvidia-361/lib… libcuda.so (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libcuda.so ... $ lsmod | grep nvidia nvidia_uvm 711531 2 nvidia_modeset 742329 0 nvidia 10058469 80 nvidia_modeset,nvidia_uvm Key Problem：VERSION MISMATCH $ nvidia-smi Failed to initialize NVML: Driver/library version mismatch ② VOLUME: nvidia-docker/nvidia_driver_361.48 ① Labeled Image LABEL com.nvidia.volumes.needed="nvidia_driver" LABEL com.nvidia.cuda.version="7.5" • detect/check cuda version • hook docker cli args --volume-driver / --volume nvidia
- 9. HUAWEI TECHNOLOGIES CO., LTD. 9 Do we really need “nvidia-docker”? If so, we also need: dpdk-docker, qat-docker, fpga-docker, … But, if we need both GPU and DPDK? nvidia DPDK QAT FPGA DPDK nvidia Pluggable Accelerator
- 10. HUAWEI TECHNOLOGIES CO., LTD. 10 Device Files (-d device) /dev/nvidia0 /dev/nvidia-smi /dev/nvidia-uvm /dev/uio0 /dev/fpga0 File Bindings (-v src:dest) /usr/lib/nvidia-361/libnvidia-ml.so /usr/lib/nvidia-361/libnvidia-glcore.so.361.48 /usr/lib/nvidia-361/libnvidia- compiler.so.361.48 /usr/lib/x86_64-linux-gnu/libcuda.so ... /sys/bus/pci/devices/xxxx:xx:xx.x /dev/hugepages /usr/lib/libfpga.so /sys/bus/pci/devices/xxxx:xx:xx.x /dev/hugepages Environments (-e env) PATH=$PATH:/usr/lib/nvidia-361/bin LD_LIBRARY_PATH=… Pluggable Accelerator, How? What’s the difference? nvidia DPDK FPGA Accelerator Plugin Interface in-house FPGA Accel
- 11. HUAWEI TECHNOLOGIES CO., LTD. 11 Implementation (1/3): IDENTIFY Accelerator Requirements We are using the function provided by device, not the device itself Identified by runtime instead of device e.g. “A SHA256 accelerator” instead of “A FPGA with XXXX-bitstream” Identify user requirements Integrate into image Specify when start container Identify device capabilities Identified by “accelerator plugin” Report to engine through plugin api LABEL runtime gpu0=cuda:7.5 $ docker run --accel fpga:crypto huawei/crypto-image image cli / rest-api dockerd Device Accel Runtime /dev/nvidia0 cuda:7.5, cuda:8.0, OpenCL 2.0, … /dev/qat0 md5, sha256, aes, … /dev/fpga0 crypto, compress, decode, … Accelerator Plugin Plugin API
- 12. HUAWEI TECHNOLOGIES CO., LTD. 12 Implementation (2/3): PREPARE Select plugin from accelerator requirements Device initialization by plugin Dynamic inject required libraries & devices DPDK-plugin - eal.so/… - igb_uio.ko - /dev/uioX FPGA-plugin - fpga.so - fpga.ko - /dev/fpgaX container Host Application fpga.ko v1 dockerd /dev/fpga0 generate bind libacc accel requirements PaaS fpga.so/dev/fpga0 fpga.so v1 (2) mount userspace lib (3) mount device GPU-plugin - gpu.so / cuda.so - gpu.ko - /dev/gpuX QAT-plugin - qat.so - qat.ko - /dev/qatX
- 13. HUAWEI TECHNOLOGIES CO., LTD. 13 Implementation (3/3): MANAGEMENT Running CreatedStopped CreatingDeleting INIT Container Life-Cycles allocate persistent allocate non-persistent release non-persistent release persistent prepare parse accel requirements Accelerator Plugin Accelerator Life-Cycle Allocate Prepare Release Plugin hooks for each stage Integrate with Container Life- Cycle Persistent：CreateDelete Non-persistent： StartStop
- 14. HUAWEI TECHNOLOGIES CO., LTD. 14 Example Flow
- 15. HUAWEI TECHNOLOGIES CO., LTD. 15 Further Developments Standardize “Accelerator Runtime” OCI Runtime/Image Spec should be good place Integrate with Docker Swarm / K8S / etc Cluster level accelerator schedule&management More Features Accelerator Sharing Accelerator Exception Interface Accelerator Hot-Plug …
- 16. Copyright© 2011 Huawei Technologies Co., Ltd. All Rights Reserved. The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any time without notice. Thank you www.huawei.com