Heterogeneous Computing on POWER - IBM and OpenPOWER technologies to accelerate your business
Download as PPTX, PDF2 likes1,420 views
Heterogeneous computing refers to systems that use more than one kind of processor and direct applications to run in the processor that is the most efficient for that specific task. Power Systems servers based on the POWER8 processor support several accelerators that are integrated into the system to improve the efficiency of an application.
![Java acceleration with GPUs
“IBM’s POWER group has partnered with
NVIDIA to make GPUs available on a high-
performance server platform, promising the
next generation of parallel performance for
Java applications.”
The Next Wave of Enterprise Performance with Java, POWER Systems and
NVIDIA GPUs
Performance of java.util.Arrays#sort(int[]) on NVIDIA Tesla K40 GPU (ECC enabled) and IBM Power8 CPU.](https://image.slidesharecdn.com/heterogeneous-151209203110-lva1-app6892/85/Heterogeneous-Computing-on-POWER-IBM-and-OpenPOWER-technologies-to-accelerate-your-business-21-320.jpg)
![Query A
Query B
Query C
0
200
400
600
800
Query A Query B Query C Query D Query E Query F
Throughput [MB/s]
SW 1 thread SW 64 threads FPGA 4 streams
Performance](https://image.slidesharecdn.com/heterogeneous-151209203110-lva1-app6892/85/Heterogeneous-Computing-on-POWER-IBM-and-OpenPOWER-technologies-to-accelerate-your-business-41-320.jpg)
Ad
More Related Content
What's hot (20)
Similar to Heterogeneous Computing on POWER - IBM and OpenPOWER technologies to accelerate your business (20)
Ad
Recently uploaded (20)
Ad
Heterogeneous Computing on POWER - IBM and OpenPOWER technologies to accelerate your business
- 1. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM. 9.0 Heterogeneous computing on POWER IBM and OpenPOWER technologies to accelerate your business César Diniz Maciel Executive IT Specialist IBM Corporate Strategy
- 2. Session objectives In this session we present how accelerators provide significant application performance improvements and how they can be deployed in a solution. We evaluate different types of accelerators, and focus on the latest accelerator technologies available for IBM Power Systems and OpenPOWER solutions.
- 3. Acknowledgements I would like to thank the following people for providing invaluable information on POWER8, CAPI and FPGAs • Jeff Stuecheli, POWER Hardware Architect • Bill Starke, DE, POWER Server Nest Architect • Bruce Wile, STG Hardware Design and Verification • Jonathan Dement, Program Director, • Power Systems and OpenPower Innovation • IBM US
- 4. What is heterogeneous computing? From Wikipedia “Heterogeneous computing refers to systems that use more than one kind of processor. These are systems that gain performance not just by adding the same type of processors, but by adding dissimilar processors, usually incorporating specialized processing capabilities to handle particular tasks.“
- 5. What is heterogeneous computing? Not a new concept, widely used in the industry On-chip accelerators: Cryptography and compression accelerators inside the POWER8 processor PCIe based accelerators GPGPUs, such as Nvidia Tesla PCIe adapters for SSL acceleration, cryptography and compression CAPI adapters for POWER8 systems Appliance-based accelerators Netezza appliances to accelerate queries on the IBM DB2 Analytics Accelerator
- 6. Why heterogeneous computing? Applications are becoming more complex and demanding more computing resources Application speedup limited to performance of the slowest algorithm If algorithm execution can be accelerated, application runs faster – code optimization If algorithm can be built on silicon, execution speeds up – ASICs, FPGA If algorithm can be broken in multiple pieces, and these can run simultaneously, application runs faster – parallelization “Heterogeneous (or asymmetric) chip multiprocessors present unique opportunities for improving system throughput, reducing processor power, and mitigating Amdahl’s law. On-chip heterogeneity allows the processor to better match execution resources to each application’s needs and to address a much wider spectrum of system loads—from low to high thread parallelism—with high efficiency.” IEEE Computer, November 2005,
- 7. Why heterogeneous computing? Scaling & power Wall Issues, Chip Design and Fabrication Economics, and Time To Market Demands are all intersecting enabling/requiring change to Business As Usual
- 8. On-chip accelerators • Excellent performance and integration • Accelerators are part of the microprocessor design and share the same silicon die. • Algorithms are implemented in silicon • Fastest performance • Application transparency – Hypervisor/OS abstract the accelerator so that applications do not need to be modified • Close integration with processor design and capabilities • However, on-chip accelerators use same silicon space that could be used for caches, processor features, etc • Tradeoff between accelerators and core features • Once built, they cannot be changed/updated
- 9. Examples of performance benefits of accelerators POWER7+ and POWER8 processors include memory compression accelerators, and cryptographic accelerators Memory Compression Asymmetric Mathematical Functions Cryptographic Engines The on-chip loosely coupled accelerators are part of the processor chip and managed by the Power Hypervisor. Multiple partitions can use the accelerators and the Power Hypervisor manages the QoS and address mapping.
- 10. Examples of performance benefits of accelerators Performance comparison of Active Memory Expansion (AME) on an SAP workload (SD 2-Tier benchmark) on POWER7 and POWER7+ (output of amepat command) Significant reduction in core consumption
- 11. PCIe-based accelerators • Provide great flexibility for adding capability to existing systems • Allow many more options in terms of types of accelerators, algorithms, performance characteristics and implementation devices • GPUs • FPGAs • ASICs • Easy to replace with newer/faster accelerators • However, PCIe-based accelerators are seen as an I/O device, and therefore communicate with the processor and main memory through an I/O subsystem • Programming model needs to incorporate the I/O device • Need of OS device drivers for the accelerator • Data needs to be copied from/to main memory to accelerator memory • Data transfer performance limited by the I/O subsystem
- 12. Java gzip PCIe-based accelerator • Feature code #EJ12/ #EJ13 - PCIe3 FPGA Accelerator Adapter • Also known as Generic Work Queue Engine (GenWQE) accelerator • Hardware compression accelerator for AIX and Linux • FPGA-based PCIe adapter, provides high performance, low latency compression without significant CPU overhead • • Based on the standard compression library – zlib • Widely used open source C library that provides compression and decompression • zlib supports RFC1950, RFC1951, and RFC1952 • • Enabled transparently in IBM Java 7.1.1 release and later on Linux and AIX on POWER8 processor-based systems • HW offload enabled by setting • env variables: • ZLIB_INFLATE_IMPL = 1 • ZLIB_DEFLATE_IMPL = 1
- 13. Java gzip PCIe-based accelerator - Performance IBM internal testing
- 14. GPUs From NVIDIA: • • “GPU-accelerated computing is the use of a graphics processing unit (GPU) together with a CPU to accelerate scientific, analytics, engineering, consumer, and enterprise applications... • …GPU-accelerated computing offers unprecedented application performance by offloading compute-intensive portions of the application to the GPU, while the remainder of the code still runs on the CPU. From a user's perspective, applications simply run significantly faster.”
- 15. General Purpose GPUs GPUs are well suited for parallel processing tasks. They have thousands of cores than work in parallel Coupled with a high performance processor and a GPU programming model, significant application acceleration can be achieved
- 16. 16 NVIDIA K40 GPU • Systems • Up to 2 K40 GPU in S824L • GPU Spec • Kepler-2 architecture GPU • ASIC: GK110B • PCIe interface • PCIe Gen3 x16 • Full length / double wide PCIe form factor • Plugs in using existing double wide cassette • Power • 235W Max power draw :75W via PCIe slot plus 160W via 8-pin Aux. cable. • OS support on POWER • Ubuntu 14.10 or later http://www.nvidia.com/object/tesla-servers.html 16
- 18. NVIDIA and POWER8 From http://www.ecmwf.int/sites/default/files/HPC-WS-Appleyard.pdf
- 19. NVIDIA on Power “ The combination of POWER8 CPUs & NVIDIA Tesla accelerators is amazing. It is the highest performance we have ever seen in individual cores, and the close integration with accelerators is outstanding for heterogeneous parallelization. Thanks to the little endian chip and standard CUDA environment it took us less than 24 hours to port and accelerate GROMACS.” Erik Lindahl, Professor of Biophysics at the Science for Life Laboratory, Stockholm University & KTH. Developer of GROMACS
- 20. GPUs are not only for technical computing “ Concurrent execution of an analytical workload on a POWER8 server with K40 GPUs - A Technology Demonstration” http://on-demand.gputechconf.com/gtc/2015/presentation/S5835-Sina- Meraji.pdf
- 21. Java acceleration with GPUs “IBM’s POWER group has partnered with NVIDIA to make GPUs available on a high- performance server platform, promising the next generation of parallel performance for Java applications.” The Next Wave of Enterprise Performance with Java, POWER Systems and NVIDIA GPUs Performance of java.util.Arrays#sort(int[]) on NVIDIA Tesla K40 GPU (ECC enabled) and IBM Power8 CPU.
- 22. Kepler CUDA 5.5 – 7.0 Unified Memory Buffered Memory POWER8 PCIe Current Pascal CUDA 8 Full GPU Paging Pascal POWER8+ Near Future NVLink SXM2 Volta CUDA 9 Cache Coherent POWER9 More in the future NVLink 2.0 SXM2 Volta Kepler NVIDIA Roadmap on POWER
- 23. FPGA as an Accelerator • FPGA: Field Programmable Gate Array • From Wikipedia: “A field-programmable gate array (FPGA) is an integrated circuit designed to be configured by a customer or a designer after manufacturing – hence "field-programmable". The FPGA configuration is generally specified using a hardware description language (HDL), similar to that used for an application-specific integrated circuit (ASIC). “ • It implements the algorithm as a hardware logic circuit • Not as optimized as an ASIC, but faster than running a software-based algorithm • It can run fast (cycle times of 250 – 500 MHz or more) • It has industry standard interfaces like PCIe Gen3 • The major FPGA Suppliers, Altera and Xilinx, are OpenPOWER Foundation members gzip Encrypt Monte Carlo FPGA Library Source code for FPGAs has traditionally been written in RTL* (VHDL** or Verilog). Now, we also have OpenCL, a more programmer friendly language. *RTL = Register Transfer Level **VHDL = VHSIC*** Hardware Description Language ***VHSIC = Very High Speed Integrated Circuit
- 24. Why FPGAs • Transistor Efficiency & Extreme Parallelism • Bit-level operations • Variable-precision floating point • Compare divides on GPU vs. FPGA • Power-Performance Advantage • >2x compared to a general multicore processor or GPGPU • Unused lookup tables (LUTs) are powered off • Technology Scaling better than CPU/GPU • FPGAs are not frequency or power limited yet • 3D has great potential • Dynamic reconfiguration • Flexibility for application tuning at run-time vs. compile-time • Additional advantages when FPGAs are network connected • allows network as well as compute specialization
- 25. Several IT companies looking at FPGA Baidu FPGA accelerator Microsoft Bing accelerator
- 26. Combining the best of both architectures
- 27. Coherent Accelerator base principles Function Based Acceleration • Main Application executed on Host Processor Computational heavy functions on Accelerator • Single binary image encapsulates both HW and SW version of accelerated functions • Application calls Accelerator for common or custom libraries • Enable an application to work with or without available Accelerator • Accelerator call faults when function not available • Host processor executes function when Accelerator not available • No special requirements on data structures • System software virtualization of accelerator’s function(s) Full Peer to Processor • IBM-designed processor interface • Maintains a trusted coherent interface to system • Direct communications with application • Accelerator function(s) use an unmodified EA • Full access to real address space • Utilize processor’s page tables directly • Page faults handled by system software • Multiple functions can exist in a single accelerator
- 28. Customizable Hardware Application Accelerator • Specific system SW, middleware, or user application • Written to durable interface provided by PSL Virtual Addressing • Accelerator can work with same memory addresses that the processors use • Pointers de-referenced same as the host application • Removes OS & device driver overhead Hardware Managed Cache Coherence • Enables the accelerator to participate in “Locks” as a normal thread Lowers Latency over IO communication model POWER8 CAPI (Coherent Accelerator Processor Interface) Ecosystem CAPP PCIe Power Processor FPGA I/O Ethernet, DASD, etc… Func tion n Func tion 0 Func tion 1 Func tion 2 CAPI IBM Supplied POWER Service Layer
- 29. Coherent Accelerator Processor Interface (CAPI) overview CAPP PCIe POWER8 Processor FPGA Accelerator Function Unit (AFU) CAPI IBM-Supplied POWER Service Layer Typical I/O model flow Flow with a coherent model Shared Memory Notify Accelerator Acceleration Shared Memory Completion DD Call Copy or Pin Source Data MMIO Notify Accelerator Acceleration Poll / Interrupt Completion Copy or Unpin Result Data Return From DD Completion Advantages of coherent attachment over I/O attachment Virtual addressing and data caching – Shared memory – Lower latency for highly referenced data Easier, more natural programming model – Traditional thread-level programming – Long latency of I/O typically requires restructuring of application Enables applications not possible on I/O – Pointer chasing, and so on
- 30. Typical I/O Model Flow: Flow with a Coherent Model: Shared Mem. Notify Accelerator Acceleration Shared Memory Completion DD Call Copy or Pin Source Data MMIO Notify Accelerator Acceleration Poll / Interrupt Completion Copy or Unpin Result Data Ret. From DD Completion Application Dependent, but Equal to below Application Dependent, but Equal to above 300 Instructions 10,000 Instructions 3,000 Instructions 1,000 Instructions 1,000 Instructions 7.9µs 4.9µs Total ~13µs for data prep <400 Instructions <100 Instructions 0.3µs 0.06µs Total 0.36µs CAPI vs. I/O Device Driver: Data Prep
- 31. Time spent by a GPU-accelerated application “ Figure 7 shows that the processing phases ofthe FD-OCT algorithm (DC- Removal, Resample, DispersionCompensation, FFT and Logarithmic Scaling) account for∼40% of the total run-time time, while the memory (data)transfers (host-to-device and device- to-host) require approximately 60% of the time.” From: Scalable, High Performance Fourier Domain Optical Coherence Tomography: why FPGAs and not GPGPUs Jian Li, Marinko V. Sarunic, and Lesley Shannon School of Engineering Science Simon Fraser University, Burnaby BC, Canada
- 32. strategy ( ) CAPI Attached Flash Optimization • Attach IBM FlashSystem to POWER8 via CAPI coherent Attach • Issues Read/Write Commands from applications to eliminate 97% of code pathlength • Saves 20-30 cores per 1M IOPs Pin buffers, Translate, Map DMA, Start I/O Application Read/Write Syscall Interrupt, unmap, unpin,Iodone scheduling 20K instructions reduced to <500 Disk and Adapter DD strategy ( ) iodone ( ) FileSystem Application User Library Posix Async I/O Style API Shared Memory Work Queue aio_read() aio_write() iodone ( ) LVM
- 33. Innovative “In-Memory” NoSQL/KVS IBM Data Engine for NoSQL 24:1 Reduction in infrastructure 2.4x Price reduction 12x Less Energy 6x Less rack space 40TB of extended memory 4U
- 34. Demonstrating the Value of CAPI Attachment IBM Data Engine for NoSQL
- 35. Comparison of FFT on three paradigms Same FFT Algorithm as CAPI, but large latency impact P8 Core working much harder to deliver data to FPGA as device driver code is invoked Small FFTs can be implemented in a fully streamed fashion on FPGA Performance scales with host-AFU bandwidth Use radix-2 pipeline for up to 4 GB/s Radix-2 needs 2 complex samples per cycle → 16B / 250MHz cycle → 4 GB/s Switch to (more efficient) radix-4 for up to 8 GB/s Software FFT from the IBM Engineering and Scientific Subroutine Library (ESSL) Poor performance for small FFTs using multi-threaded software Little data reuse and strided access patterns lead to software inefficiencies P8 Core P8 Core FPGA P8 Core FPGA CAPI
- 36. FFT Results
- 37. CAPI projects – PDT planning • FPGA-CAPI accelerator for PDT (Photodynamic Therapy) • Monte-Carlo analysis of light scattering in tissue • Used to plan for non-invasive cancer treatment • Project led by Jeff Cassidy (University of Toronto) and • Lothat Lilge (Princess Margaret Cancer Centre), Canada, • with IBM Austin support.
- 38. CAPI projects – Text Analytics
- 41. Query A Query B Query C 0 200 400 600 800 Query A Query B Query C Query D Query E Query F Throughput [MB/s] SW 1 thread SW 64 threads FPGA 4 streams Performance
- 42. Additional CAPI projects • Algo acceleration for risk analysis for High Performance Trading • Biometric (image and voice recognition) acceleration for secure • identification for banking system • Flash acceleration (NVMe) • Video transcoding and compression for IPTV/VoD • Video analytics • • .. And many more • • • • If you have an application, or know an application that can benefit from FPGA and CAPI... Talk to IBM!!!
- 43. References First Annual OpenPOWER Summit – presentations and solutions on CAPI and GPU accelerators Porting GPU-Accelerated Applications to POWER8 Systems POWER8 Coherent Accelerator Processor Interface (CAPI) IBM Data Engine for NoSQL – Power Systems Edition Accelerating performance with the Generic Work Queue Engine (GenWQE)
- 44. © Copyright IBM Corporation 2015 Continue growing your IBM skills ibm.com/training provides a comprehensive portfolio of skills and career accelerators that are designed to meet all your training needs. • Training in cities local to you - where and when you need it, and in the format you want – Use IBM Training Search to locate public training classes near to you with our five Global Training Providers – Private training is also available with our Global Training Providers • Demanding a high standard of quality – view the paths to success – Browse Training Paths and Certifications to find the course that is right for you • If you can’t find the training that is right for you with our Global Training Providers, we can help. – Contact IBM Training at [email protected] 44 Global Skills Initiative