Implement Runtime Environments for HSA using LLVM
17 likes6,074 views
* Transition to heterogeneous - mobile phone to data centre * LLVM and HSA * HSA driven computing environment
![OpenCL Framework overview
Host Program
int main(int argc, char **argv)
{
...
clBuildProgram(program, ...);
clCreateKernel(program, “dot”...);
...
}
OpenCL Kernels
__kernel void dot(__global const float4 *a
__global const float4 *b
__global float4 *c)
{
int tid = get_global_id(0);
c[tid] = a[tid] * b[tid];
}
Application
Separate programs into
host-side and kernel-side
code fragment
OpenCL Framework
OpenCL Runtime
Runtime APIs
Platform APIs
OpenCL Compiler
Compiler
• compile OpenCL C
language just-in-time
Runtime
• allow host program to
manipulate context
Front-end
Front-end
Back-end
Back-end
Platform
MPU
GPU
MPU : host, kernel program
GPU : kernel program
Source: OpenCL_for_Halifux.pdf, OpenCL overview, Intel Visual Adrenaline](https://image.slidesharecdn.com/hsa-llvm-140123185623-phpapp02/85/Implement-Runtime-Environments-for-HSA-using-LLVM-13-320.jpg)
![Supporting OpenCL
• Syntax parsing by compiler
–
–
–
–
qualif i r
e
vector
built-in function
Optimizations on
single core
__kernel void add( __global float4 *a,
__global float4 *b,
__global float4 *c)
{
int gid = get_global_id(0);
float4 data = (float4) (1.0, 2.0, 3.0, 4.0);
c[gid] = a[gid] + b[gid] + data;
}
• Runtime implementation
– handle multi-core issues
– Co-work with device vendor
Platform APIs
MPU
Runtime APIs
GPU](https://image.slidesharecdn.com/hsa-llvm-140123185623-phpapp02/85/Implement-Runtime-Environments-for-HSA-using-LLVM-15-320.jpg)
![HSAIL example
ld_kernarg_u64 $d0, [%_out];
ld_kernarg_u64 $d1, [%_in];
sqrt((float
(i*i +
(i+1)*(i+1) +
(i+2)*(i+2)))
@block0:
workitemabsid_u32 $s2, 0;
cvt_s64_s32 $d2, $s2;
mad_u64 $d3, $d2, 8, $d1;
ld_global_u64 $d3, [$d3];
//pointer
ld_global_f32 $s0, [$d3+0]; // x
ld_global_f32 $s1, [$d3+4]; // y
ld_global_f32 $s2, [$d3+8]; // z
mul_f32 $s0, $s0, $s0;
// x*x
mul_f32 $s1, $s1, $s1;
// y*y
add_f32 $s0, $s0, $s1; // x*x + y*y
mul_f32 $s2, $s2, $s2;
// z*z
add_f32 $s0, $s0, $s2;// x*x + y*y + z*z
sqrt_f32 $s0, $s0;
mad_u64 $d4, $d2, 4, $d0;
st_global_f32 $s0, [$d4];
ret;](https://image.slidesharecdn.com/hsa-llvm-140123185623-phpapp02/85/Implement-Runtime-Environments-for-HSA-using-LLVM-36-320.jpg)
Ad
More Related Content
What's hot (20)
Viewers also liked (9)
Ad
Similar to Implement Runtime Environments for HSA using LLVM (20)
Ad
More from National Cheng Kung University (9)
Recently uploaded (20)
Implement Runtime Environments for HSA using LLVM
- 1. Implement Runtime Environments for HSA using LLVM Jim Huang ( 黃敬群 ) <[email protected]> Oct 17, 2013 / ITRI, Taiwan
- 2. About this presentation • Transition to heterogeneous – mobile phone to data centre • LLVM and HSA • HSA driven computing environment
- 5. Herb Sutter’s new outlook http://herbsutter.com/welcome-to-the-jungle / “In the twilight of Moore’s Law, the transitions to multicore processors, GPU computing, and HaaS cloud computing are not separate trends, but aspects of a single trend – mainstream computers from desktops to ‘smartphones’ are being permanently transformed into heterogeneous supercomputer clusters. Henceforth, a single computeintensive application will need to harness different kinds of cores, in immense numbers, to get its job done.” “The free lunch is over. Now welcome to the hardware jungle.”
- 6. Four causes of heterogeneity • Multiple types of programmable core • • • CPU (lightweight, heavyweight) GPU Accelerators and application-specific • Interconnect asymmetry • Memory hierarchies • Service oriented software demanding
- 7. High Performance vs. Embedded Embedded Type of processors Size Memory HPC Heterogeneous Homogeneous Small Massive Shared Distributed but getting closer day by day...
- 8. Heterogeneity is mainstream Dual-core ARM 1.4GHz, ARMv7s CPU Quad-core ARM Cortex A9 CPU Quad-core SGX543MP4+ Imagination GPU Triple-core SGX554MP4 Imagination GPU Most tablets and smartphones are already powered by heterogeneous processors.
- 9. Source: Heterogeneous Computing in ARM
- 11. Current limitations • Disjoint view of memory spaces between CPUs and GPUs • Hard partition between “host” and “devices” in programming models • Dynamically varying nested parallelism almost impossible to support • Large overheads in scheduling heterogeneous, parallel tasks
- 13. OpenCL Framework overview Host Program int main(int argc, char **argv) { ... clBuildProgram(program, ...); clCreateKernel(program, “dot”...); ... } OpenCL Kernels __kernel void dot(__global const float4 *a __global const float4 *b __global float4 *c) { int tid = get_global_id(0); c[tid] = a[tid] * b[tid]; } Application Separate programs into host-side and kernel-side code fragment OpenCL Framework OpenCL Runtime Runtime APIs Platform APIs OpenCL Compiler Compiler • compile OpenCL C language just-in-time Runtime • allow host program to manipulate context Front-end Front-end Back-end Back-end Platform MPU GPU MPU : host, kernel program GPU : kernel program Source: OpenCL_for_Halifux.pdf, OpenCL overview, Intel Visual Adrenaline
- 14. OpenCL Execution Scenario kernel { code fragment 1 code fragment 2 code fragment 3 } Task-level parallelism 1 2 3 X O Data-level parallelism kernel A { code 1 code 2 code 3 } A:1,2,3 data(0) A:1,2,3 data(2) A:1,2,3 data(3) kernel A { code 1 } A:1,2,3 data(1) A:1,2,3 data(4) A:1,2,3 data(5) kernel B { code 2 } kernel C { code 3 } OpenCL Runtime A:1 B:2 C:3 A:1 B:2 C:3 14
- 15. Supporting OpenCL • Syntax parsing by compiler – – – – qualif i r e vector built-in function Optimizations on single core __kernel void add( __global float4 *a, __global float4 *b, __global float4 *c) { int gid = get_global_id(0); float4 data = (float4) (1.0, 2.0, 3.0, 4.0); c[gid] = a[gid] + b[gid] + data; } • Runtime implementation – handle multi-core issues – Co-work with device vendor Platform APIs MPU Runtime APIs GPU
- 16. LLVM-based compilation toolchain Data in Module 1 . . . opt & link Front ends Linked module Profiling ? yes lli Module N no Back end 1 Exe 1 . . . Exe M . . . Back end M Module 1 . . . Module M Partitioning & mapping Estimation Profile info
- 18. HSA overview • open architecture specification • HSAIL virtual (parallel) instruction set • HSA memory model • HSA dispatcher and run-time • Provides an optimized platform architecture for heterogeneous programming models such as OpenCL, C++AMP, et al 18
- 20. Heterogeneous Programming • Unified virtual address space for all cores • CPU and GPU • distributed arrays • Hardware queues per code with lightweight user mode task dispatch • Enables GPU context switching, preemption, efficient heterogeneous scheduling • First class barrier objects • Aids parallel program composability
- 21. HSA Intermediate Layer (HSAIL) Virtual ISA for parallel programs Similar to LLVM IR and OpenCL SPIR Finalised to specific ISA by a JIT compiler Make late decisions on which core should run a task • Features: • • • • • Explicitly parallel • Support for exceptions, virtual functions and other high-level features • syscall methods (I/O, printf etc.) • Debugging support
- 22. HSA stack for Android (conceptual)
- 23. HSA memory model • Compatible with C++11, OpenCL, Java and .NET memory models • Relaxed consistency • Designed to support both managed language (such as Java) and unmanaged languages (such as C) • Will make it much easier to develop 3rd party compilers for a wide range of heterogeneous products • E.g. Fortran, C++, C++AMP, Java et al
- 24. HSA dispatch • HSA designed to enable heterogeneous task queuing • A work queue per core (CPU, GPU, …) • Distribution of work into queues • Load balancing by work stealing • Any core can schedule work for any other, including itself • Significant reduction in overhead of scheduling work for a core
- 25. Today’s Command and Dispatch Flow Command Flow Application A Direct3D Data Flow User Mode Driver Soft Queue Command Buffer Kernel Mode Driver DMA Buffer GPU HARDWARE A Hardware Queue
- 26. Today’s Command and Dispatch Flow Command Flow Application A Direct3D Data Flow User Mode Driver Soft Queue Kernel Mode Driver Command Buffer Command Flow DMA Buffer Data Flow B A B C Application B Direct3D User Mode Driver Soft Queue Kernel Mode Driver Command Buffer Command Flow Application C Direct3D GPU HARDWARE A DMA Buffer Data Flow User Mode Driver Soft Queue Command Buffer Hardware Queue Kernel Mode Driver DMA Buffer
- 27. HSA Dispatch Flow ✔ ✔ ✔ ✔ ✔ ✔ Software View User-mode dispatch to hardware No kernel mode driver overhead Low dispatch latency ✔ ✔ ✔ ✔ ✔ ✔ Hardware View HW / microcode controlled HW scheduling HW-managed protection
- 28. HSA enabled dispatch Application / Runtime CPU1 CPU2 GPU
- 29. HSA Memory Model • compatible with C++11/Java/.NET memory models • Relaxed consistency memory model • Loads and stores can be re-ordered by the finalizer
- 30. Data Flow of HSA
- 31. HSAIL • intermediate language for parallel compute in HSA – Generated by a high level compiler (LLVM, gcc, Java VM, etc) – Compiled down to GPU ISA or parallel ISAFinalizer – Finalizer may execute at run time, install time or build time • low level instruction set designed for parallel compute in a shared virtual memory environment. • designed for fast compile time, moving most optimizations to HL compiler – Limited register set avoids full register allocation in finalizer
- 32. GPU based Languages • Dynamic Language for exploration of heterogeneous parallel runtimes • LLVM-based compilation – Java, Scala, JavaScript, OpenMP • Project Sumatra:GPU Acceleration for Java in OpenJDK • ThorScript
- 33. Accelerating Java Source: HSA, Phil Rogers
- 34. Open Source software stack for HSA A Linux execution and compilation stack is open-sourced by AMD • Jump start the ecosystem • Allow a single shared implementation where appropriate • Enable university research in all areas Component Name Purpose HSA Bolt Library Enable understanding and debug OpenCL HSAIL Code Generator Enable research LLVM Contributions Industry and academic collaboration HSA Assembler Enable understanding and debug HSA Runtime Standardize on a single runtime HSA Finalizer Enable research and debug HSA Kernel Driver For inclusion in Linux distros
- 35. Open Source Tools • • • • Hosted at GitHub: https://github.com/HSAFoundation HSAIL-Tools: Assembler/Disassembler Instruction Set Simulator HSA ISS Loader Library for Java and C++ for creation and dispatch HSAIL kernel
- 36. HSAIL example ld_kernarg_u64 $d0, [%_out]; ld_kernarg_u64 $d1, [%_in]; sqrt((float (i*i + (i+1)*(i+1) + (i+2)*(i+2))) @block0: workitemabsid_u32 $s2, 0; cvt_s64_s32 $d2, $s2; mad_u64 $d3, $d2, 8, $d1; ld_global_u64 $d3, [$d3]; //pointer ld_global_f32 $s0, [$d3+0]; // x ld_global_f32 $s1, [$d3+4]; // y ld_global_f32 $s2, [$d3+8]; // z mul_f32 $s0, $s0, $s0; // x*x mul_f32 $s1, $s1, $s1; // y*y add_f32 $s0, $s0, $s1; // x*x + y*y mul_f32 $s2, $s2, $s2; // z*z add_f32 $s0, $s0, $s2;// x*x + y*y + z*z sqrt_f32 $s0, $s0; mad_u64 $d4, $d2, 4, $d0; st_global_f32 $s0, [$d4]; ret;
- 37. Conclusions • Heterogeneity is an increasingly important trend • The market is finally starting to create and adopt the necessary open standards • HSA should enable much more dynamically heterogeneous nested parallel programs and programming models
- 38. Reference • Heterogeneous Computing in ARM (2013) • Project Sumatra: GPU Acceleration for Java in OpenJDK – http://openjdk.java.net/projects/sumatra/ • HETEROGENEOUS SYSTEM ARCHITECTURE, Phil Rogers