Ad
More Related Content
What's hot (19)
Similar to GPUs vs CPUs for Parallel Processing (20)
Ad
Recently uploaded (20)
Ad
GPUs vs CPUs for Parallel Processing
- 1. GPUs vs CPUs For Parallel Processing Mohammed Billoo MAB Labs, LLC
- 2. Outline ● Overview ● Processor Trends ● CPU Design ● GPU Design ● Comparison ● Summary/Follow-Up 2
- 3. Overview ● GPUs blow CPUs out of the water when it comes to raw processing horsepower of a specific problem set ○ “Specific”: Where computation can be whittled down to a singular algorithm that can be applied across a wide dataset ● Why? ○ The original purpose of CPUs (and their resulting design) has led to this limitation ○ The original purpose of GPUs (and their resulting design) has made them ideal for use in this particular application 3
- 4. Processor Trends ● Previously, for about 20 years, the driving factor in processor design has been performance ○ Processor design had been targeted to provide more features and functionality to users ○ Processor design had been driven by increased clock rate ■ “CPU arms race” ○ Fundamental CPU architecture has been developed to minimize responsiveness of a single application run by a single user ● Since 2003, reduction in size of computational devices has shifted focus from raw processing to energy consumption and heat dissipation ○ Battery life! ○ Resulted in vendors shifting focus from pure clock rate to the number of “cores” in a processor ■ Core = processing element 4
- 5. CPU Design ● Traditionally, most CPU software was developed to behave in a sequential manner ○ Before the advent of multiple cores that can operate in true parallel fashion, either: ■ SW had to play tricks to make it seem that multiple applications were being executed in parallel (relying on increasing CPU clock rates) ■ HW enhancements to make sequential processing “look” parallel (i.e. pipelining) ● With increasing number of cores that can truly run in parallel on a single CPU silicon die, SW developers have had to rethink app development ○ Emphasis has been placed on parallel programs ○ But parallel development is not new! ○ Programs that truly run in parallel have been developed for decades ■ High performance computing applications ■ Run on expensive, dedicated HW 5
- 6. CPU Design ● The fundamental architecture of CPUs has limited the number of cores that can exist on a single silicon die ○ Premise of CPU architecture was to (originally) optimize responsiveness of a single application executed by a single user ○ HW design to support true parallel behavior has had to be “shoehorned”, limiting the number of cores that is attainable ■ Maximum number of CPU cores ⇒ ~10ish ● Nature of original CPU architectures has required additions to support efficient floating point operations ○ Again, because there was no original need to perform floating operations efficiently ○ Required additions to the Instruction Set Architecture (ISA), and in turn modifications to the underlying HW ○ Another alternative was to add a dedicated controller in the processor for floating point operations (i.e. FPU) 6
- 7. CPU Design ● Why can’t more cores be easily included in CPU designs? ○ Over the past ~17 years (since 2003), number of HW cores has increased from 1 → 10ish, 20-ish in CPU designs ○ Limited by the original CPU architecture, since more silicon “real estate” was devoted to: ■ The control logic to transfer instructions and data to the core ■ The processor cache to avoid having to fetch instructions that are frequently used ■ Goal was/has been to keep instruction and data access latencies to a minimum ○ Unfortunately, there is less real estate available for the actual processing cores ● Transfer of data has been another issue ○ Again, because the original problem that CPUs were meant to solve didn’t involve a significant amount of data ○ Data transfer speeds is another issue/bottleneck for faster parallel processing 7
- 8. (Simplified) CPU Design * Fewer resources devoted to “actual” processing (i.e. core) Contr ol Core Core Core Core Cach e 8
- 9. GPU Design ● GPUs were originally (and still are) designed for graphics intensive applications ● Graphics applications are inherently parallel in nature ○ Each pixel is (usually) independent of another pixel ○ The same operations are (usually) performed on each pixel ○ Each frame usually consists of 100k, 1M pixels ● Because of the nature of the problem that GPUs were originally meant to solve, they have become ideal candidates for highly parallel, non-graphics applications ○ Machine Learning ○ Artificial Intelligence ○ Data Science 9
- 10. GPU Design ● The fundamental problem that GPUs were meant to solve has allowed for many more cores to be easily added over the years ○ Don’t really care about responsiveness of a single application but rather the overall execution throughput ■ Gamer doesn’t care about how long it takes for a particular pixel to be rendered, but rather an entire frame ■ A video editor doesn’t care about how long it takes for a particular pixel (or even frame) to be processed, but rather an entire video ○ “Manycore” computing device vs CPU-based “multi-core” computing device ■ Manycore: 10k, 100k, 1M cores ■ Multi-core: Single, double-digit cores ● Nature of graphics applications resulted in native support for fast floating point operations in GPUs ○ Ray-tracing, 2D, 3D graphics inherently must be done using floating point numbers ○ HW was designed to support optimal floating point operations 10
- 11. GPU Design ● Due to original problem that GPUs were meant to solve, adding more cores is much easier than on a CPU ○ Increase in number of cores in a GPU is by orders of magnitude year-over-year (e.g. 10x) ○ GPU architecture allows for fast execution of instructions on a large dataset in parallel ○ More silicon “real estate” devoted to the processing cores themselves vs control logic to transfer instructions and data ● GPU Architecture was developed to allow for transfer of large datasets ○ Graphics processing involves transferring a ton of data at once (e.g. individual frame of pixels) ○ Memory was optimized to NOT be a bottleneck 11
- 12. (Simplified) GPU Design Control Core CoreCore Cache . . . . . . . . . . . Control Core CoreCore . . . . . . . . . . . Control Core CoreCore . . . . . . . . . . . Control Core CoreCore . . . . . . . . . . . * More processor resources devoted to cores 12 Cache Cache Cache
- 13. Comparison Category CPU GPU Number of cores Few (10s, 100s (maybe?)) Many (10k, 100k, 1M) Capability of each core Can perform more complex operations Can perform simpler operations Floating Point Support Added later (either via modifications to the ISA or with a dedicated FPU) Native support in computation core Memory Transfer Slower and much more frequent (can use cache to alleviate this) Faster and much less frequent (usually transfer large dataset between system memory and GPU memory “at once”) SW Development Effort Simpler Complex (requires dataset to be structured a certain way and have to write SW a particular way to leverage HW) 13
- 14. Summary/Follow-Up ● CPUs are the optimal choice for one set of problems and GPUs are the optimal choice for another set of problems ● Can’t use a single processor type ● Need to use both in a complete system ○ Even in a GPU-based system, need file-transfer, network operations, etc.. which are ideally suited for a CPU ● Follow-Up ○ How to implement a simple algorithm on an Nvidia GPU using CUDA C ■ Discuss the challenges that are usually associated with such a task ● Data structure ● Core interactions ● Data transfer from system memory to GPU memory ■ CUDA C ⇒ Extension of the C language to support optimal operations on an Nvidia GPU 14