Spark Summit EU talk by Sameer Agarwal
Download as PPTX, PDF4 likes2,206 views
This document discusses Project Tungsten, which aims to substantially improve the memory and CPU efficiency of Spark. It describes how Spark has optimized IO but the CPU has become the bottleneck. Project Tungsten focuses on improving execution performance through techniques like explicit memory management, code generation, cache-aware algorithms, whole-stage code generation, and columnar in-memory data formats. It shows how these techniques provide significant performance improvements, such as 5-30x speedups on operators and 10-100x speedups on radix sort. Future work includes cost-based optimization and improving performance on many-core machines.
![Phase 1
Spark 1.4 - 1.6
Memory Management
Code Generation
Cache-aware Algorithms
Phase 2
Spark 2.0+
Whole-stage Code Generation
Columnar in Memory Support
Both whole stage codegen [SPARK-12795] and the vectorized
parquet reader [SPARK-12992] are enabled by default in Spark 2.0+
Demo](https://image.slidesharecdn.com/sparkperf-sparksummiteu16-final-161103182837/85/Spark-Summit-EU-talk-by-Sameer-Agarwal-30-320.jpg)
Spark Summit EU talk by Sameer Agarwal
- 1. Apache Spark’s Performance Project Tungsten and Beyond Sameer Agarwal Spark Summit| Brussels| Oct 26th 2016
- 2. About Me • Software Engineer at Databricks (Spark Core/SQL) • PhD in Databases (AMPLab, UC Berkeley) • Research on BlinkDB (Approximate Queries in Spark)
- 5. Hardware Trends 2010 2016 Storage 50+MB/s (HDD) 500+MB/s (SSD) Network 1Gbps 10Gbps CPU ~3GHz ~3GHz
- 6. Hardware Trends 2010 2016 Storage 50+MB/s (HDD) 500+MB/s (SSD) 10X Network 1Gbps 10Gbps 10X CPU ~3GHz ~3GHz
- 7. On the flip side Spark IO has been optimized • Reduce IO by pruning input data that is not needed • New shuffle and network implementations (2014 sort record) Data formats have improved • E.g. Parquet is a “dense” columnar format CPU increasingly the bottleneck; trend expected to continue
- 8. Goals of Project Tungsten Substantially improve the memory and CPU efficiency of Spark backend execution and push performance closer to the limits of modern hardware. Note the focus on “execution” not “optimizer”: relatively easy to pick broadcast hash join that is 1000X faster than Cartesian join, but hard to optimize broadcast join to be an order of magnitude faster.
- 9. Phase 1 Foundation Memory Management Code Generation Cache-aware Algorithms Phase 2 Order-of-magnitude Faster Whole-stage Codegen Vectorization
- 10. Phase 1 Laying The Foundation
- 11. Summary Perform explicit memory management instead of relying on Java objects • Reduce memory footprint • Eliminate garbage collection overheads • Use sun.misc.unsafe rows and off heap memory Code generation for expression evaluation • Reduce virtual function calls and interpretation overhead Cache conscious sorting • Reduce bad memory access patterns
- 13. Going back to the fundamentals Difficult to get order of magnitude performance speed ups with profiling techniques • For 10x improvement, would need of find top hotspots that add up to 90% and make them instantaneous • For 100x, 99% Instead, look bottom up, how fast should it run?
- 14. Scan Filter Project Aggregate select count(*) from store_sales where ss_item_sk = 1000
- 15. G. Graefe, Volcano— An Extensible and Parallel Query Evaluation System, In IEEE Transactions on Knowledge and Data Engineering 1994
- 16. Volcano Iterator Model Standard for 30 years: almost all databases do it Each operator is an “iterator” that consumes records from its input operator class Filter( child: Operator, predicate: (Row => Boolean)) extends Operator { def next(): Row = { var current = child.next() while (current == null ||predicate(current)) { current = child.next() } return current } }
- 17. What if we hire a college freshman to implement this query in Java in 10 mins? select count(*) from store_sales where ss_item_sk = 1000 long count = 0; for (ss_item_sk in store_sales) { if (ss_item_sk == 1000) { count += 1; } }
- 18. Volcano model 30+ years of database research college freshman hand-written code in 10 mins vs
- 19. Volcano 13.95 million rows/sec college freshman 125 million rows/sec Note: End-to-end, single thread, single column, and data originated in Parquet on disk High throughput
- 20. How does a student beat 30 years of research? Volcano Model 1. Too many virtual function calls 2. Intermediate data in memory (or L1/L2/L3 cache) 3. Can’t take advantage of modern CPU features -- no loop unrolling, SIMD, pipelining, prefetching, branch prediction etc. Hand-written code 1. No virtual function calls 2. Data in CPU registers 3. Compiler loop unrolling, SIMD, pipelining Take advantage of all the information that is known after query compilation
- 21. Whole-stage Codegen Fusing operators together so the generated code looks like hand optimized code: - Identify chains of operators (“stages”) - Compile each stage into a single function - Functionality of a general purpose execution engine; performance as if hand built system just to run your query
- 23. Scan Filter Project Aggregate long count = 0; for (ss_item_sk in store_sales) { if (ss_item_sk == 1000) { count += 1; } } Whole-stage Codegen: Spark as a “Compiler”
- 24. T Neumann, Efficiently compiling efficient query plans for modern hardware. In VLDB 2011
- 25. But there are things we can’t fuse Complicated I/O • CSV, Parquet, ORC, … • Sending across the network External integrations • Python, R, scikit-learn, TensorFlow, etc
- 26. Columnar in memory format mike In-memory Row Format 1 john 4.1 2 3.5 3 sally 6.4 1 2 3 john mike sally 4.1 3.5 6.4 In-memory Column Format
- 27. Why columnar? 1. More efficient: denser storage, regular data access, easier to index into 2. More compatible: Most high-performance external systems are already columnar (numpy, TensorFlow, Parquet); zero serialization/copy to work with them 3. Easier to extend: process encoded data
- 28. Parquet 11 million rows/sec Parquet vectorized 90 million rows/sec Note: End-to-end, single thread, single column, and data originated in Parquet on disk High throughput
- 29. Putting it All Together
- 30. Phase 1 Spark 1.4 - 1.6 Memory Management Code Generation Cache-aware Algorithms Phase 2 Spark 2.0+ Whole-stage Code Generation Columnar in Memory Support Both whole stage codegen [SPARK-12795] and the vectorized parquet reader [SPARK-12992] are enabled by default in Spark 2.0+ Demo
- 31. Operator Benchmarks: Cost/Row (ns) 5-30x Speedups
- 32. Operator Benchmarks: Cost/Row (ns) Radix Sort 10-100x Speedups
- 33. Operator Benchmarks: Cost/Row (ns) Shuffling still the bottleneck
- 34. Operator Benchmarks: Cost/Row (ns) 10x Speedup
- 35. TPC-DS (Scale Factor 1500, 100 cores) QueryTime Query # Spark 2.0 Spark 1.6 Lower is Better
- 36. What’s Next?
- 37. Spark 2.1, 2.2 and beyond 1. SPARK-16026: Cost Based Optimizer - Leverage table/column level statistics to optimize joins and aggregates - Statistics Collection Framework (Spark 2.1) - Cost Based Optimizer (Spark 2.2) 2. Boosting Spark’s Performance on Many-Core Machines - Qifan’s Talk Today at 2:55pm (Research Track) - In-memory/ single node shuffle 3. Improving quality of generated code and better integration with the in-memory column format in Spark
- 38. Questions? I’ll also be at the Databricks booth tomorrow from 12-2pm!