Memory Management in Apache Spark
22 likes10,205 views
The document discusses memory management in Apache Spark, highlighting its critical importance for performance. It outlines three main challenges: memory arbitration between execution and storage, across parallel tasks, and between operators within a task, along with solutions for each. The presentation provides insights into unified memory management introduced in Spark 1.6 and various optimization techniques for efficient memory use.
![Scan
Project
Aggregate
Sort
Map { // age → heights
20 → [154, 174, 175]
21 → [167, 168, 181]
22 → [155, 166, 188]
23 → [160, 168, 178, 183]
}](https://image.slidesharecdn.com/apache-spark-memory-dbtb-2016-160520191452/85/Memory-Management-in-Apache-Spark-43-320.jpg)
Memory Management in Apache Spark
- 1. Deep Dive: Memory Management in Apache Andrew Or May 18th, 2016 @andrewor14
- 2. How familiar are you with Apache Spark? a) I contribute to it b) I use it in production c) I am evaluating it d) I have nothing to do with it 2
- 3. What is Apache ? 3 Fast and general engine for big data processing Fast to run code – In-memory data sharing – General computation graphs Fast to write code – Rich APIs in Java, Scala, Python – Interactive shell
- 4. 4 Spark Core Spark Streaming real-time Spark SQL structured data MLlib machine learning GraphX graph … What is Apache ?
- 5. About Databricks 5 Team that created Spark at UC Berkeley Offer a hosted service – Spark in the cloud – Notebooks – Plot visualizations – Cluster management
- 6. About Me 6 Apache Spark committer Software eng @ Databricks Hadoop Summit ‘15 Spark Summit Europe ‘15 Some other meetup talks
- 7. 7 Efficient memory use is critical to good performance
- 9. Memory contention poses three challenges for Apache Spark 9 How to arbitrate memory between execution and storage? How to arbitrate memory across tasks running in parallel? How to arbitrate memory across operators running within the same task?
- 10. Two usages of memory in Apache Spark 10 Execution Memory used for shuffles, joins, sorts and aggregations Storage Memory used to cache data that will be reused later
- 11. Iterator 4, 3, 5, 1, 6, 2, 8 4 3 5 1 6 2 8 Sort
- 12. 4 3 5 1 6 2 8 Sort Execution memory
- 13. Iterator 1, 2, 3, 4, 5, 6, 8 1 2 3 4 5 6 8 Sort Execution memory
- 14. Iterator Map Iterator 1, 2, 3, 4, 5, 6, 8 +1 2, 3, 4, 5, 6, 7, 9
- 15. Iterator Map 1, 2, 3, 4, 5, 6, 8 +1 Map +1 Iterator 1, 2, 3, 4, 5, 6, 8 ... Iterator 2, 3, 4, 5, 6, 7, 9 Iterator 2, 3, 4, 5, 6, 7, 9
- 16. Cached Iterator 1, 2, 3, 4, 5, 6, 8 Map Iterator Map Iterator ... Storage memory Map +1 Iterator 2, 3, 4, 5, 6, 7, 9
- 17. Challenge #1 How to arbitrate memory between execution and storage?
- 18. Easy, static allocation! 18 Total available memory Execution Storage Spark 1.0 May 2014
- 19. Easy, static allocation! 19 Execution Storage Spill to disk Spark 1.0 May 2014
- 20. Easy, static allocation! 20 Execution Storage Spark 1.0 May 2014
- 21. Easy, static allocation! 21 Execution Storage Evict LRU block to disk Spark 1.0 May 2014
- 22. 22 Inefficient memory use means bad performance
- 23. Easy, static allocation! 23 Execution can only use a fraction of the memory, even when there is no storage! Execution Storage Spark 1.0May 2014
- 24. Storage Easy, static allocation! 24 Efficient use of memory required user tuning Execution Spark 1.0May 2014
- 25. 25 Fast forward to 2016… How could we have done better?
- 27. 27 Execution Storage Unified memory management Spark 1.6+ Jan 2016 What happens if there is already storage?
- 28. 28 Execution Storage Unified memory management Spark 1.6+ Jan 2016 Evict LRU block to disk
- 29. 29 Execution Storage Unified memory management Spark 1.6+ Jan 2016 What about the other way round?
- 30. 30 Execution Storage Unified memory management Spark 1.6+ Jan 2016 Evict LRU block to disk
- 31. Design considerations 31 Why evict storage, not execution? Spilled execution data will always be read back from disk, whereas cached data may not. What if the application relies on caching? Allow the user to specify a minimum unevictable amount of cached data (not a reservation!). Spark 1.6+ Jan 2016
- 32. Challenge #2 How to arbitrate memory across tasks running in parallel?
- 33. Easy, static allocation! Worker machine has 4 cores Each task gets 1/4 of the total memory Slot 1 Slot 2 Slot 3 Slot 4
- 34. Alternative: What Spark does Worker machine has 4 cores The share of each task depends on number of actively running tasks (N) Task 1
- 35. Alternative: What Spark does Now, another task comes along so the first task will have to spill Task 1
- 36. Alternative: What Spark does Each task is assigned 1/N of the memory, where N = 2 Task 1 Task 2
- 37. Alternative: What Spark does Each task is assigned 1/N of the memory, where N = 4 Task 1 Task 2 Task 3 Task 4
- 38. Alternative: What Spark does Last remaining task gets all the memory because N = 1 Task 3 Spark 1.0+ May 2014
- 39. Static allocation vs What Spark does 39 Both are fair and starvation free Static allocation is simpler What Spark does handles stragglers better
- 40. Challenge #3 How to arbitrate memory across operators running within the same task?
- 41. SELECT age, avg(height) FROM students GROUP BY age ORDER BY avg(height) students.groupBy("age") .avg("height") .orderBy("avg(height)") .collect() Scan Project Aggregate Sort
- 42. Worker has 6 pages of memory Scan Project Aggregate Sort
- 43. Scan Project Aggregate Sort Map { // age → heights 20 → [154, 174, 175] 21 → [167, 168, 181] 22 → [155, 166, 188] 23 → [160, 168, 178, 183] }
- 44. Scan Project Aggregate Sort All 6 pages were used by Aggregate, leaving no memory for Sort!
- 45. Solution #1: Reserve a page for each operator Scan Project Aggregate Sort
- 46. Solution #1: Reserve a page for each operator Scan Project Aggregate Sort Starvation free, but still not fair… What if there were more operators?
- 49. Scan Project Aggregate Sort Solution #2: Cooperative spilling Sort forces Aggregate to spill a page to free memory
- 50. Scan Project Aggregate Sort Solution #2: Cooperative spilling Sort needs more memory so it forces Aggregate to spill another page (and so on)
- 51. Scan Project Aggregate Sort Solution #2: Cooperative spilling Sort finishes with 3 pages Aggregate does not have to spill its remaining pages Spark 1.6+ Jan 2016
- 52. Recap: Three sources of contention 52 How to arbitrate memory … ● between execution and storage? ● across tasks running in parallel? ● across operators running within the same task? Instead of avoid statically reserving memory in advance, deal with memory contention when it arises by forcing members to spill
- 53. Project Tungsten 53 Binary in-memory data representation Cache-aware computation Code generation (next time) Spark 1.4+ Jun 2015
- 54. “abcd” 54 • Native: 4 bytes with UTF-8 encoding • Java: 48 bytes – 12 byte header – 2 bytes per character (UTF-16 internal representation) – 20 bytes of additional overhead – 8 byte hash code Java objects have large overheads
- 55. 55 Schema: (Int, String, string) Row Array String(“data”) String(“bricks”) 5+ objects, high space overhead, expensive hashCode() BoxedInteger(123) Java objects based row format
- 56. 6 “bricks” 56 0x0 123 32L 48L 4 “data” (123, “data”, “bricks”) Null tracking bitmap Offset to var. length data Offset to var. length data Tungsten row format
- 57. Cache-aware Computation 57 ptr key rec ptr key rec ptr key rec Naive layout Poor cache locality ptrkey prefix rec ptrkey prefix rec ptrkey prefix rec Cache-aware layout Good cache locality E.g. sorting a list of records
- 58. For more info... Deep Dive into Project Tungsten: Bringing Spark Closer to Bare Metal https://www.youtube.com/watch?v=5ajs8EIPWGI Spark Performance: What’s Next https://www.youtube.com/watch?v=JX0CdOTWYX4 Unified Memory Management https://issues.apache.org/jira/browse/SPARK-10000