JVM Memory Management Details
12 likes3,622 views
The document provides a detailed overview of Java Virtual Machine (JVM) architecture, highlighting its components such as interpreters, JIT compilers, and memory management modules. It covers essential concepts of garbage collection, including its different types, methodologies, and tools for analyzing memory issues. Additionally, the document emphasizes the significance of concurrent garbage collection and its impact on scalability limitations in Java applications.
![Tools: jmap
Usage:
jmap [option] <pid>
(to connect to running process)
jmap [option] <executable <core>
(to connect to a core file)
jmap [option] [server_id@]<remote server IP or hostname>
(to connect to remote debug server)
where <option> is one of:
<none>
to print same info as Solaris pmap
-heap
to print java heap summary
-histo[:live]
to print histogram of java object heap; if the "live"
suboption is specified, only count live objects
-permstat
to print permanent generation statistics
-finalizerinfo
to print information on objects awaiting finalization
-dump:<dump-options> to dump java heap in hprof binary format
dump-options:
live
dump only live objects; if not specified,
all objects in the heap are dumped.
format=b binary format
file=<file> dump heap to <file>
Example: jmap -dump:live,format=b,file=heap.bin <pid>
-F
force. Use with -dump:<dump-options> <pid> or -histo
to force a heap dump or histogram when <pid> does not
respond. The "live" suboption is not supported
in this mode.
-h | -help
to print this help message
-J<flag>
to pass <flag> directly to the runtime system
©2011 Azul Systems, Inc.
33](https://image.slidesharecdn.com/jvmmemorymgt-131127080553-phpapp02/85/JVM-Memory-Management-Details-33-320.jpg)
JVM Memory Management Details
- 1. JVM: Memory Management Details Balaji Iyengar Senior Software Engineer, Azul Systems
- 2. Presenter • Balaji Iyengar ─ JVM Engineer at Azul Systems for the past 5+ years. ─ Currently a part-time PhD student. ─ Research in concurrent garbage collection. ©2011 Azul Systems, Inc. 2
- 3. Agenda • • • • • • What is a JVM? JVM Components JVM Concepts/Terminology Garbage Collection Basics Concurrent Garbage Collection Tools for analyzing memory issues ©2011 Azul Systems, Inc. 3
- 4. What is a Java Virtual Machine? • Abstraction is the driving principal in the Java Language specification ─ Bytecodes => processor instruction set ─ Java memory model => hardware memory model ─ Java threading model => OS threading model • Abstract the ‘underlying’ platform details and provide a standard development environment ©2011 Azul Systems, Inc. 4
- 5. What is a Java Virtual Machine? • Java Virtual Machine along with a set of tools implements the Java Language Specification ─ Bytecodes ─ Generated by the Javac compiler ─ Translated to the processor instruction set by the JVM ─ Java threads ─ Mapped to OS threads by the JVM ─ Java memory model ─ JVM inserts the right ‘memory barriers’ when needed ©2011 Azul Systems, Inc. 5
- 6. What is a Java Virtual Machine (JVM)? • Layer between platform and the application • Abstracts away operating system details • Abstracts away hardware Java Application C++ Application Operating System Operating System Hardware Hardware JVM architecture details • Key to Java’s ‘write once run anywhere’ capability. ©2011 Azul Systems, Inc. 6
- 7. Portability Compile once, run everywhere Same code! Java Application Java Application JVM JVM Operating System Operating System Hardware Architecture #1 ©2011 Azul Systems, Inc. Hardware Architecture #2 7
- 8. The JVM Components • An Interpreter ─ Straightforward translation from byte-codes to hardware instructions ─ One byte-code at a time ─ No optimizations, simple translation engine • JIT Compilers ─ Compiles byte-codes to hardware instructions ─ A lot more optimizations ─ Two different flavors targeting different optimizations ─ Client compiler for short running applications ─ Server compiler for long running applications ─ Server compiler generates more optimized code ©2011 Azul Systems, Inc. 8
- 9. The JVM Components • A Runtime environment ─ Implements a threading model Creates and manages Java threads Each thread maps to an OS thread ─ Implements synchronization primitives, i.e., locks ─ Implements dynamic class loading & unloading ─ Implements features such as Reflection ─ Implements support for tools ©2011 Azul Systems, Inc. 9
- 10. The JVM Components • Memory management module ─ Manages all of the program memory ─ Handles allocation requests ─ Recycles unused memory Free Memory Garbage Collection Unused Memory ©2011 Azul Systems, Inc. Allocation Memory In Use Program Activity 10
- 11. JVM Concepts/Terminology • Java Threads ─ Threads spawned by the application ─ Threads come and go during the life of a program ─ JVM allocates and cleanups resources on thread creation and death ─ Each thread has a stack and several thread-local data structures, i.e., execution context ─ Also referred to as ‘mutators’ since it mutates heap objects ©2011 Azul Systems, Inc. 11
- 12. JVM Concepts/Terminology • Java objects ─ ─ ─ ─ Java is an object oriented language Each allocation creates an object in memory The JVM adds meta-data to each object: “object-header” Object-header information useful for GC, synchronization, etc. • Object Reference ─ Pointer to a Java object ─ Present in thread-stacks, registers, other heap objects ─ Top bits in a reference can be used for meta-data ©2011 Azul Systems, Inc. 12
- 13. JVM Concepts/Terminology • Safepoints ─ The JVM has the ability to stop all Java threads ─ Used as a barrier mechanism between the JVM and the Java threads – ‘Safe’ place in code • Function calls • Backward branches ─ JVM has precise knowledge about mutator stacks/registers etc. at a safepoint. – Useful for GC purposes, e.g., STW GC happens at a safepoint. Safepoints reflect as application ‘pauses’ ©2011 Azul Systems, Inc. 13
- 14. Garbage Collection Taxonomy • • • • Has been around for over 40 years in academia For over 10 years in the enterprise Identifies ‘live’ memory and recycles the ‘dead’ memory Part of the memory management module in the JVM. ©2011 Azul Systems, Inc. 14
- 15. Garbage Collection Taxonomy • Several ways to skin this cat: – – – – – – – Stop-The-World vs. Concurrent Generational vs. Full Heap Mark vs. Reference counting Sweep vs. Compacting Real Time vs. Non Real Time Parallel vs. Single-threaded GC Dozens of mechanisms • Read-barriers • Write-barriers • Virtual memory tricks, etc.. ©2011 Azul Systems, Inc. 15
- 16. Garbage Collection Taxonomy • Stop-The-World GC ─ Recycles memory at safepoints only. • Concurrent GC ─ Recycles memory without stopping mutators • Generational GC ─ Divide the heap into smaller age-based regions ─ Empirically known that most garbage is found in ‘younger’ regions ─ Focus garbage collection work on ‘younger’ regions ©2011 Azul Systems, Inc. 16
- 17. Garbage Collection Basics • What is ‘live’ memory ─ Liveness == Accessibility ─ Objects that can be directly or transitively accessed by mutators ─ Objects with pointers in mutator execution contexts, i.e., ‘root- set’ ─ Objects that can be reached via the root-set ─ Implemented using ‘mark’ or by ‘reference counting’ • What is ‘dead’ memory ─ Everything that is not ‘live’ ©2011 Azul Systems, Inc. 17
- 18. Garbage Collection • How does the garbage collector identify ‘live’ memory ─ Starts from the root set of mutator threads ─ Does a depth-first or breadth-first walk of the object graph ─ ‘Marks’ each object that is found, i.e., sets a bit in a liveness bitmap ─ Referred to as the ‘mark-phase’ ─ Could use reference counting ─ Problems with cyclic garbage ─ Problems with fragmentation A D B C ©2011 Azul Systems, Inc. E 18
- 19. Garbage Collection Basics • How does GC recycle ‘dead’ memory Sweep: ─ Sweep ‘dead’ memory blocks into free-lists sorted by size ─ Hand out the right sized blocks to allocation requests ─ Pros: ─ Easy to do without stopping mutator threads ─ Cons ─ Slows down allocation path, reduces throughput ─ Can causes fragmentation ©2011 Azul Systems, Inc. 19
- 20. Garbage Collection Basics • How does GC recycle ‘dead’ memory Compaction: Copy ‘live’ memory blocks into contiguous memory locations Update pointers to old-locations Recycle the original memory locations of live objects Pros: ─ Supports higher allocation rates, i.e., higher throughputs ─ Gets rid of memory fragmentation ─ Cons: Concurrent versions are hard to get right ─ ─ ─ ─ ©2011 Azul Systems, Inc. 20
- 21. Garbage Collection • Desired Characteristics – – – – Concurrent Compacting Low application overhead Scalable to large heaps • These map best to current application characteristics • These map best to current multi-core hardware ©2011 Azul Systems, Inc. 21
- 22. Concurrent Garbage Collection • GC works in two phases ─ Mark Phase ─ Recycle Phase (Sweep/Compacting) • Either one or both phases can be concurrent with mutator threads • Different set of problems to implement the two phases concurrently • GC needs to synchronize with application threads ©2011 Azul Systems, Inc. 22
- 23. Concurrent Garbage Collection • Synchronization mechanisms between GC and mutators Read Barrier – Synchronization mechanism between GC and mutators – Implemented only in code executed by the mutator – Instruction or a set of instructions that follow a load of an object – – – – – reference JIT compiler spits out the ‘read-barrier’ Precedes ‘use’ of the loaded reference. Used to check GC invariants on the loaded reference Expensive because of the frequency of reads Functionality depends on the ‘algorithm’ ©2011 Azul Systems, Inc. 23
- 24. Concurrent Garbage Collection • Synchronization mechanisms between GC and mutators Write Barrier ─ ─ ─ ─ ─ ─ ─ Similar to read-barrier Implemented only in code executed by the mutator Instruction or a set of instructions that follow/precede a write JIT compiler spits out the ‘write-barrier’ Generally used to track pointer writes Cheaper, since writes are less common Functionality depends on the ‘algorithm’ ©2011 Azul Systems, Inc. 24
- 25. Concurrent Garbage Collection • Concurrent Mark ─ Scanning the heap graph while mutators are actively changing it ─ Multiple-readers, single-writer coherence problem ─ Mutators are the multiple writers ─ GC only needs to read the graph structure ©2011 Azul Systems, Inc. 25
- 26. Concurrent Garbage Collection • Concurrent Mark: What can go wrong? Mutator writes a pointer to a yet ‘unseen’ object into an object already ‘marked-through’ by GC • Can be caught by write barriers A • Can be caught by read barriers as well • Mutator write • GC considers object C ‘dead’. • Will recycle object C, causing a crash • Avoid by: • Marking object C ‘live’ OR • Re-traverse object A C B Unmarked Marked Marked-Through ©2011 Azul Systems, Inc. 26
- 27. Concurrent Garbage Collection • Concurrent Compaction: What can go wrong ─ Concurrent writes to old locations of objects can be lost A 4 5 6 A’ 0 0 0 Start Copy • • • • A 4 5 6 A’ 4 0 0 A 8 5 6 A’ 4 5 0 A 8 5 6 A’ 4 5 6 Mutator Write End Copy Timeline Object A is being copied to new location A’ A is the ‘From-Object’ ; A’ is the To-Object Mutator writes to ‘From-Object’ field after it has been copied Happens because mutator still holds a pointer to ‘From-Object’ Need to make sure that writes to object A, during and after the copy are reflected in the new location A’ ©2011 Azul Systems, Inc. 27
- 28. Concurrent Garbage Collection Concurrent Compaction: What can go wrong Propagating pointers to the old-location B B A C D A’ A Relocate A C D E During or after the object copy is done, the mutator writes a pointer to the old-location of the object in an object that is not known to the collector ©2011 Azul Systems, Inc. 28
- 29. Concurrent Garbage Collection • Propagating pointers to the old-location ─ Collector thinks object A has been copied to A’ ─ Recycles old-location A ─ Mutator attempts to access A via object E and crashes • Can be prevented by using ─ Read barriers, e.g., Azul’s C4 Collector ─ Compacting in ‘stop-the-world’ mode, e.g., CMS Collector ©2011 Azul Systems, Inc. 29
- 30. Biggest Java Scalability Limitation • For MOST JVMs, compaction pauses are the biggest current challenge and key limiting factor to Java scalability • The larger heap and live data / references to follow, the bigger challenge for compaction • Today: most JVMs limited to 3-4GB ─ ─ ─ ─ To keep “FullGC” pause times within SLAs Design limitations to make applications survive in 4GB chunks Horizontal scale out / clustering solutions In spite of machine memory increasing over the years… This is why I find Zing so interesting, as it has implemented concurrent compaction… ─ But that is not the topic of this presentation… ©2011 Azul Systems, Inc. 30
- 31. Tools: Memory Usage ©2011 Azul Systems, Inc. 31
- 32. Tools: Memory Usage Increasing ©2011 Azul Systems, Inc. 32
- 33. Tools: jmap Usage: jmap [option] <pid> (to connect to running process) jmap [option] <executable <core> (to connect to a core file) jmap [option] [server_id@]<remote server IP or hostname> (to connect to remote debug server) where <option> is one of: <none> to print same info as Solaris pmap -heap to print java heap summary -histo[:live] to print histogram of java object heap; if the "live" suboption is specified, only count live objects -permstat to print permanent generation statistics -finalizerinfo to print information on objects awaiting finalization -dump:<dump-options> to dump java heap in hprof binary format dump-options: live dump only live objects; if not specified, all objects in the heap are dumped. format=b binary format file=<file> dump heap to <file> Example: jmap -dump:live,format=b,file=heap.bin <pid> -F force. Use with -dump:<dump-options> <pid> or -histo to force a heap dump or histogram when <pid> does not respond. The "live" suboption is not supported in this mode. -h | -help to print this help message -J<flag> to pass <flag> directly to the runtime system ©2011 Azul Systems, Inc. 33
- 34. Tools: jmap Command to Collect /jdk6_23/bin/jmap -dump:live,file=SPECjbb2005_2_warehouses 15395 File sizes -rw-------. 1 me users 86659277 2011-06-15 15:23 SPECjbb2005_2_warehouses.hprof -rw-------. 1 me users 480108823 2011-06-15 15:25 SPECjbb2005_12_warehouses.hprof ©2011 Azul Systems, Inc. 34
- 35. Tools: JProfiler Memory Snapshot ©2011 Azul Systems, Inc. 35
- 36. Tools: JProfiler Objects (2 warehouses) ©2011 Azul Systems, Inc. 36
- 37. Tools: JProfiler Biggest Retained Sets ©2011 Azul Systems, Inc. 37
- 38. Tools: JProfiler Objects (12 warehouses) ©2011 Azul Systems, Inc. 38
- 39. Tools: JProfiler Biggest Retained Sets ©2011 Azul Systems, Inc. 39
- 40. Tools: JProfiler Difference Between 2/12 ©2011 Azul Systems, Inc. 40
- 41. Tools: madmap ©2011 Azul Systems, Inc. 41
- 42. GC and Tool Support • The Heap dump tools uses the GC interface ─ Walks the object graph using the same mechanism as GC ─ Writes out per-object data to a file that can later be analyzed. • GC also outputs detailed logs ─ These are very useful in identifying memory related bottle necks ─ Quite a few tools available to analyze GC logs ©2011 Azul Systems, Inc. 42
- 43. 2c for the Road What to (not) Think About 1. Why not use multiple threads, when you can? ─ Number of cores per server continues to grow… 2. Don’t be afraid of garbage, it is good! 3. I personally don’t like finalizers…error prone, not guaranteed to run (resource wasting) 4. Always be careful around locking ─ If it passes testing, hot locks can still block during production load 5. Benchmarks are often focused on throughput, but miss out on real GC impact – test your real application! ─ ─ “Full GC” never occurs during the run, not running long enough to see impact of fragmentation Response time std dev and outliers (99.9…%) are of importance for a real world app, not throughput alone!! ©2011 Azul Systems, Inc. 43
- 44. Summary • JVM – a great abstraction, provides convenient services so the Java programmer doesn’t have to deal with environment specific things • Compiler – “intelligent and context-aware translator” who helps speed up your application • Garbage Collector – simplifies memory management, different flavors for different needs • Compaction – an inevitable task, which impact grows with live size and data complexity for most JVMs, and the current largest limiter of Java Scalability ©2011 Azul Systems, Inc. 44
- 45. For the Curious: What is Zing? • Azul Systems has developed scalable Java platforms for 8+ years ─ Vega product line based on proprietary chip architecture, kernel enhancements, and JVM innovation ─ Zing product line based on x86 chip architecture, virtualization and kernel enhancements, and JVM innovation • Most famous for our Generational Pauseless Garbage Collector, which performs fully concurrent compaction ©2011 Azul Systems, Inc. 45
- 47. Additional Resources • For more information on… …JDK internals: http://openjdk.java.net/ (JVM source code) …Memory management: http://java.sun.com/j2se/reference/whitepapers/memorymanagement_w hitepaper.pdf (a bit old, but very comprehensive) …Tuning: http://download.oracle.com/docs/cd/E13150_01/jrockit_jvm/jrockit/genin fo/diagnos/tune_stable_perf.html (watch out for increased rigidity and re-tuning pain) …Generational Pauseless Garbage Collection: http://www.azulsystems.com/webinar/pauseless-gc (webinar by Gil Tene, 2011) …Compiler internals and optimizations: http://www.azulsystems.com/blogs/cliff (Dr Cliff Click’s blog) ©2011 Azul Systems, Inc. 47