Understanding JVM GC: advanced!

Understanding JVM GC
Jean-Philippe Bempel
@jpbempel

Agenda
GC basics
G1
Shenandoah
Azul C4
Z GC

Marking for Minor GC
Traversing references to mark lived objects
Stops when reaching old gen
From GC Roots (static fields, thread stacks, JNI)
Young Old

Card Table
Young
Card scanning for young-to-old refs
1 1 0
Write barrier to update card table
mov QWORD PTR [rsi+0x20],rdx
shr rsi,0x9
mov rdi,0x7f2d42817000
mov BYTE PTR [rsi+rdi*1],0x0

Garbage First
Region based
Pause time target (soft real-time)
-XX:MaxGCPauseMillis=n (default 200)
Middle ground throughput/latency
Default GC since JDK9

Regions
Heap divided into fixed-size regions

Garbage First
Credit: Kirk Pepperdine

G1 phases
Young Collection (STW)
Initial Mark (STW)
Concurrent Marking
Final Remark (STW)
Cleanup (STW/Concurrent)
Mixed Collection (STW)

Young Collection
Stop-The-World Event
Evacuates lived objects to survivor or Old regions
Only objects in young generation are considered

Remembered Sets
Card table per region
Avoid scanning the entire heap to update reference

Remembered Sets: Post Write Barrier
For each reference assignment (X.f = Y) need to check:
● References (X & Y) are not in the same region
● Y is not null
● => enqueue for RS processing
Refinement Threads to process the queue
Additional instructions added after assignment
if (!isInSameRegion(X, Y)
&& Y != null)
RSEnqueue(X)
mov DWORD PTR [rbp+0x74],r10d
mov r11,rbp
mov r8,r10
shl r8,0x3
xor r8,r11
shr r8,0x14
test r8,r8
je cont
test r10d,r10d
je cont
shr r11,0x9
movabs rcx,0x2965ecc3000
add rcx,r11
cmp BYTE PTR [rcx],0x20
je cont
mov r10,QWORD PTR [r15+0x70]
lock add DWORD PTR [rsp-0x40],0x0
cmp BYTE PTR [rcx],0x0
je cont
mov BYTE PTR [rcx],0x0
test r10,r10
jne 0x000002965edc62bc
mov rdx,r15
movabs r10,0x7ffac2febc30
call r10
jmp cont
mov QWORD PTR [r11+r10*1-0x8],rcx
add r10,0xfffffffffffffff8
mov QWORD PTR [r15+0x70],r10

Concurrent Marking
Triggered based on Initiating Heap Occupancy Percent flag (45%)
Try to mark the whole object graph concurrently with application
running
Based on tri-color abstraction & Snapshot-at-the-beginning algorithm

Concurrent Marking: Tri-Color abstraction
GC root

Concurrent Marking: Issues
New allocations during mark phase can be handled in 2 ways:
● Marking automatically object at allocation
● Not considering new allocations for the current cycle
Tri-Color abstraction provides 2 properties of missed objects:
1. The mutator stores a reference to a white object into a black object
2. All paths from any gray objects to a white objects are destroyed

Concurrent Marking: Issues
A
B
C
A.field1 = C;
B.field2 = null;
OOPS!

Concurrent Marking: Resolving misses
2 ways to ensure not missing any marking
For SATB, pre-write barriers, recording object
for marking
SATB barrier is active only when Marking is on
(global state)
if (SATB_WriteBarrier) {
if (X.f != null)
SATB_enqueue(X.f);
}
cmp BYTE PTR [r15+0x30],0x0
jne 0x000002965edc62e5
[...]
mov r11d,DWORD PTR [rbp+0x74]
test r11d,r11d
je 0x000002965edc6253
mov rcx,r11
shl rcx,0x3
test r10,r10
je 0x000002965edc6318
mov QWORD PTR
[r11+r10*1-0x8],rcx
add r10,0xfffffffffffffff8
mov QWORD PTR [r15+0x38],r10
jmp 0x000002965edc6253
mov rdx,r15
movabs r10,0x7ffac2febc50
call r10
jmp 0x000002965edc6253

CollectionSet
At the end of Marking, we have per region liveness information
Regions are sorted by liveness (ascending, so garbage first!)
Regions full of garbage are collected during concurrent cleanup
CollectionSet used during Mixed event is built based on:
● Liveness, up until threshold (G1HeapWastePercent and
G1MixedGCLiveThresholdPercent )
● Maximum number of regions (G1OldCSetRegionThresholdPercent )

Mixed GC
Based on CollectionSet, G1 schedules to collect part of old regions
When Young is triggered, old regions to collect are piggy-backed
Some part of old regions are considered for each GC based on cost
model to reach the pause goal
Several young GCs can be used to collect old regions (mixed event)

Humoungous
If an object is more than half of a region, it is humongous
Humongous objects are too expensive to move or relocate
Too many Humongous alloc leads to early triggering of GC
Avoid having too much humongous allocations

Full GC
Fallback to FullGC (serial < JDK10)
Fragmentation can still happen (regions with lots of lived objects)
Unpredictable

Shenandoah GC
Non-generational (still option for partial collection)
Region based
Use Read Barrier: Brooks pointer
Self-Healing:
● Cooperation between mutator thread & GC threads
● Only for concurrent compaction
Mostly based on G1

Shenandoah Phases
Initial Marking (STW)
Concurrent Marking
Final Remark (STW)
Concurrent Cleanup
Concurrent Evacuation
Init Update References (STW)
Concurrent Update References
Final Update References (STW)
Concurrent Cleanup

Concurrent Marking
SATB-style (like G1)
2 STW pauses for Initial Mark & Final Remark
Conditional Write Barrier
To deal with concurrent modification of object graph

Concurrent Evacuation
Same principle than G1:
Build CollectionSet with Garbage First!
Evacuate to new regions to release the region for reuse
Concurrent Evacuation done with the help of:
1 Read Barrier : Brooks pointer
4 Write Barriers
Barriers help to keep the to-space invariant:
All Writes are made into an object in to-space

Brooks pointers
All objects have an additional forwarding pointer
Placed before the regular object
mov r13,QWORD PTR [r12+r14*8-0x8]
Header
Brooks pointer
Dereference the forwarding pointer for each access
Memory footprint overhead
Throughput overhead

Concurrent copy: GC thread
Header
Brooks pointer
Header
Brooks pointer
From-Space To-Space
GC thread

Concurrent copy: Reader thread
Header
Brooks pointer
From-Space To-Space
Reader
thread
Reader
thread

Concurrent copy: Writer thread
Header
Brooks pointer
From-Space To-Space
Writer
thread
Writer
thread
Header
Brooks pointer
Header
Brooks pointer

Write Barriers
Any writes (even primitives) to from-space object needs to be
protected
if (evacInProgress
&& inCollectionSet(obj)
&& notCopyYet(obj)) {
evacuateObject(obj)
}
test BYTE PTR [r15+0x3c0],0x2
jne 0x000000000281bcbc
[...]
mov r10d,DWORD PTR [r13+0xc]
test r10d,r10d
je 0x000000000281bc2b
mov rcx,r10
shl rcx,0x3
test r11,r11
je 0x000000000281bd0d
[...]
mov rdx,r15
movabs r10,0x62d1f660
call r10
jmp 0x000000000281bc2b
Exotic barriers:
acmp (pointer comparison)
CAS
clone

Extreme cases
Late memory release
Only happens when all refs updated (Concurrent Cleanup
phase)
Allocations can overrun the GC
Failure modes:
Pacing
Degenerated GC
FullGC

Shenandoah 2.0?
Since JDK13, notable changes introduced
Load Reference Barrier
Baker-style barrier (Relocation)
Evaluated at reference load time
Eliminating forward pointer word
Store forward information into Mark word
Remove Brooks pointer

Continuously Concurrent Compacting Collector
Generational (young & old)
Region based (pages)
Use Read Barrier: Loaded Value Barrier
Self-Healing
Cooperation between mutator threads & GC threads
Pauseless algorithm but implementation requires safepoints
Pauses are most of the time < 1ms

LVB
Baker-style based Barrier
move objects through forwarding addresses stored aside
Applied at load time, not when dereferencing
Fused marking & relocation state
Ensure C4 invariants:
Marked Through the current cycle
Not relocated
If not => Self-healing process to correct it
Mark object
Relocate & correct reference
Checked for each reference loads
Benefits from JIT optimization for caching loaded value (registers)

LVB
States of objects stored inside reference address => Colored
pointers
NMT bit
Remapped/Generation
Checked against a global expected value during the GC cycle
Thread local, almost always L1 cache hits
Register
Unmasking reference addresses:
Linux Kernel module, aliasing
memfd, multi-mapping
test r9, rax
jne 0x3001443b
mov r10d, dword ptr [rax + 8]

Virtual Memory vs Physical Memory
Virtual Memory
Physical Memory
0 2^64
0 2^37

C4 Phases
Mark
Marking all objects in graph
Relocation
Moving objects to release pages
Remap
Fixup references in object graph
Folded with next mark cycle

Mark Phase
Precise Wavefront Marking
Single pass
No final mark/remark
Self-Healing: Mark object that are not marked for the current
cycle

Mark Phase: Concurrent Modfication
A
B
C
A.field1 = C;
B.field2 = null;
LVB

Relocation Phase
Select pages with the greatest number of dead objects (garbage first!)
Protect page selected from being accessed by mutators thread
Move objects to new allocated pages
Build side arrays (off heap tables) for forwarding information
Self-Healing: As protected, LVB will trigger a trap to:
Copy object to the new location if not done
Use forward pointer to fix the reference

Virtual
Physical
Relocation Phase
Forwarding table

Remap Phase
Traverse Object Graph and fixup references
Execute LVB barrier for each object
Self-Healing: fixup references using forward information
As we traverse again, mark for the next phase
Mark & Remap phases are folded!

Remap - Kernel module
Algorithm requires a sustainable rate or remapping operations
Linux limitations:
TLB invalidation
Only 4KB pages can be remapped
Single threaded remapping (write lock)
Kernel module implements API for the Zing JVM to increase significantly the
remapping rate
Implements also virtual address aliasing for addressing objects with metadata

C4 in real life scenario
Ullink:
Low latency order router (100us)
Order Management System (512GB heap)
Criteo:
Name Node on a 2000+ nodes Hadoop cluster
580GB heap
Hard to tune with G1 (JDK8)
Once in prod no issue in 2 years

Z GC
Non generational
Region based (zPages, dynamically sized)
Concurrent Marking, Compaction, Ref processing
Use Colored Pointers & Read/Load Barrier
Self-Healing
Cooperation between mutator threads & GC threads
Experimental in JDK 11 (-XX:+UnlockExperimentalVMOptions
–XX:+UseZGC)
mov r10,QWORD PTR [r11+0xb0]
test QWORD PTR [r15+0x20],r10
jne 0x00007f9594cc54b5

Z GC phases:
Initial Mark (STW)
Concurrent Mark/Remap
Final Mark (STW)
Concurrent Prepare for Relocation
Start Relocate (STW)
Concurrent Relocate

Colored Pointers
Store metadata in unused bits of reference address
42 bits for addressing (4TB)
44 bits (16TB) for JDK 13
4 bits for metadata
Marked0
Marked1
Remapped
Finalizable

Multi-Mapping
Colored pointers needs to be unmasked for dereferencing
Some HW support masking (SPARC, Aarch64))
On linux/windows, overhead if done with classical
instructions
Only one view is active at any point
Plenty of Virtual Space

Multi-Mapping
Virtual Memory
Physical Memory
0 2^64
0 2^37
(marked0)
001<address>
(marked1)
010<address>
(remapped)
100<address>

Difference between C4 & Z GC
Unmasking ref addresses
C4: Kernel module aliasing
Z: Multi-mapping or HW support
Pages & Relocation
C4:
Page are fixed (2MB)
relocation for large objects by remapping
Z:
zPages are dynamic, a zPage can be 100MB large
No relocation for large objects

Low Latency GCs
You have to run on Windows
Shenandoah
Battlefield tested GC (maturity)
C4
Shenandoah
Minimizing any kind of JVM pauses
C4
Z
You don’t want pay for it:
Shenandoah
Z

References GC basics
Java Garbage Collection distilled by Martin Thompson
Java GC minibook
Oracle’s white paper on JVM memory management & GC
What differences JVM makes by Nitsan Wakart
Memory Management Reference

References G1
Garbage-First Garbage Collection
G1 One Garbage Collector to rule them all by Monica Beckwith
Tips for Tuning The G1 GC by Monica Beckwith
G1 Garbage Collector Details and Tuning by Simone Bordet
Write Barriers in Garbage-First Garbage Collector by Monica Beckwith

References Shenandoah
Shenandoah: An open-source concurrent compacting garbage collector
for OpenJDK
Shenandoah: The Garbage Collector That Could by Aleksey Shipilev
Shenandoah GC Wiki
Load Reference Barriers by Roman Kennke
Eliminating forward pointer word by Roman Kennke

References C4
The Pauseless GC algorithm (2005)
C4: Continuously Concurrent Compacting Collector (2011)
Azul GC in Detail by Charles Humble
2010 version source code

References Z GC
ZGC - Low Latency GC for OpenJDK by Per Liden
Java's new Z Garbage Collector (ZGC) is very exciting by Richard
Warburton
A first look into ZGC by Dominik Inführ
Architectural Comparison with C4/Pauseless
ZGC Heap Size and RSS counters

Thank You!
Jean-Philippe Bempel
@jpbempel

Understanding JVM GC: advanced!

More Related Content

What's hot (20)

Similar to Understanding JVM GC: advanced! (20)

More from Jean-Philippe BEMPEL (14)

Recently uploaded (20)

Understanding JVM GC: advanced!