JIT vs. AOT: Unity And Conflict of Dynamic and Static Compilers

JIT vs. AOT: Unity And Conflict of
Dynamic and Static Compilers
for Java
Nikita Lipsky
Excelsior LLC

Once upon a time,
when C++ was new and cool,
+++++++++
4

Once upon a time,
+++++++++
5

Once upon a time,
all compilers were static
6

Until into this world came…
Java
7

Until into this world came…
Java

Two Ways To Execute Java Bytecode
• Interpret
Slow but portable
• Compile to machine code
Runs on the actual hardware
9

When to compile to native code?
• At application run time
Dynamic (Just-In-Time, JIT) Compilation
• Before the application is launched/deployed
Static (Ahead-Of-Time, AOT) Compilation
10

JIT vs. AOT: Unity And Conflict of Dynamic and Static Compilers

JIT vs. AOT: Agenda
• Myths around Java AOT
• Java AOT challenges and advantages
• Performance:
– JIT vs. AOT compiler optimizations comparison
• JIT and AOT in various fields of use:
– client-side, server-side, embedded, IOT, mobile
12

Nikita Lipsky
• 20+ years in software development
• Excelsior JET project initiator
– 16+ years of contributions
– compiler engineer
– team lead
– product lead
– etc.
• Twitter: @pjBooms
13

Who needs Java AOT
15
Кто знает про Excelsior JET?Кто знает про Excelsior JET?

Myth 1. Java is “too dynamic”
Reflection Dynamic class loading
17

Myth 2. AOT kills WORA
WORA
Write Once
Run Anywhere
BORA
Build Once
Run Anywhere
18
!=

Myth 3. AOT = Small EXE
”I would get a small executable that works
without a JVM (as if I wrote my app in C)”
19

Myth 3. AOT = Small EXE
• What about thousands of standard classes?
• Who will collect the garbage?
• What about reflection?
20

AOT = Smaller Package
Yes, AOT compilation can help you create
a JRE-independent installer smaller than the JRE
installer alone
Javа Runtime Slim-Down:
www.excelsiorjet.com/solutions/java-download-size
21

Non-standard Class Loaders
• Override the default reference resolution logic
• Unique namespaces
• Dependency management frameworks
– OSGi: Felix, Equanox
Solve JAR hell problem
• Java EE servers, Eclipse RCP, plugin architectures
23

How to compile such classes statically?
• Compile each class in isolation
– Bad for performance
• Reproduce reference resolution logic of popular
class loaders
– Does not work for arbitrary class loaders
24

25
Classes
Classe
sClasses

Classes
CL1
CL2
CL3
CL4
26CLi: class loader

• CL4• CL3
• CL2• CL1
classes classes
classesclasses
27

• CL4• CL3
• CL2• CL1
classes
classes
classesclasses
28

• CL4• CL3
• CL2• CL1
classes classes
classesclasses
29

30
• CL4• CL3
• CL2• CL1
classes classes
classes

• CL4• CL3
• CL2• CL1
classes classes
31

AOT class loader support scheme:
• Reference resolver in the AOT compiler:
– Determines which class loaders will be created at
run time and assigns a static ID to each
– Breaks down the set of classes by class loader IDs
– Resolves references between classes
32

AOT class loader support scheme contd.:
• At application run time:
– For each class loader instance, compute its ID
– Knowing the ID, “load” the precompiled classes:
• Create a java.lang.Class instance
• Fill it with reflection information
• Let the O/S load code from the executable
33

AOT class loader support scheme contd.:
• At application run time:
– For each class loader instance, compute its ID
– Knowing the ID, “load” the precompiled classes
Known to work for Eclipse RCP
and Tomcat Web apps
34

Protect Code from Decompilers
36

Application Code Protection
• Bytecode emitted by javac is extremely easy to
decompile almost to original source
Proof: http://bytecodeviewer.com/
• Reflection makes name obfuscation labor-
consuming and error prone, hinders code
maintenance
• Possible to guess what obfuscated code does
References to JDK classes remain intact
37

Application Code Protection
• Native machine code can only be effectively
disassembled, but not decompiled
• Application structure not deductible
from disassembler output
• Aggressively optimized native code only
remotely resembles original
38

Cold Start vs Warm Start
Why a re-started application starts much faster?
– No need to load app code and data from disk
– No need to load system/third-party dynamic
libraries (.DLL, .so) either
40

AOT Is Faster?
• Native code is “thicker” than Java bytecode
• Reading code from disk often takes more
time than its execution, esp. on cold starts
41

AOT IS Faster!
1. Identify code that gets executed on startup
2. Place it at the beginning of the executable
3. Preload the entire “startup segment” in one
sequential read (Makes most difference on rotating media,
slow USB sticks, etc., not so much on SSDs)
42

Myth 4. AOT Is Faster
“By compiling Java statically, we are making it
equivalent to C, C is faster than Java,
therefore statically compiled Java is faster”
46

Myth 5. JIT Is Faster
“Effective optimization of a Java application
is only possible in the presence
of its dynamic execution profile”
48

Compiler Optimizations
– Constant propagation
– Dead code elimination
– Common subexpression
elimination
– Inline substitution
– Method specialization
– Loop unrolling
– Loop versioning
– Loop-invariant code motion
– Tail recursion elimination
– Call devirtualization
– On-stack allocation and
explosion of objects
– Runtime checks removal
– Excess synchronization removal
– Optimal code selection and
bundling
– Instruction scheduling
– Optimal register allocation
– etc.
49

Java == OOP
• Lots of methods
• Lots of small methods (get/set)
• Lots of virtual calls of small methods
50

Call Devirtualization
• Precondition for subsequent inline substitution
• Class hierarchy analysis
Method not overridden => non-virtual call
• Type analysis
new T().foo(); // Non-virtual
// call of T.foo()
• Profile guided
51

A a;
…
a.foo(); 
if (RT_Type(a) in CHA) {
inlined body of foo()
} else {
a.foo();
}
Idea: method not overriden => non-virtual call
Class Hierarchy Analysis (CHA)
52

Type Analysis
Idea:
new A().foo(); // non-virtual call. Always!
53

Type Analysis
A a = b ? new B() : new C();
a.foo();// is it virtual call?
// B, C extend A and do not override A.foo
54

Type Analysis
A a = b ? new B() : new C();
a.foo();
If neither B nor С override A.foo(),
then a.foo()is a non-virtual call again!
(can be inlined)
55

Type Analysis
A a = b ? bar() : baz();
…
a.foo();
If bar() only returns results of new B,
and baz() - only returns results of new С,
then a.foo() is again non-virtual
(can be inlined)
56

Type Analysis
How do we know
what bar() and baz() return?
57

Global Analysis
• Analyses all methods of the program, computing
useful information about each
– whether a method always returns new T();
– whether an argument of a method does not escape to
a shared memory
• Results are then used for optimization of each
particular method
58

Stack Allocation of Objects
• All Java objects are supposed to reside in dynamic
memory – on the Java heap
• But, most objects are small and temporary
• It is desirable to allocate them on the stack
Escape analysis determines whether a locally
created object escapes from the method
stack frame into shared memory
59

Example
for (Object o: getCollection()) {
doSomething(o);
}
60

Example
Iterator iter = getCollection().iterator();
while (iter.hasNext()) {
Object o = iter.next();
doSomething(o);
}
61

Example
Suppose analysis has shown that getCollection()
always returns an ArrayList
ArrayList list = getCollection();
Iterator iter = list.iterator();
doSomething(o);
}
62

Example
ArrayList.ListItr iter = new ListItr(list);
doSomething(o);
}
63

Example
ArrayList.ListItr iter = onStack ListItr();
iter.this$0 = list;
iter.cursor = 0;
iter.size = list.elemData.length;
doSomething(o);
}
64

Example
ArrayList.ListItr iter = onStack ListItr(list);
iter.this$0 = list;
iter.cursor = 0;
iter.size = list.elemData.length;
while (iter.cursor < iter.size) {
int index = iter.cursor++;
Object o = iter.this$0.elemData[index];
doSomething(o);
}
65

Example
int cursor = 0;
int size = list.elemData.length;
while (cursor < size) {
Object o = list.elemData[cursor++];
doSomething(o);
}
66

Example
int size = list.elemData.length;
for (int i = 0; i < size; i++) {
doSomething(list.elemData[i]);
}
67

Analysis & Optimizations…
• …are often quite complicated
• …require iterative re-computation
• …, if global, depend on the entire program
68

• …require iterative recomputation
Can a JIT compiler afford
all or any of that?

• …require iterative recomputations
all or any of that?
70

• …require iterative recomputations
all or any of that?

Dynamic Optimizations
• Profiling and selective compilation
• Inline substitution based on execution profile
• Hot execution traces optimization
• Optimal instruction selection

Hot Code vs Warm Code
Q: What happens when an app with no
distinctly hot code runs on a typical JVM?
A: Long warmup with results not stored
for future reuse
Typical case: UI-centric applications
73

JFCMark (Short Run)
0%
50%
100%
150%
200%
250%
2 core Celeron, 2.60Ghz 4 core i5, 3.8GZ
Excelsior JET
HotSpot client
HotSpot server
Bigger – better
74

JFCMark (Long Run)
0%
20%
40%
60%
80%
100%
120%
140%
160%
2 core Celeron, 2.60Ghz 4 core i5, 3.8GZ
Excelsior JET
HotSpot client
HotSpot server
75
Bigger – better

Profile Guided Optimizations
Can a static compiler use
dynamic execution profiles as input?
76

Server side
• CPU time in the cloud costs
money
• After some time, execution profile of a server app
stabilizes
• Why not pass it over to the AOT compiler?
77

AOT on the Server side
• Stable performance and predictable latency
– no code de-optimizations occur at run-time
• Work at full speed right from the start
– good for load balancing
• Better startup time
– good when many servers start simultaneously
• Still protects the code from decompilation
78

Embedded/IoT
• The less powerful the hardware, the more
expensive dynamic compilation becomes
• Embedded systems typically have less
computing power than desktops and servers
79

Mobile
• JVM on Mobile:
– Platform classes
– Memory Manager and GC
– Reflection
– JIT Compiler
What is missing?
80

Mobile
• JVM on Mobile:
– Reflection
– JIT Compiler
What is missing?
81

Mobile
• JVM on Mobile:
– Reflection
– JIT Compiler
What is missing?
82

Mobile
• JVM on Mobile:
– Reflection
– JIT Compiler
What is missing?
83

Mobile
• JVM on Mobile:
– Reflection
– JIT Compiler
What is missing?
84

Mobile
• JVM on Mobile:
– Reflection
– JIT Compiler
What is missing?
85

Mobile
• JVM on Mobile:
– Reflection
– JIT Compiler
Charger!
86

Mobile
• Wireless devices have batteries
• Does it make sense to spend power on
dynamic compilation?
87

iOS
• iOS policy prohibits creation of any native
code during application run time
• Getting Java apps to work on iOS requires
either an interpreting JVM or an AOT compiler
88

AOT Java Compilers in 2000
Desktop/Server:
BulletTrain
Excelsior JET
GNU Compiler for Java (GCJ)
IBM VisualAge for Java
Supercede/JOVE
TowerJ
Embedded/Mobile:
Diab FastJ
Esmertec Jbed ME
GCJ
IBM J9
Sun Java ME
(custom offerings)
89

Desktop/Server:
Excelsior JET
GCJ (RIP)
IBM Java SDK for AIX
IBM Java SDK for z/OS
Coming soon:
HotSpot AOT (JEP-295)
Embedded/Mobile:
Excelsior JET Embedded
IBM WebSphere Real Time
Oracle Java ME Embedded Client
Codename One
Avian
Android ART
RoboVM (RIP)
Multi-OS Engine
90

Desktop/Server:
Excelsior JET
GCJ (RIP)
IBM Java SDK for AIX
IBM Java SDK for z/OS
Coming soon:
HotSpot AOT (JEP-295)
Embedded/Mobile:
Excelsior JET Embedded
IBM WebSphere Real Time
Oracle Java ME Embedded Client
Codename One
Avian
Android ART
RoboVM (RIP)
Multi-OS Engine
91

Static compilation of Java apps:
92
Conclusion

93
Possible
Conclusion

94
Possible
Preserving all Java features
Conclusion

95
Possible
Preserving all Java features
Useful in many aspects
Conclusion

Conclusion
• JIT and AOT have their unique advantages
• Synthesis of two approaches will give
maximum benefits
96

Conclusion
• JIT and AOT have their unique advantages
• Synthesis of the two approaches will bring
maximum benefits
97

Q & A
Nikita Lipsky,
Excelsior
nlipsky@excelsior-usa.com
twitter: @pjBooms 98

JIT vs. AOT: Unity And Conflict of Dynamic and Static Compilers

Recommended

More Related Content

Similar to JIT vs. AOT: Unity And Conflict of Dynamic and Static Compilers (20)

More from Nikita Lipsky (20)

Recently uploaded (20)

JIT vs. AOT: Unity And Conflict of Dynamic and Static Compilers