High performance Serverless Java on AWS- GoTo Amsterdam 2024

Vadym Kazulkin | @VKazulkin |ip.labs GmbH
by Vadym Kazulkin
ip.labs GmbH
17.10.2023
How to Reduce
Cold Starts for
Java Serverless
Applications in AWS
GraalVM, AWS SnapStart and Co

Vadym Kazulkin
ip.labs GmbH Bonn, Germany
Co-Organizer of the Java User Group Bonn
v.kazulkin@gmail.com
@VKazulkin
https://dev.to/vkazulkin
https://github.com/Vadym79/
https://de.slideshare.net/VadymKazulkin/
https://www.linkedin.com/in/vadymkazulkin
https://www.iplabs.de/
Contact

About ip.labs
High performance Serverless Java on AWS
6

Java Popularity
7 High performance Serverless Java on AWS Vadym Kazulkin | @VKazulkin | ip.labs GmbH

8 High performance Serverless Java on AWS
https://distantjob.com/blog/programming-languages-rank/

Life of the Java
(Serverless)
Developer
on AWS

▪ Corretto Java 8
▪ With extended long-term support until 2026
▪ Coretto Java 11 (since 2019)
▪ Coretto Java 17 (since April 2023)
▪ Corretto Java 21(since November 2023)
▪ Only Long Term Support (LTS) by AWS
AWS Java Versions Support
10

AWS People behind Amazon Corretto
11

… but
serverless
adoption of
Java looks like
this!
Java is a very
fast and
mature
programming
language…

Percent of AWS Lambda Invocations by Language
2021 vs 2023
13
https://www.datadoghq.com/state-of-serverless-2021
https://www.datadoghq.com/state-of-serverless/
PHYTON IS THE
MOST POPULAR
LAMDA
RUNTIME

Developers love Java and will be happy
to use it for Serverless applications
But what are the challenges ?
14

▪ “cold start” times (latencies)
▪ memory footprint (high cost in AWS)
Serverless with Java Challenges
15

Challenge No. 1
A Big Cold-Start

Lambda function lifecycle – a full cold start
19
Sources: Ajay Nair „Become a Serverless Black Belt” https://www.youtube.com/watch?v=oQFORsso2go
Tomasz Łakomy "Notes from Optimizing Lambda Performance for Your Serverless Applications“ https://tlakomy.com/optimizing-lambda-performance-for-serverless-applications

▪ Start Firecracker VM (execution environment)
▪ AWS Lambda starts the Java runtime
▪ Java runtime loads and initializes Lambda function code
(Lambda handler Java class)
▪ Class loading
▪ Static initializer block of the handler class is executed (i.e. AWS service client
creation)
▪ Runtime dependency injection
▪ Just-in-Time (JIT) compilation
▪ Init-phase has full CPU access up to 10 seconds for free for the managed execution
environments)
▪ Lambda invokes the handler method
20
Sources: Ajay Nair „Become a Serverless Black Belt” https://www.youtube.com/watch?v=oQFORsso2go
Tomasz Łakomy "Notes from Optimizing Lambda Performance for Your Serverless Applications“ https://tlakomy.com/optimizing-lambda-performance-for-serverless-applications
Michael Hart: „Shave 99.93% off your Lambda bill with this one weird trick“ https://hichaelmart.medium.com/shave-99-93-off-your-lambda-bill-with-this-one-weird-trick-33c0acebb2ea
Lambda function lifecycle

Demo Application
21
https://github.com/Vadym79/AWSLambdaJavaSnapStart
▪ Lambda has 1024 MB memory setting
▪ Default (Apache) Http Client for
communication with DynamoDB
▪ 15 MB artifact size, , all dependencies in
the POM file
▪ Java compilation option -
XX:+TieredCompilation -
XX:TieredStopAtLevel=1
▪ Info about the experiments:
▪ Approx. 1 hour duration
▪ Approx. first* 100 cold starts
▪ Approx. first 100.000 warm starts
*after Lambda function being re-deployed

https://dev.to/aws-builders/aws-snapstart-part-13-measuring-warm-starts-with-java-21-using-different-lambda-memory-settings-160k
Measurements in
ms
p50 p75 p90 p99 p99.9 max
Amazon Corretto
Java 21 cold start
3158 3214 3270 3428 3601 3725
Amazon Corretto
Java 21 warm start
5,77 6,50 7,81 20,65 90,20 1423,63
Cold and warm starts with Java 21

▪ General best practices
▪ AWS SnapStart
▪ GraalVM (Native Image)
▪ Provisioned concurrency
Options to Reduce Cold Start Time
23

▪ Initialize dependencies during initialization phase
▪ Use static initialization in the handler class, instead of in the handler method (e.g.
handleRequest) to take the advantage of the access to the full CPU core for max 10
seconds
▪ In case of DynamoDB client put the following code outside of the Lambda handler
method:
▪ DynamoDbClient client = DynamoDbClientBuilder.builder()...build();
Best Practices & Recommendations
24

Provide all known values (for building clients i.e. DynamoDB client) to avoid
auto-discovery
▪ region, credential provider
DynamoDbClient client =
DynamoDbClientBuilder.builder().region(Regions.US_WEST_2).build();
25

▪ Prime dependencies during initialization phase (when it worth doing)
▪ „Fake“ the calls to pre-initialize „some other expensive stuff“ (this technique is
called Priming)
▪ In case of DynamoDB client put the following code outside of the handler method
to pre-initialize the Jackson Marshaller:
DynamoDbClient client =
DynamoDbClientBuilder.builder().region(Regions.US_WEST_2).build();
GetItemResponse getItemResponse = client.getItem(GetItemRequest.builder()
.key(Map.of("PK", AttributeValue.builder().s(0).build()))
.tableName(PRODUCT_TABLE_NAME).build());
26
invocation forces Jackson Marshallers to initialize

Word of caution
27
Re-measure it yourselves!

In case a priming is possible, but not
done and cold start happens, the warm
start/execution time takes longer
Effect of Priming
29

Avoid:
▪ reflection
▪ runtime byte code generation
▪ runtime generated proxies
▪ dynamic class loading
Use DI Frameworks which aren‘t
reflection-based
30

AWS SnapStart

„AWS Lambda SnapStart „ series
32
https://dev.to/vkazulkin/measuring-java-11-lambda-cold-starts-with-snapstart-part-1-first-impressions-30a4
Article series covers they why and what
behind Lambda SnapStart and priming
techniques including measurements for cold
and warm starts with different settings for
▪ Java 11
▪ Java 17
▪ Java 21
▪ Micronaut
▪ Quarkus
▪ Spring Boot 2.7

AWS SnapStart Deployment & Invocation
34
https://aws.amazon.com/de/blogs/compute/reducing-java-cold-starts-on-aws-lambda-functions-with-snapstart/
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Lambda uses
the CRaC APIs
for runtime
hooks
C
Create
Snapshot
Firecracker microVM
create & restore
snapshot

35

36
https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html
Firecracker microVM
create & restore
snapshot

▪ Linux CRIU available since 2012 allows a running application to be paused and restarted at some
point later in time, potentially on a different machine.
▪ The overall goal of the project is to support the migration of containers.
▪ When performing a checkpoint, essentially, the full context of the process is saved: program
counter, registers, stacks, memory-mapped and shared memory
▪ To restore the application, all this data can be reloaded and (theoretically) it continues from the
same point.
▪ Challenges
▪ open files
▪ network connections
▪ sudden change in the value of the system clock
▪ time-based caches
CRIU (Checkpoint/Restore in Userspace)
37
https://criu.org/Main_Page

▪ Compared to CRIU and Docker checkpoint and restore, Firecracker
snapshotting is different in the sense that it does not only saves a single
process or container but a whole running OS.
▪ Advantages : we don't have to care about file handles because they will be still valid
after resume.
▪ Drawbacks : the need to reseed /dev/random and to sync the system clock.
Snapshot /checkpointing and restore
Firecracker microVM vs CRIU
38
https://mail.openjdk.org/pipermail/discuss/2021-July/005864.html

39

AWS SnapStart Deployment Phase
40

AWS SnapStart Invocation Phase
41

42
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Lambda uses the
CRaC APIs for
runtime hooks
for Priming
C
Create
Snapshot
Firecracker microVM
create & restore snapshot

Speed up warmup time of the Java applications
▪ The C2 compiler is used for very hot methods, which uses profiling data collected from
the running application to optimize as much as possible.
▪ Techniques like aggressive method inlining and speculative optimizations can easily
lead to better performing code than generated ahead of time (AOT) using a static
compiler.
▪ JVM needs both time and compute resources to determine which methods to compile
and compiling them. This same work has to happen every time we run an application
▪ Ideally, we would like to run the application and then store all the state about the
compiled methods, even the compiled code and state associated with the application
and then we’d like to be able to restore it
Ideas behind CRaC (Coordinated Restore at
Checkpoint)
44
https://www.azul.com/blog/superfast-application-startup-java-on-crac/ https://github.com/CRaC/docs

CRaC (Coordinated Restore at
Checkpoint) Project
▪ Based on Linux
Checkpoint/Restore in Userspace
(CRIU)
▪ Simple API: beforeCheckpoint()
and afterRestore() methods
CRaC (Coordinated Restore at Checkpoint)
45
https://www.azul.com/blog/superfast-application-startup-java-on-crac/ | https://github.com/CRaC/docs | https://criu.org/Main_Page

AWS SnapStart Enabled with Java Priming
46
https://dev.to/aws-builders/measuring-java-11-lambda-cold-starts-with-snapstart-part-5-priming-end-to-end-latency-and-deployment-time-jem
<dependency>
<groupId>io.github.crac</groupId>
<artifactId>org-crac</artifactId>
<version>0.1.3</version>
</dependency>
if a refer to Priming in this
talk, I mean concretely
DynamoDB request
invocation priming

Lambda SnapStart Priming Guide
47
guide aims to explain techniques
for priming Java applications.
It assumes a base understanding
of AWS Lambda, Lambda
SnapStart, and CRaC.
https://github.com/marksailes/snapstart-priming-guide

AWS Lambda under the Hood
48
https://www.infoq.com/articles/aws-lambda-under-the-hood/
https://www.infoq.com/presentations/aws-lambda-arch/

▪ SnapStart supports the Java 11, 17 and 21 (Corretto) managed runtime
only
▪ Deployment with SnapStart enabled takes more than 2-2,5 minutes
additionally
▪ Snapshot is deleted from cache if Lambda function is not invoked for 14
days
▪ SnapStart currently does not support :
▪ Provisioned concurrency
▪ arm64 architecture (supports only x86)
▪ Amazon Elastic File System (Amazon EFS)
▪ Ephemeral storage greater than 512 MB
AWS SnapStart Challenges & Limitations
49

▪ Perform Priming out of the box without writing the logic on our own
▪ If snapshot not found do regular cold start and create snapshot under
the hood
▪ Snapshot creation on first Lambda function invocation instead of during
the deployment phase
▪ Regular cold start as long as the snapshot hasn’t been fully taken
▪ Trade off between duration of the deployment phase and several regular
bigger cold starts until the snapshot is taken and SnapStart becomes
effective (and therefore much shorter cold starts)
AWS SnapStart Possible Next Steps
50
https://dev.to/aws-builders/reducing-cold-starts-on-aws-lambda-with-java-runtime-future-ideas-about-snapstart-graalvm-and-co-3a1b

0
500
1000
1500
2000
2500
3000
3500
4000
w/o SnapStart with SnapStart w/o Priming with SnapStart with Priming
Cold starts of Lambda function with Java 21 runtime with
1024 MB memory setting, Apache Http Client, compilation
-XX:+TieredCompilation -XX:TieredStopAtLevel=1
p50 p75 p90 p99 p99.9 max

0,00
5,00
10,00
15,00
20,00
25,00
Warm starts of Lambda function with Java 21 runtime with
1024 MB memory setting, Apache Http Client compilation -
XX:+TieredCompilation -XX:TieredStopAtLevel=1
p50 p75 p90 p99

0
200
400
600
800
1000
1200
1400
1600
1024 MB memory setting, Apache Http Client compilation -
XX:+TieredCompilation -XX:TieredStopAtLevel=1
p99.9 max

Demo Application
54
https://github.com/Vadym79/AWSLambdaJavaSnapStart
▪ Lambda has 1024 MB memory setting
▪ Default (Apache) Http Client for
communication with DynamoDB
▪ 15 MB artifact size, , all dependencies in
the POM file
▪ Java compilation option -
XX:+TieredCompilation -
XX:TieredStopAtLevel=1
▪ Info about the experiments:
▪ Approx. 1 hour duration
▪ Approx. first* 100 cold starts
▪ Approx. first 100.000 warm starts
*after Lambda function being re-deployed

Lambda Memory Setting
55
ONE SECOND ONE GB

0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
256 MB 512 MB 768 MB 1024 MB 1536 MB 2048 MB
Cold starts of Lambda function with Java 21 runtime
without SnapStart for different memory settings
p50 p75 p90 p99 p99.9 max

0,00
20,00
40,00
60,00
80,00
100,00
120,00
140,00
160,00
180,00
256 MB 512 MB 768 MB 1024 MB 1536 MB 2048 MB
Warm starts of Lambda function with Java 21 runtime
without SnapStart for different memory settings
p50 p75 p90 p99 p99.9

0
1000
2000
3000
4000
5000
6000
7000
8000
256 MB 512 MB 768 MB 1024 MB 1536 MB 2048 MB
SnapStart without Priming for different memory settings
p50 p75 p90 p99 p99.9 max

0
20
40
60
80
100
120
140
160
180
256 MB 512 MB 768 MB 1024 MB 1536 MB 2048 MB
with SnapStart without Priming for different memory settings
p50 p75 p90 p99 p99.9

0
200
400
600
800
1000
1200
1400
1600
1800
2000
256 MB 512 MB 768 MB 1024 MB 1536 MB 2048 MB
SnapStart with Priming for different memory settings
p50 p75 p90 p99 p99.9 max

0,00
100,00
200,00
300,00
400,00
500,00
600,00
256 MB 512 MB 768 MB 1024 MB 1536 MB 2048 MB
with SnapStart with Priming for different memory settings
p50 p75 p90 p99 p99.9

Lambda Deployment Artifact Size
62

0
500
1000
1500
2000
2500
3000
3500
using deployment artifact sizes for p90
p90 small p90 medium p90 big
https://dev.to/aws-builders/aws-snapstart-part-11-measuring-cold-starts-with-java-21-using-different-deployment-artifact-sizes-4g29
▪ Small -137 KB (“Hello World”)
▪ Medium – 15 MB (our sample
application)
▪ Big -50 MB (our sample
application + additional
dependencies other to AWS
services)

▪ Less (dependencies, classes) is more
▪ Include only required dependencies (e.g. not the whole AWS SDK 2.0 for Java, but the
dependencies to the clients to be used in Lambda)
▪ Exclude dependencies, which you don‘t need at runtime i.e. test frameworks like Junit
64
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>dynamodb</artifactId>
</dependency>
<dependency>
<artifactId>bom</artifactId>
<type>pom</type>
<scope>import</scope>
</dependency>

Using Lambda Layers
65
https://docs.aws.amazon.com/lambda/latest/dg/chapter-layers.html
https://docs.aws.amazon.com/lambda/latest/dg/java-layers.html#java-layer-compatibility
• aws-lambda-java-core
• aws-lambda-java-events
• org-crac
• slf4j-simple
• jackson-dataformat-xml
Type: AWS::Serverless::Function
Properties:
FunctionName: MyFunctionWithLayer
Layers:
- !Sub
arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:layer:a
ws-pure-java-21-common-lambda-layer:1
Handler:
software…handler.GetProductByIdHandler::handleRequest

0
500
1000
1500
2000
2500
3000
3500
4000
and without the usage of the Lambda Layer options for
p90
all dependencies in the POM file using Lambda Layer
https://github.com/Vadym79/AWSLambdaJavaSnapStart/tree/main/pure-lambda-21-with-common-layer

7,00
7,10
7,20
7,30
7,40
7,50
7,60
7,70
7,80
7,90
8,00
8,10
with and without the usage of the Lambda Layer options
for p90
all dependencies in the POM file with Lambda Layer
https://github.com/Vadym79/AWSLambdaJavaSnapStart/tree/main/pure-lambda-21-with-common-layer

Java Compilation
68
https://www.geeksforgeeks.org/what-are-the-roles-of-java-compiler-and-interpreter/

Java Compilation Options
69
Mark Sailes: "Optimizing AWS Lambda function performance for Java” https://aws.amazon.com/de/blogs/compute/optimizing-aws-lambda-function-performance-for-java/

0
1000
2000
3000
4000
5000
6000
using different compilation options for p90
Tiered StopAtLevel 1 StopAtLevel 2 StopAtLevel 3 StopAtLevel 4
https://dev.to/aws-builders/aws-snapstart-part-14-measuring-cold-and-warm-starts-with-java-21-using-different-compilation-options-el4

0,00
2,00
4,00
6,00
8,00
10,00
12,00
using different compilation options for p90
Tiered StopAtLevel 1 StopAtLevel 2 StopAtLevel 3 StopAtLevel 4
https://dev.to/aws-builders/aws-snapstart-part-14-measuring-cold-and-warm-starts-with-java-21-using-different-compilation-options-el4

HTTP Clients to connect to DynamoDB
72

DynamoDbClient client = DynamoDbClient.builder().region(Region.EU_CENTRAL_1)
. httpClient(ApacheHttpClient.create())
//.httpClient(UrlConnectionHttpClient.create())
//.httpClient(AwsCrtHttpClient.create())
.build();
Add dependency to the synchronous HTTP Client in use to pom.xml, i.e.
<dependency>
<artifactId>aws-crt-client</artifactId>
</dependency>
Setting synchronous DynamoDB Client
73

0
500
1000
1500
2000
2500
3000
3500
w/o AWS SnapStart with AWS SnapStart w/o priming with AWS SnapStart with priming
Cold starts of Lambda function with Java 21 runtime and StopAtLevel 1 compilation
and different synchronous HTTP Clients for p90
URL Connection Apache AWS CRT
https://dev.to/aws-builders/aws-snapstart-part-15-measuring-cold-and-warm-starts-with-java-21-using-different-synchronous-http-clients-579o

0,00
2,00
4,00
6,00
8,00
10,00
12,00
Warm starts of Lambda function with Java 21 runtime and StopAtLevel 1
compilation and different synchronous HTTP Clients for p90
URL Connection Apache AWS CRT
https://dev.to/aws-builders/aws-snapstart-part-15-measuring-cold-and-warm-starts-with-java-21-using-different-synchronous-http-clients-579o

DynamoDbAsyncClient dynamoAsyncDbClient =
DynamoDbAsyncClient.builder().region(Region.EU_CENTRAL_1)
. httpClient(NettyNioAsyncHttpClient.create())
//.httpClient(AwsCrtAsyncHttpClient.create())
.build();
CompletableFuture<GetItemResponse> getItemReponseAsync =
dynamoAsyncDbClient.getItem(GetItemRequest.builder().
key(Map.of("PK",AttributeValue.builder().
s(id).build())).tableName(PRODUCT_TABLE_NAME).build());
GetItemResponse getItemResponse = getItemReponseAsync.join();
Setting asynchronous DynamoDB Client
76

Add dependency to the asynchronous HTTP Client in use to pom.xml, i.e.
<dependency>
<artifactId>netty-nio-client</artifactId>
</dependency>
Setting asynchronous DynamoDB Client
77

0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Cold starts of Lambda function with Java 21 runtime and StopAtLevel 1 compilation
and different asynchronous HTTP Clients for p90
NettyNio AWS CRT
https://dev.to/aws-builders/aws-snapstart-part-16-measuring-cold-and-warm-starts-with-java-21-using-different-asynchronous-http-clients-4n2

0,00
2,00
4,00
6,00
8,00
10,00
12,00
Warm starts of Lambda function with Java 21 runtime and StopAtLevel 1
compilation and different asynchronous HTTP Clients for p90
NettyNio AWS CRT
https://dev.to/aws-builders/aws-snapstart-part-16-measuring-cold-and-warm-starts-with-java-21-using-different-asynchronous-http-clients-4n2

HTTP client recommendations
80
https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/http-configuration.html

81
Firecracker microVM
create & restore
snapshot

0
500
1000
1500
2000
2500
with SnapStart w/o Priming with SnapStart with Priming
p50 p75 p90 p99 p99.9 max
https://dev.to/aws-builders/aws-snapstart-part-17-impact-of-the-snapshot-tiered-cache-on-the-cold-starts-with-java-21-52ef

0
500
1000
1500
2000
2500
with SnapStart w/o Primingwith SnapStart w/o Priming
(last 50)
with SnapStart with
Priming
with SnapStart with
Priming (last 50)
Cold starts of all approx.100 and last 50 Lambda functions with
Java 21 runtime with
p50 p75 p90 p99 p99.9 max
https://dev.to/aws-builders/aws-snapstart-part-17-impact-of-the-snapshot-tiered-cache-on-the-cold-starts-with-java-21-52ef
Due to the effect of
snapshot tiered cache,
cold start times
reduces with the
number of invocations

▪ Low footprint ahead-of-time mode for JVM-based languages
▪ High performance for all languages
▪ Convenient language interoperability and polyglot tooling
GraalVM Goals
85
Source: „Everything you need to know about GraalVM by Oleg Šelajev & Thomas Wuerthinger” https://www.youtube.com/watch?v=ANN9rxYo5Hg

GraalVM Architecture
86

GraalVM Ahead-of-Time Compilation
87
Source: Oleg Šelajev, Thomas Wuerthinger, Oracle: “Deep dive into using GraalVM for Java and JavaScript”
https://www.youtube.com/watch?v=a-XEZobXspo

AOT vs JIT
88
Source: „Everything you need to know about GraalVM by Oleg Šelajev & Thomas Wuerthinger” https://www.youtube.com/watch?v=ANN9rxYo5Hg

Java Function compiled into a native executable using GraalVM on SubstrateVM
reduces
▪ “cold start” times
▪ memory footprint
by order of magnitude compared to running on JVM.
GraalVM on SubstrateVM as another option for Java &
Serverless
89

▪ AWS doesn’t provide GraalVM (Native Image) as Java Runtime out of the box
▪ AWS provides Custom Runtime Option
Current Challenges with Native Executable using
GraalVM
90

Custom Lambda Runtimes
91
https://github.com/Vadym79/AWSLambdaGraalVMNativeImage

GraalVM Release Calendar
92
https://www.graalvm.org/release-calendar/

GraalVM CE is based on OpenJDK
93
https://www.graalvm.org/2022/openjdk-announcement/
https://blogs.oracle.com/java/post/graalvm-free-license

0
500
1000
1500
2000
2500
3000
3500
4000
w/o SnapStart with SnapStart w/o Priming with SnapStart with
Priming
GraalVM Native Image
Cold starts of Lambda function with Java 21 runtime with.
1024 MB memory setting, Apache Http Client
p50 p75 p90 p99 p99.9 max

0,00
5,00
10,00
15,00
20,00
25,00
w/o SnapStart with SnapStart w/o
Priming
with SnapStart with
Priming
p50 p75 p90 p99

0
200
400
600
800
1000
1200
1400
1600
w/o SnapStart with SnapStart w/o Priming with SnapStart with
Priming
Max warm starts of Lambda function with Java 21 runtime with
p99.9 max

Lambda Memory Setting for Custom Runtime (GraalVM
Native Image)
97
ONE SECOND ONE GB

0
200
400
600
800
1000
1200
256 MB 512 MB 768 MB 1024 MB 1280 MB 1536 MB
Cold starts of Lambda function with GraalVM 21 Native
Image for different memory settings
p50 p75 p90 p99 p99.9 max
https://github.com/Vadym79/AWSLambdaGraalVMNativeImage/tree/master/pure-lambda-graalvm-jdk-21-native-image

0,00
2,00
4,00
6,00
8,00
10,00
12,00
14,00
16,00
18,00
20,00
256 MB 512 MB 768 MB 1024 MB 1280 MB 1536 MB
Warm starts of Lambda function with GraalVM 21 Native
p50 p75 p90 p99

0,00
50,00
100,00
150,00
200,00
250,00
300,00
350,00
400,00
450,00
500,00
256 MB 512 MB 768 MB 1024 MB 1280 MB 1536 MB
Warm starts of Lambda function with GraalVM 21 Native
p99.9 max

▪ GraalVM is really powerful and has a lot of potential
▪ GraalVM Native Image improves cold starts and memory footprint
significantly
▪ GraalVM Native Image is currently not without challenges
▪ AWS Lambda Custom Runtime requires Linux executable only
▪ Building Custom Runtime requires some additional effort
▪ e.g. you need a scalable CI/CD pipeline to build memory-intensive native image
▪ Build time is a factor
▪ You pay for the init-phase of the function packaged as AWS Lambda
Custom and Docker Runtime
▪ Init-phase is free for the managed runtimes like Java 11, Java17 and Java 21
GraalVM Conclusion
101

▪ You can run into errors when application
is running
GraalVM Conclusion
102
https://github.com/Vadym79/AWSLambdaGraalVMNativeImage/blob/master/pure-lambda-graalvm-jdk-21-native-image/src/main/reflect.json

Frameworks and libraries Ready for GraalVM Native Image
103
https://www.graalvm.org/native-image/libraries-and-frameworks/

Lambda Provisioned Concurrency
104
Requires manually managing start and end
time when provisioned concurrency should
apply (can be tricky for spikey workloads)
▪ You pay for unused capacity
https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/

Lambda Proactive Initialization
105
In June 2023 AWS updated the documentation for the Lambda Function lifecycle and included this
new statement: for functions using unreserved (on-demand) concurrency, Lambda may
proactively initialize a function instance, even if there's no invocation. When this happens,
you can observe an unexpected time gap between your function's initialization and invocation
phases. This gap can appear similar to what you would observe when using provisioned
concurrency.
https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtime-environment.html
https://aaronstuyvenberg.com/posts/understanding-proactive-initialization
Running this query over several days
across multiple runtimes and
invocation methods, between 50%
and 75% of initializations were
proactive (versus 50% to 25% which
were true cold starts)

“Data API for Amazon Aurora Serverless v2
with AWS SDK for Java” series
106
Article series also covers
cold and warm start time
measurements and
optimization techniques
https://dev.to/aws-builders/data-api-for-amazon-aurora-serverless-v2-with-aws-sdk-for-java-part-1-introduction-and-set-up-of-the-sample-application-3g71

“Spring Boot 3 application on AWS Lambda” series
107
Article series covers different
ways to write Spring Boot 3
application on AWS Lambda
▪ AWS Serverless Java Container
▪ AWS Lambda Web Adapter
▪ Spring Cloud Functions
▪ Custom Docker Image
▪ GraalVM Native Image
Cold and warm start time
measurements are also provided
https://dev.to/aws-builders/spring-boot-3-application-on-aws-lambda-part-1-introduction-to-the-series-2m5g

Project Leyden
108
The primary goal of this
Project is to improve the
startup time, time to
peak performance, and
footprint of Java
programs.
https://www.youtube.com/watch?v=lnth19Kf-x0
https://openjdk.org/projects/leyden/

Word of caution
109
Re-measure for your use case!
Even with my examples measurements
might already produce different results due
to:
▪ Lambda Amazon Corretto Java 21
managed runtime minor version changes
▪ Lambda SnapStart snapshot create and
restore improvements
▪ Firecracker VM improvements
▪ Graal VM (major and minor version) and
Native Image improvements
▪ There are still servers behind Lambda
▪ Java Memory Model impact (L or RAM
caches hits and misses)

Thank you

Image: burst.shopify.com/photos/a-look-across-the-landscape-with-view-of-the-sea

High performance Serverless Java on AWS- GoTo Amsterdam 2024

Recommended

More Related Content

Similar to High performance Serverless Java on AWS- GoTo Amsterdam 2024 (20)

More from Vadym Kazulkin (14)

Recently uploaded (20)

High performance Serverless Java on AWS- GoTo Amsterdam 2024