High performance Serverless Java on AWS at Froscon 2024

Vadym Kazulkin | @VKazulkin |ip.labs GmbH
High Performance Serverless Java on AWS
Vadym Kazulkin, ip.labs, FroOSCon 18 August 2024

Vadym Kazulkin
ip.labs GmbH Bonn, Germany
Co-Organizer of the Java User Group Bonn
v.kazulkin@gmail.com
@VKazulkin
https://dev.to/vkazulkin
https://github.com/Vadym79/
https://de.slideshare.net/VadymKazulkin/
https://www.linkedin.com/in/vadymkazulkin
https://www.iplabs.de/
Contact

About ip.labs
High performance Serverless Java on AWS
4

Java Popularity
5 High performance Serverless Java on AWS Vadym Kazulkin | @VKazulkin | ip.labs GmbH

6 High performance Serverless Java on AWS
https://distantjob.com/blog/programming-languages-rank/

Life of the Java
(Serverless)
Developer
on AWS

▪ Corretto Java 8
▪ With extended long-term support until 2026
▪ Coretto Java 11 (since 2019)
▪ Coretto Java 17 (since April 2023)
▪ Corretto Java 21(since November 2023)
▪ Only Long Term Support (LTS) by AWS
AWS Java Versions Support
8

AWS People behind Amazon Corretto
9

… but
serverless
adoption of
Java looks like
this!
Java is a very
fast and
mature
programming
language…

Percent of AWS Lambda Invocations by Language
2021 vs 2023
11
https://www.datadoghq.com/state-of-serverless-2021
https://www.datadoghq.com/state-of-serverless/
PHYTON IS THE
MOST POPULAR
LAMDA
RUNTIME

Developers love Java and will be happy
to use it for Serverless applications
But what are the challenges ?
12

▪ “cold start” times (latencies)
▪ memory footprint (high cost in AWS)
Serverless with Java Challenges
13

Challenge No. 1
A Big Cold-Start

Lambda function lifecycle – a full cold start
17
Sources: Ajay Nair „Become a Serverless Black Belt” https://www.youtube.com/watch?v=oQFORsso2go
Tomasz Łakomy "Notes from Optimizing Lambda Performance for Your Serverless Applications“ https://tlakomy.com/optimizing-lambda-performance-for-serverless-applications

▪ Start Firecracker VM (execution environment)
▪ AWS Lambda starts the Java runtime
▪ Java runtime loads and initializes Lambda function code
(Lambda handler Java class)
▪ Class loading
▪ Static initializer block of the handler class is executed (i.e. AWS service client
creation)
▪ Runtime dependency injection
▪ Just-in-Time (JIT) compilation
▪ Init-phase has full CPU access up to 10 seconds for free for the managed execution
environments)
▪ Lambda invokes the handler method
18
Sources: Ajay Nair „Become a Serverless Black Belt” https://www.youtube.com/watch?v=oQFORsso2go
Tomasz Łakomy "Notes from Optimizing Lambda Performance for Your Serverless Applications“ https://tlakomy.com/optimizing-lambda-performance-for-serverless-applications
Michael Hart: „Shave 99.93% off your Lambda bill with this one weird trick“ https://hichaelmart.medium.com/shave-99-93-off-your-lambda-bill-with-this-one-weird-trick-33c0acebb2ea
Lambda function lifecycle

Demo Application
19
https://github.com/Vadym79/AWSLambdaJavaSnapStart
▪ Lambda has 1024 MB memory setting
▪ Lambda uses x86 architecture
▪ Default (Apache) Http Client for
communication with DynamoDB
▪ 18 MB artifact size, , all dependencies in
the POM file
▪ Java compilation option -
XX:+TieredCompilation -
XX:TieredStopAtLevel=1
▪ Info about the experiments:
▪ Approx. 1 hour duration
▪ Approx. first* 100 cold starts
▪ Approx. first 100.000 warm starts
*after Lambda function being re-deployed

https://dev.to/aws-builders/aws-snapstart-part-13-measuring-warm-starts-with-java-21-using-different-lambda-memory-settings-160k
Measurements in
ms
p50 p75 p90 p99 p99.9 max
Amazon Corretto
Java 21 cold start
3158 3214 3270 3428 3601 3725
Amazon Corretto
Java 21 warm start
5,77 6,50 7,81 20,65 90,20 1423,63
Cold and warm starts with Java 21

Latency is the amount of time it takes from when a request
is made by the user to the time it takes for the response to
get back to that user
Latency
How to develop, run and optimize Spring Boot 3 application on AWS Lambda
21

For our application:
Overall latency =
t(client to create and send request to the API Gateway endpoint) +
t(API Gateway to process request and invoke Lambda function) +
t(Lambda cold start time) +
t(Lambda warm start time) +
t(API Gateway to processes Lambda response and sends it back to the client) +
t(client to receive and process response from the API Gateway endpoint)
Latency
22
involves DNS resolve
and SSL handshake

▪ General best practices
▪ AWS SnapStart
▪ GraalVM (Native Image)
▪ Provisioned concurrency
Options To Reduce Cold Start Time
23

▪ Initialize dependencies during initialization phase
▪ Use static initialization in the handler class, instead of in the handler method (e.g.
handleRequest) to take the advantage of the access to the full CPU core for max 10
seconds
▪ In case of DynamoDB client put the following code outside of the Lambda handler
method:
▪ DynamoDbClient client = DynamoDbClientBuilder.builder()...build();
Best Practices & Recommendations
24

Provide all known values (for building clients i.e. DynamoDB client) to avoid
auto-discovery
▪ region, credential provider
DynamoDbClient client =
DynamoDbClientBuilder.builder().region(Regions.US_WEST_2).build();
25

▪ Prime dependencies during initialization phase (when it worth doing)
▪ „Fake“ the calls to pre-initialize „some other expensive stuff“ (this technique is called
Priming)
▪ In case of DynamoDB client put the following code outside of the handler method to
pre-initialize the HTTP Client and Jackson Marshaller:
DynamoDbClient client = DynamoDbClientBuilder.builder().region(Regions.US_WEST_2).build();
GetItemResponse getItemResponse = client.getItem(GetItemRequest.builder()
.key(Map.of("PK", AttributeValue.builder().s(id).build()))
.tableName(PRODUCT_TABLE_NAME).build());
……
26
invocation forces HTTP Client and Jackson
Marshallers to pre-initialize
getProductById (int id)
method

Word of caution
27
Re-measure it yourselves!

In case a priming is possible, but not
done and cold start happens, the warm
start/execution time takes longer
Effect of Priming
29

Avoid:
▪ reflection
▪ runtime byte code generation
▪ runtime generated proxies
▪ dynamic class loading
Use DI Frameworks which aren‘t
reflection-based
30

AWS Lambda
SnapStart
31 How to develop, run and optimize Spring Boot 3 application on AWS Lambda

AWS Lambda SnapStart != SnapChat
AWS Lambda SnapStart
32
https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html

„AWS Lambda SnapStart „ series
33
https://dev.to/vkazulkin/measuring-java-11-lambda-cold-starts-with-snapstart-part-1-first-impressions-30a4
Article series covers they why and what
behind Lambda SnapStart and priming
techniques including measurements for cold
and warm starts with different settings for
▪ Java 11
▪ Java 17
▪ Java 21
▪ Micronaut
▪ Quarkus
▪ Spring Boot 2.7

▪ Lambda SnapStart for Java can improve startup performance for latency-
sensitive applications
▪ SnapStart is fully managed
34

AWS SnapStart Deployment & Invocation
35
https://aws.amazon.com/de/blogs/compute/using-aws-lambda-snapstart-with-infrastructure-as-code-and-ci-cd-pipelines/

▪ Currently available for Lambda managed Java Runtimes (Java 11, 17 and 21)
▪ Not available for all other Lambda runtimes:
▪ Docker Container Image
▪ Custom (Lambda) Runtime (a way to ship GraalVM Native Image)
36
https://github.com/Vadym79/AWSLambdaJavaDockerImage/

38
https://aws.amazon.com/de/blogs/compute/reducing-java-cold-starts-on-aws-lambda-functions-with-snapstart/
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Lambda uses
the CRaC APIs
for runtime
hooks
C
Create
Snapshot
Firecracker microVM
create & restore
snapshot

AWS SnapStart Deployment Phase
39

AWS SnapStart Invocation Phase
40

▪ Linux CRIU available since 2012 allows a running application to be paused and restarted at some
point later in time, potentially on a different machine.
▪ The overall goal of the project is to support the migration of containers.
▪ When performing a checkpoint, essentially, the full context of the process is saved: program
counter, registers, stacks, memory-mapped and shared memory
▪ To restore the application, all this data can be reloaded and (theoretically) it continues from the
same point.
▪ Challenges
▪ open files
▪ network connections
▪ sudden change in the value of the system clock
▪ time-based caches
CRIU (Checkpoint/Restore in Userspace)
41
https://criu.org/Main_Page

▪ Comparing Firecracker and CRIU snapshotting:
▪ Firecracker snapshotting saves a whole running OS
▪ CRIU snapshotting saves a single process or container
▪ Advantages of the Firecracker snapshot : we don't have to care about
file handles because they will be still valid after resume.
▪ Drawbacks the Firecracker snapshot : the need to reseed /dev/random
and to sync the system clock.
Snapshot /checkpointing and restore
Firecracker microVM vs CRIU
42
https://mail.openjdk.org/pipermail/discuss/2021-July/005864.html

45
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Lambda uses the
CRaC APIs for
runtime hooks
for Priming
C
Create
Snapshot
Firecracker microVM
create & restore snapshot

▪ CRaC is a JDK project that allows you to start Java programs with a
shorter time to first transaction, combined with less time and resources
to achieve full code speed.
▪ CRaC effectively takes a snapshot of the Java process (checkpoint) when
it is fully warmed up, then uses that snapshot to launch any number of
JVMs from this captured state.
▪ CRaC is based on CRIU
Ideas behind CRaC (Coordinated Restore at
Checkpoint)
46
https://www.azul.com/blog/superfast-application-startup-java-on-crac/ https://github.com/CRaC/docs

CRaC (Coordinated Restore at
Checkpoint) Project
▪ Based on Linux
Checkpoint/Restore in Userspace
(CRIU)
▪ Simple API: beforeCheckpoint()
and afterRestore() methods
CRaC (Coordinated Restore at Checkpoint)
47
https://www.azul.com/blog/superfast-application-startup-java-on-crac/ | https://github.com/CRaC/docs | https://criu.org/Main_Page

AWS SnapStart Enabled with Java Priming
48
https://dev.to/aws-builders/measuring-java-11-lambda-cold-starts-with-snapstart-part-5-priming-end-to-end-latency-and-deployment-time-jem
<dependency>
<groupId>io.github.crac</groupId>
<artifactId>org-crac</artifactId>
<version>0.1.3</version>
</dependency>
if a refer to Priming in this
talk, I mean concretely
DynamoDB request
invocation priming

Lambda SnapStart Priming Guide
49
guide aims to explain techniques
for priming Java applications.
It assumes a base understanding
of AWS Lambda, Lambda
SnapStart, and CRaC.
https://github.com/marksailes/snapstart-priming-guide

▪ SnapStart supports the Java 11, 17 and 21 (Corretto) managed runtime
only
▪ Deployment with SnapStart enabled takes more than 2-2,5 minutes
additionally
▪ Snapshot is deleted from cache if Lambda function is not invoked for 14
days
▪ SnapStart currently does not support :
▪ Provisioned concurrency
▪ arm64 architecture (supports only x86). Supports arm64 architecture since
18 July 2024
▪ Amazon Elastic File System (Amazon EFS)
▪ Ephemeral storage greater than 512 MB
AWS SnapStart Challenges & Limitations
50

0
500
1000
1500
2000
2500
3000
3500
4000
w/o SnapStart with SnapStart w/o Priming with SnapStart with Priming
Cold starts of Lambda function with Java 21 runtime with
1024 MB memory setting, Apache Http Client, compilation
-XX:+TieredCompilation -XX:TieredStopAtLevel=1
p50 p75 p90 p99 p99.9 max
ms

0,00
5,00
10,00
15,00
20,00
25,00
Warm starts of Lambda function with Java 21 runtime with
1024 MB memory setting, Apache Http Client compilation -
XX:+TieredCompilation -XX:TieredStopAtLevel=1
p50 p75 p90 p99
ms

0
200
400
600
800
1000
1200
1400
1600
1024 MB memory setting, Apache Http Client compilation -
XX:+TieredCompilation -XX:TieredStopAtLevel=1
p99.9 max
ms

Demo Application
55
the POM file

Lambda Memory Setting
56
ONE SECOND ONE GB

0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
256 MB 512 MB 768 MB 1024 MB 1536 MB 2048 MB
Cold starts of Lambda function with Java 21 runtime
without SnapStart for different memory settings
p50 p75 p90 p99 p99.9 max
ms

0,00
20,00
40,00
60,00
80,00
100,00
120,00
140,00
160,00
180,00
256 MB 512 MB 768 MB 1024 MB 1536 MB 2048 MB
Warm starts of Lambda function with Java 21 runtime
without SnapStart for different memory settings
p50 p75 p90 p99 p99.9
ms

0
1000
2000
3000
4000
5000
6000
7000
8000
256 MB 512 MB 768 MB 1024 MB 1536 MB 2048 MB
SnapStart without Priming for different memory settings
p50 p75 p90 p99 p99.9 max
ms

0
20
40
60
80
100
120
140
160
180
256 MB 512 MB 768 MB 1024 MB 1536 MB 2048 MB
with SnapStart without Priming for different memory settings
p50 p75 p90 p99 p99.9
ms

0
200
400
600
800
1000
1200
1400
1600
1800
2000
256 MB 512 MB 768 MB 1024 MB 1536 MB 2048 MB
SnapStart with Priming for different memory settings
p50 p75 p90 p99 p99.9 max
ms

0,00
100,00
200,00
300,00
400,00
500,00
600,00
256 MB 512 MB 768 MB 1024 MB 1536 MB 2048 MB
with SnapStart with Priming for different memory settings
p50 p75 p90 p99 p99.9
ms

Lambda Deployment Artifact Size
63

0
500
1000
1500
2000
2500
3000
3500
using deployment artifact sizes for p90
p90 small p90 medium p90 big
ms
https://dev.to/aws-builders/aws-snapstart-part-11-measuring-cold-starts-with-java-21-using-different-deployment-artifact-sizes-4g29
▪ Small -137 KB (“Hello World”)
▪ Medium – 18 MB (our sample
application)
▪ Big -50 MB (our sample
application + additional
dependencies other to AWS
services)

▪ Less (dependencies, classes) is more
▪ Include only required dependencies (e.g. not the whole AWS SDK 2.0 for Java, but the
dependencies to the clients to be used in Lambda)
▪ Exclude dependencies, which you don‘t need at runtime i.e. test frameworks like Junit
65
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>dynamodb</artifactId>
</dependency>
<dependency>
<artifactId>bom</artifactId>
<type>pom</type>
<scope>import</scope>
</dependency>

Using Lambda Layers
66
https://docs.aws.amazon.com/lambda/latest/dg/chapter-layers.html
https://docs.aws.amazon.com/lambda/latest/dg/java-layers.html#java-layer-compatibility
• aws-lambda-java-core
• aws-lambda-java-events
• org-crac
• slf4j-simple
• jackson-dataformat-xml
Type: AWS::Serverless::Function
Properties:
FunctionName: MyFunctionWithLayer
Layers:
- !Sub
arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:layer:a
ws-pure-java-21-common-lambda-layer:1
Handler:
software…handler.GetProductByIdHandler::handleRequest

0
500
1000
1500
2000
2500
3000
3500
4000
and without the usage of the Lambda Layer options for
p90
all dependencies in the POM file using Lambda Layer
https://github.com/Vadym79/AWSLambdaJavaSnapStart/tree/main/pure-lambda-21-with-common-layer
ms

7,00
7,10
7,20
7,30
7,40
7,50
7,60
7,70
7,80
7,90
8,00
8,10
with and without the usage of the Lambda Layer options
for p90
all dependencies in the POM file with Lambda Layer
https://github.com/Vadym79/AWSLambdaJavaSnapStart/tree/main/pure-lambda-21-with-common-layer
ms

Java Compilation
69
https://www.geeksforgeeks.org/what-are-the-roles-of-java-compiler-and-interpreter/

Java Compilation Options
70
Mark Sailes: "Optimizing AWS Lambda function performance for Java” https://aws.amazon.com/de/blogs/compute/optimizing-aws-lambda-function-performance-for-java/

0
1000
2000
3000
4000
5000
6000
using different compilation options for p90
Tiered StopAtLevel 1 StopAtLevel 2 StopAtLevel 3 StopAtLevel 4
https://dev.to/aws-builders/aws-snapstart-part-14-measuring-cold-and-warm-starts-with-java-21-using-different-compilation-options-el4
ms

0,00
2,00
4,00
6,00
8,00
10,00
12,00
using different compilation options for p90
Tiered StopAtLevel 1 StopAtLevel 2 StopAtLevel 3 StopAtLevel 4
https://dev.to/aws-builders/aws-snapstart-part-14-measuring-cold-and-warm-starts-with-java-21-using-different-compilation-options-el4
ms

Java Garbage Collection Algorithms
73
• Tested with default GC algorithms settings
• G1 default GC produces the best results
for cold and warm starts without and with
SnapStart (and priming) followed by :
• Parallel GC
• Shenandoah GC
• Couldn‘t make it work with ZGC
(generational) GC

HTTP Clients to connect to DynamoDB
74

DynamoDbClient client = DynamoDbClient.builder().region(Region.EU_CENTRAL_1)
. httpClient(ApacheHttpClient.create())
//.httpClient(UrlConnectionHttpClient.create())
//.httpClient(AwsCrtHttpClient.create())
.build();
Add dependency to the synchronous HTTP Client in use to pom.xml, i.e.
<dependency>
<artifactId>aws-crt-client</artifactId>
</dependency>
Setting synchronous DynamoDB Client
75

0
500
1000
1500
2000
2500
3000
3500
w/o AWS SnapStart with AWS SnapStart w/o priming with AWS SnapStart with priming
Cold starts of Lambda function with Java 21 runtime and StopAtLevel 1 compilation
and different synchronous HTTP Clients for p90
URL Connection Apache AWS CRT
https://dev.to/aws-builders/aws-snapstart-part-15-measuring-cold-and-warm-starts-with-java-21-using-different-synchronous-http-clients-579o
ms

0,00
2,00
4,00
6,00
8,00
10,00
12,00
Warm starts of Lambda function with Java 21 runtime and StopAtLevel 1
compilation and different synchronous HTTP Clients for p90
URL Connection Apache AWS CRT
https://dev.to/aws-builders/aws-snapstart-part-15-measuring-cold-and-warm-starts-with-java-21-using-different-synchronous-http-clients-579o
ms

DynamoDbAsyncClient dynamoAsyncDbClient =
DynamoDbAsyncClient.builder().region(Region.EU_CENTRAL_1)
. httpClient(NettyNioAsyncHttpClient.create())
//.httpClient(AwsCrtAsyncHttpClient.create())
.build();
CompletableFuture<GetItemResponse> getItemReponseAsync =
dynamoAsyncDbClient.getItem(GetItemRequest.builder().
key(Map.of("PK",AttributeValue.builder().
s(id).build())).tableName(PRODUCT_TABLE_NAME).build());
GetItemResponse getItemResponse = getItemReponseAsync.join();
Setting asynchronous DynamoDB Client
78

Add dependency to the asynchronous HTTP Client in use to pom.xml, i.e.
<dependency>
<artifactId>netty-nio-client</artifactId>
</dependency>
Setting asynchronous DynamoDB Client
79

0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Cold starts of Lambda function with Java 21 runtime and StopAtLevel 1 compilation
and different asynchronous HTTP Clients for p90
NettyNio AWS CRT
ms
https://dev.to/aws-builders/aws-snapstart-part-16-measuring-cold-and-warm-starts-with-java-21-using-different-asynchronous-http-clients-4n2

0,00
2,00
4,00
6,00
8,00
10,00
12,00
Warm starts of Lambda function with Java 21 runtime and StopAtLevel 1
compilation and different asynchronous HTTP Clients for p90
NettyNio AWS CRT
ms

HTTP client recommendations
82
https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/http-configuration.html

Lambda x86_64 vs arm64 architecture
83
https://aws.amazon.com/de/about-aws/whats-new/2024/07/aws-lambda-snapstart-java-functions-arm64-architecture/

Lambda x86_64 vs arm64 architecture
84
https://aws.amazon.com/lambda/pricing/?nc1=h_ls
For the same memory setting Lambda and
Lambda function execution duration the choice of
arm64 over x86_64 architecture is approx. 25%
cheaper

0
500
1000
1500
2000
2500
3000
3500
4000
4500
-XX:+TieredCompilation -XX:TieredStopAtLevel=1 for p90
x86_64 arm64
ms
+8,6%
+23,9%
+24,8%

6,40
6,60
6,80
7,00
7,20
7,40
7,60
7,80
8,00
-XX:+TieredCompilation -XX:TieredStopAtLevel=1 for p90
x86_64 arm64
ms
+9,1%
+10,1%
+4,8%

Demo Application
87
the POM file

88
Firecracker microVM
create & restore
snapshot

AWS SnapStart tiered cache
89
https://dev.to/aws-builders/aws-snapstart-part-17-impact-of-the-snapshot-tiered-cache-on-the-cold-starts-with-java-21-52ef
• Due to the effect of
snapshot tiered cache, cold
start times reduces with the
number of invocations
• After certain number of
invocations reached the
cold start times becomes
stable

0
500
1000
1500
2000
2500
with SnapStart w/o Primingwith SnapStart w/o Priming
(last 50)
with SnapStart with
Priming
with SnapStart with
Priming (last 50)
Cold starts of all approx.100 and last 50 Lambda functions with
Java 21 runtime with
-XX:+TieredCompilation -XX:TieredStopAtLevel=1
p50 p75 p90 p99 p99.9 max
https://dev.to/aws-builders/aws-snapstart-part-17-impact-of-the-snapshot-tiered-cache-on-the-cold-starts-with-java-21-52ef
Due to the effect of
snapshot tiered cache,
cold start times
reduces with the
number of invocations
ms

AWS Lambda under the Hood
92
https://www.infoq.com/articles/aws-lambda-under-the-hood/
https://www.infoq.com/presentations/aws-lambda-arch/

▪ Low footprint ahead-of-time mode for JVM-based languages
▪ High performance for all languages
▪ Convenient language interoperability and polyglot tooling
GraalVM Goals
94
Source: „Everything you need to know about GraalVM by Oleg Šelajev & Thomas Wuerthinger” https://www.youtube.com/watch?v=ANN9rxYo5Hg

GraalVM Architecture
95

GraalVM Ahead-of-Time Compilation
96
Source: Oleg Šelajev, Thomas Wuerthinger, Oracle: “Deep dive into using GraalVM for Java and JavaScript”
https://www.youtube.com/watch?v=a-XEZobXspo

AOT vs JIT
97
Source: „Everything you need to know about GraalVM by Oleg Šelajev & Thomas Wuerthinger” https://www.youtube.com/watch?v=ANN9rxYo5Hg

Compilationinto a native executable using GraalVM reduces
▪ “cold start” times
▪ memory footprint
by order of magnitude compared to running on JVM.
GraalVM Native Image
98

▪ AWS doesn’t provide GraalVM (Native Image) as Java Runtime out of the box
▪ AWS provides Custom Runtime Option
Current Challenges with Native Executable using
GraalVM
99

Custom Lambda Runtimes
100
https://github.com/Vadym79/AWSLambdaGraalVMNativeImage

GraalVM Release Calendar
101
https://www.graalvm.org/release-calendar/

GraalVM CE is based on OpenJDK
102
https://www.graalvm.org/2022/openjdk-announcement/
https://blogs.oracle.com/java/post/graalvm-free-license

0
500
1000
1500
2000
2500
3000
3500
4000
w/o SnapStart with SnapStart w/o Priming with SnapStart with
Priming
Cold starts of Lambda function with Java 21 runtime with.
1024 MB memory setting, Apache Http Client
p50 p75 p90 p99 p99.9 max
ms

0,00
5,00
10,00
15,00
20,00
25,00
w/o SnapStart with SnapStart w/o
Priming
with SnapStart with
Priming
p50 p75 p90 p99
ms

0
200
400
600
800
1000
1200
1400
1600
w/o SnapStart with SnapStart w/o Priming with SnapStart with
Priming
Max warm starts of Lambda function with Java 21 runtime with
p99.9 max
ms

Lambda Memory Setting for Custom Runtime (GraalVM
Native Image)
106
ONE SECOND ONE GB

0
200
400
600
800
1000
1200
256 MB 512 MB 768 MB 1024 MB 1280 MB 1536 MB
Cold starts of Lambda function with GraalVM 21 Native
Image for different memory settings
p50 p75 p90 p99 p99.9 max
https://github.com/Vadym79/AWSLambdaGraalVMNativeImage/tree/master/pure-lambda-graalvm-jdk-21-native-image
ms

0,00
2,00
4,00
6,00
8,00
10,00
12,00
14,00
16,00
18,00
20,00
256 MB 512 MB 768 MB 1024 MB 1280 MB 1536 MB
Warm starts of Lambda function with GraalVM 21 Native
p50 p75 p90 p99
ms

0,00
50,00
100,00
150,00
200,00
250,00
300,00
350,00
400,00
450,00
500,00
256 MB 512 MB 768 MB 1024 MB 1280 MB 1536 MB
Warm starts of Lambda function with GraalVM 21 Native
p99.9 max
ms

▪ GraalVM is really powerful and has a lot of potential
▪ GraalVM Native Image improves cold starts and memory footprint
significantly
▪ GraalVM Native Image is currently not without challenges
▪ AWS Lambda Custom Runtime requires Linux executable only
▪ Building Custom Runtime requires some additional effort
▪ e.g. you need a scalable CI/CD pipeline to build memory-intensive native image
▪ Build time is a factor
▪ You pay for the init-phase of the function packaged as AWS Lambda
Custom and Docker Runtime
▪ Init-phase is free for the managed runtimes like Java 11, Java17 and Java 21
GraalVM Conclusion
110

▪ You can run into errors when application
is running
GraalVM Conclusion
111
https://github.com/Vadym79/AWSLambdaGraalVMNativeImage/blob/master/pure-lambda-graalvm-jdk-21-native-image/src/main/reflect.json

Frameworks and libraries Ready for GraalVM Native Image
112
https://www.graalvm.org/native-image/libraries-and-frameworks/

Lambda Provisioned Concurrency
113
Requires manually managing start and end
time when provisioned concurrency should
apply (can be tricky for spikey workloads)
▪ You pay for unused capacity
https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/

Lambda Proactive Initialization
114
In June 2023 AWS updated the documentation for the Lambda Function lifecycle and included this
new statement: for functions using unreserved (on-demand) concurrency, Lambda may
proactively initialize a function instance, even if there's no invocation. When this happens,
you can observe an unexpected time gap between your function's initialization and invocation
phases. This gap can appear similar to what you would observe when using provisioned
concurrency.
https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtime-environment.html
https://aaronstuyvenberg.com/posts/understanding-proactive-initialization
Running this query over several days
across multiple runtimes and
invocation methods, between 50%
and 75% of initializations were
proactive (versus 50% to 25% which
were true cold starts)

“Data API for Amazon Aurora Serverless v2
with AWS SDK for Java” series
115
Article series also covers
cold and warm start time
measurements and
optimization techniques
https://dev.to/aws-builders/data-api-for-amazon-aurora-serverless-v2-with-aws-sdk-for-java-part-1-introduction-and-set-up-of-the-sample-application-3g71

“Spring Boot 3 application on AWS Lambda” series
116
Article series covers different
ways to write Spring Boot 3
application on AWS Lambda
▪ AWS Serverless Java Container
▪ AWS Lambda Web Adapter
▪ Spring Cloud Functions
▪ Custom Docker Image
▪ GraalVM Native Image
Cold and warm start time
measurements are also provided
https://dev.to/aws-builders/spring-boot-3-application-on-aws-lambda-part-1-introduction-to-the-series-2m5g

▪ With AWS SnapStart and GraalVM Native Image you can reduce cold
start times of the AWS Lambda with Java 21 runtime to the acceptable
values
▪ If you’re willing to accept slightly higher cold and warm start times for
certain the Lambda function(s) and solid priming is applicable -> use fully
managed AWS SnapStart with priming
▪ If a very high performance for certain the Lambda function(s) is really
crucial for your business -> go for GraalVM Native Image
Wrap up and personal suggestions
117

Project Leyden
118
The primary goal of
this Project is to
improve the startup
time, time to peak
performance, and
footprint of Java
programs.
https://www.youtube.com/watch?v=teXijm79vno
https://openjdk.org/projects/leyden/

Word of caution
119
Re-measure for your use case!
Even with my examples measurements
might already produce different results due
to:
▪ Lambda Amazon Corretto Java 21
managed runtime minor version changes
▪ Lambda SnapStart snapshot create and
restore improvements
▪ Firecracker VM improvements
▪ GraalVM (major and minor version) and
Native Image improvements
▪ There are still servers behind Lambda
▪ Java Memory Model impact (L or RAM
caches hits and misses)

FAQ Ask me Anything
120

Thank you

High performance Serverless Java on AWS at Froscon 2024

Recommended

More Related Content

Similar to High performance Serverless Java on AWS at Froscon 2024 (20)

More from Vadym Kazulkin (14)

Recently uploaded (20)

High performance Serverless Java on AWS at Froscon 2024