Making sense of AWS Serverless operations- AWS User Group Nuremberg

Firdaws Aboulaye & Vadym Kazulkin ip.labs
Making sense of AWS Serverless
operations
Vadym Kazulkin, ip.labs , AWS User Group Nuremberg , February 11 2025

Vadym Kazulkin
ip.labs GmbH Bonn, Germany
Co-Organizer of the Java User Group
Bonn
v.kazulkin@gmail.com
@VKazulkin
https://dev.to/vkazulkin
https://github.com/Vadym79/
https://de.slideshare.net/VadymKazulkin/
https://www.linkedin.com/in/
vadymkazulkin
https://www.iplabs.de/
Contact

Image: burst.shopify.com/photos/a-look-across-the-landscape-with-view-of-the-sea
About ip.labs

Challenges and lessons learned
Architect with AWS Serverless quotas and technical concepts in
mind
General architectural decisions
SQS vs SNS vs Kinesis vs EventBridge
Aurora (Serverless) vs DynamoDB vs Aurora
DSQL
Challenging Serverless
observability

Serverless Application

Service Quotas

account and current region limits
Service Quotas

Service Quotas Request History

https://aws.amazon.com/de/blogs/compute/building-well-architected-serverless-applications-controlling-serverless-api-access-part-2/
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-request-throttling.html
• The throttle rate then determines how
many requests are allowed per second
• The throttle burst determines how
many additional requests are allowed
per second
API Gateway throttling-related settings
are applied in the following order:
• Per-client or per-method throttling
limits that you set for an API stage in a
usage plan
• Per-method throttling limits that you
set for an API stage
• Account-level throttling per Region
• AWS Regional throttling
Token bucket algorithm
API Gateway Token Bucket Algorithm

Quota Description Value Adjustable
Default
throughput /
Throttle rate
The maximum number of requests per
second that your APIs can receive
10.000
Throttle burst
rate
The maximum number of additional
requests per second that you can send
in one burst
5.000
API Gateway Important Service Quotas

Quota Description Value Adjustabl
e
Mitigation
Max timeout The maximum integration
timeout in milliseconds
29 sec 1) Increase the limit
2) Lambda Function
URL with response
streaming
API Payload
size
Maximum payload size for
non WebSocket API
10 MB 1)The client makes an
HTTP GET request to
API Gateway, and the
Lambda function
generates and returns
a presigned S3 URL
2)The client uploads
the image to S3
API Gateway Important Service Quotas

https://docs.aws.amazon.com/lambda/latest/dg/lambda-concurrency.html
Concurrency is the number of in-flight requests your AWS Lambda function is
handling at the same time
Lambda Concurrency

Quota Description Value Adj
usta
ble
Mitigation
Concurrent
executions/
Concurrency
limit
The maximum number of
events that functions can
process simultaneously in
the current region
1.000 Rearchitect
Burst
Concurrency
Limit
After the initial burst,
concurrency scales by
1000 executions every 10
seconds up to your
account concurrency limit.
Each function within an
account now scales
independently from each
• US West (Oregon), US
East (N. Virginia), Europe
(Ireland)=3.000
• Asia Pacific (Tokyo),
Europe (Frankfurt), US
East (Ohio)=1000
• All other Regions=500
Use
provisioned
concurrency
Lambda Important Service Quotas New

https://aws.amazon.com/de/blogs/compute/understanding-aws-lambdas-invoke-throttle-limits/
Lambda Concurrency

https://aws.amazon.com/de/blogs/aws/aws-lambda-functions-now-scale-12-times-faster-when-handling-high-volume-requests/
Lambda Concurrency and throttling

https://aws.amazon.com/de/blogs/compute/understanding-aws-lambdas-invoke-throttle-limits/
Lambda concurrency limit is a limit on the
simultaneous in-flight invocations allowed
at the same time
Transaction per second (TPS) =
concurrency / function duration in
seconds
Lambda Concurrency and TPS

Quota Descriptio
n
Value Adjust
able
Mitigation
TPS
(Transaction
per Second)
The
maximum
number of
TPS
TPS = min(10 x
concurrency,
concurrency /
function duration
in seconds)
• If the function duration is
exactly 100ms (or 1/10th of a
second), both terms in the min
function are equal
• If the function duration is over
100ms, the second term is
lower and TPS is limited as per
concurrency/function duration
• If the function duration is
under 100ms, the first term is
lower and TPS is limited as per
10 x concurrency
Lambda Important Service Quotas

https://aws.amazon.com/de/blogs/compute/understanding-aws-lambdas-invoke-throttle-limits/ https://www.linkedin.com/pulse/how-aws-lambda-works-underneath-shwetabh-shekhar/
The TPS limit exists to protect the Invoke Data Plane from the high churn of
short-lived invocations. In case of short invocations of under 100ms, throughput
is capped as though the function duration is 100ms (at 10 x concurrency). This
implies that short lived invocations may be TPS limited, rather than concurrency
limited.
Lambda TPS Quota

Lambda Function level Concurrency

• Optimize for cost-performance
• Use AWS Lambda Power Tuning
• Reuse AWS Service clients/connections outside of the Lambda
handler
• Use a keep-alive directive to maintain persistent connections
• Use the newest version of AWS SDK of programming language of
your choice
• Import only dependencies that you need (especially from AWS SDK)
• Minimize dependencies and package size
• Implement (other) best practices to reduce cold starts
https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html
General Best Practices for using Lambda

Quota Description Value Adjustable Mitigation
Function
timeout
The maximum timeout
that you can configure
for a function
15 min
Synchronous
payload
The maximum size of an
incoming synchronous
invocation request or
outgoing response
6 MB For the Request:
• use API Gateway
service proxy to S3
• use pre-signed S3 URL and
upload directly to S3
For the Response:
Use response streaming (with
AWS Lambda Web Adapter )
https://theburningmonk.com/2020/04/hit-the-6mb-lambda-payload-limit-heres-what-you-can-do/
Lambda Important Service Quotas

https://aws.amazon.com/de/blogs/compute/introducing-aws-lambda-response-streaming/
You can use response streaming to send responses larger than Lambda’s 6 MB
response payload limit up to a soft limit of 20 MB.
• Response streaming currently supports the Node.js 14.x and subsequent
managed runtimes
• To indicate to the runtime that Lambda should stream your function’s
responses, you must wrap your function handler with the
streamifyResponse() decorator. This tells the runtime to use the correct
stream logic path, allowing the function to stream responses
exports.handler = awslambda.streamifyResponse(
async (event, responseStream, context) => {
responseStream.setContentType(“text/plain”);
responseStream.write(“Hello, world!”);
responseStream.end();});
Lambda Response Streaming

Quota Description Value Adju
stabl
e
Mitigation
Table-level
read/wrtie
throughput limit
read/write throughput
allocated for a table or global
secondary index
40.000 RCU/
40.000 WCU
Ask for quote
increase
Table-Level burst
capacity for
provisioned
capacity mode
During an occasional burst of
read or write activity, these
extra capacity units can be
consumed quickly
up to 300
seconds of
unused RCUs
and WCUs
Partition-level
read/write
throughput
read/write throughput
allocated for a partition
3000 RCU
/1000 WCU
Use best
practices to
avoid hot
partition
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#default-limits-throughput-capacity-modes
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html#bp-partition-key-throughput-bursting
DynamoDB Important Service Quotas

On-Demand Capacity Mode is ideal for:
• Unknown workloads
• Frequently idle workloads
• Staging and (individual) test environments
• Unpredictable application traffic
• Low management overhead (truly serverless mode) is preferable
DynamoDB Provisioned Throughput vs On-
Demand Capacity Mode

Initial throughput for
on-demand capacity
mode
Initial throughput for on-
demand capacity mode
See futher
details
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#default-limits-throughput-capacity-modes
DynamoDB Important Service Quotas

Newly created table with on-demand capacity mode:
• Enables newly created on-demand tables to serve up to 4,000 WCUs or
12,000 RCUs
• If you exceed double your previous traffic's peak within 30 minutes,
then you might experience throttling
• Solutions for pre-warming:
• Use pre-warming Amazon DynamoDB tables/indeces with warm
throughput
• Performing the load test on your own
• Creating table in provisioned mode with high enough WCUs/RCUs
and then switch to on-demand mode
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadWriteCapacityMode.html#HowItWorks.InitialThroughput
https://aws.amazon.com/blogs/database/pre-warming-amazon-dynamodb-tables-with-warm-throughput/
Initial Throughput for DynamoDB On-Demand Capacity
Mode

• Use high-cardinality attributes
• These are attributes that have distinct values for each item, like email_id,
employee_no, customer_id, session_id, order_id, and so on
• Use composite attributes
• Try to combine more than one attribute to form a unique key, if that meets your
access pattern. For example, consider an orders table with
customerid#productid#countrycode as the partition key and order_date as the
sort key, where the symbol # is used to split different field
• Add random numbers or digits from a predetermined range for write-heavy
use cases
• Suppose that you expect a large volume of writes for a partition key (for
example, greater than 1000K writes per second). In this case, use an additional
prefix or suffix (a fixed number from predetermined range, say 0–9) and add it to
the partition key, like InvoiceNumber#Random(0-N)
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-uniform-load.html
https://aws.amazon.com/de/blogs/database/choosing-the-right-dynamodb-partition-key/
Recommendation for DynamoDB partition keys

Serverless Application with Aurora (Serverless) instead of
DynamoDB

Data API HTTP
request body size
The maximum size allowed for the
HTTP request body
4 Megabytes
Data API maximum
result set size
The maximum size of the
database result set that can be
returned by the Data API.
1 Megabytes
Data API maximum
size of JSON
response string
The maximum size of the
simplified JSON response string
returned by the RDS Data API.
10 Megabytes
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/CHAP_Limits.html
Aurora Serverless v2 Important Service Quotas

Data API
requests
per second
requests to the Data API
per second allowed
The max_connections value for
Aurora Serverless v2DB
instances is based on the
memory size derived from the
maximum ACUs.
However, when you specify a
minimum capacity of 0.5 ACUs
on PostgreSQL-compatible DB
instances, the maximum value
of max_connections is capped
at 2,000.
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.setting-capacity.html#aurora-serverless-v2.max-connections

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.setting-capacity.html#aurora-serverless-v2.max-connections

e
Throughput
per Standard
Queue
Standard queues support a nearly unlimited
number of transactions per second (TPS) per
API action.
Nearly
unlimited
SQS (Standard) Important Service Quotas

https://aws.amazon.com/about-aws/whats-new/2023/11/aws-lambda-polling-scale-rate-sqs-event-source/?nc1=h_ls
https://aws.amazon.com/blogs/compute/introducing-faster-polling-scale-up-for-aws-lambda-functions-configured-with-amazon-sqs/
• When a Lambda function subscribes to an SQS
queue, Lambda polls the queue as it waits for
messages to arrive. It consumes messages in
batches, starting with 5 functions at a time
• If there are more messages in the queue,
Lambda adds up to 300 functions/concurrent
executions per minute, up to 1,000 functions
(or up to your account concurrency limit), to
consume those messages from the SQS queue
• This scaling behavior is managed by AWS and
cannot be modified
• To process more messages, you can optimize
your Lambda configuration for higher
throughput
Lambda scaling with SQS standard queues

https://aws.amazon.com/de/blogs/compute/understanding-how-aws-lambda-scales-when-subscribed-to-amazon-sqs-queues/
https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html#services-sqs-batchfailurereporting
• Increase the allocated memory for your Lambda
function
• Optimize batching behavior:
• by default, Lambda batches up to
10 messages in a queue to process them
during a single Lambda execution. You can
increase this number up to 10,000
messages, or up to 6 MB of messages in a
single batch for standard SQS queues
• If each payload size is 256KB (the maximum
message size for SQS), Lambda can only
take 23 messages per batch, regardless of
the batch size setting
• Implement partial batch responses
Lambda scaling with SQS standard queues

e
Throughput
per Standard
Queue
Standard queues support a nearly unlimited
number of transactions per second (TPS) per
API action.
Nearly
unlimited
Message size The size of a message 256KB
SQS (Standard) Important Service Quotas

The BatchWriteItem operation puts
or deletes multiple items in one or
more tables.
A single call to BatchWriteItem can
transmit up to 16MB of data over
the network, consisting of up to
25 item put or delete operations
use BatchWriteItem
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchWriteItem.html
Use BatchWriteItem request for storing to DynamoDB

https://aws.amazon.com/blogs/compute/solving-complex-ordering-challenges-with-amazon-sqs-fifo-queues/
https://jayendrapatil.com/aws-sqs-standard-vs-fifo-queue/
SQS Standard Queue SQS FIFO Queue
Ordering Best Effort Ordering First In First Out Ordering
within the Message Group
Delivery At Least Once Exactly once
SQS FIFO

Batched Message
Throughput for FIFO
Queues
The number of batched transactions
per second (TPS) for FIFO queues
3.000
In-Flight Messages
per FIFO Queue
The number of in-flight messages in a
FIFO queue
20.000
SQS (FIFO) Important Service Quotas

• Put DynamoDB Accelerator (DAX) or Momento in front of DynamoDB
• Requires putting Lambda behind VPC
• ElastiCache before Aurora Serverless
• Requires putting Lambda behind VPC
• No “pay as you go” pricing
• Enable API Gateway Caching
• Uses ElastiCache behind the scenes
• No “pay as you go” pricing for ElastiCache
• Use CloudFront (and its caching capabilities) in front of API Gateway
Other Optimizations: Caching

• Set meaningful timeouts
• For API Gateway, Lambda
• Retry with exponential backoff and jitter
• AWS SDK supports them out of the box
• Implement idempotency
• AWS Lambda Powertools (Java, Python, Typescript) supports
idempotency module
https://docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html
https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/
https://aws.amazon.com/de/blogs/architecture/exponential-backoff-and-jitter/
https://aws.amazon.com/blogs/compute/handling-lambda-functions-idempotency-with-aws-lambda-powertools/
Other Optimizations: Error Handling and Retries

Virtual Waiting Room
https://aws.amazon.com/solutions/implementations/virtual-waiting-room-on-aws/
AWS “Virtual Waiting Room” Solution

• Open-source project written in
Python that can be integrated into
existing applications
• Source code available on GitHub
• Different CloudFormation
Templates to choose from (from
minimal to extended solutions)
• Estimated costs for a 50,000-user
and a 100,000-user waiting room with
an event duration ranging 2-4 hours
• Virtual Waiting Room on AWS has
been load tested with a tool called
Locust. The simulated event sizes
ranged from 10,000 to 100,000 client
https://docs.aws.amazon.com/solutions/latest/virtual-waiting-room-on-aws/architecture-overview.html
https://docs.aws.amazon.com/pdfs/solutions/latest/virtual-waiting-room-on-aws/virtual-waiting-room-on-aws.pdf
AWS “Virtual Waiting Room” Solution

More Serverless services, more service quotas 
• CloudFront
• EventBridge
• SNS
• Kinesis
• StepFunctions
Services Quotas of other Serveress services

• Understand the concepts of distributed systems
• Token Bucket Algorithm
• How polling from the queue works
• Retries with (exponential) backoff pattern and jitter
• Understand the concepts of NoSQL databases
• Defining partition (and sort) keys and all sort of indexes
• Efficient data access pattern
• Understand individual service specific terms
• Concurrency, transactions per second (TPS)
• Throttle/Concurrency limit, burst limit
• Hot partitions
Understand Technical Concepts

• Know, understand and observe the service quotas
• Architect with service quotas in mind
• AWS adjusts them from time to time
• In case I’d like to request the quota increase, provide a valid
justification for the new desired value
• Service quotas are valid per AWS account (per region)
• Use different AWS accounts for development and testing
• Use different AWS accounts for independent
(micro-)services
• Separate AWS accounts on the team level
• Use AWS Organizations
General best practices for Service Quotas

Recent AWS Serverless Services Quota Increases

It’s also about latency

“Serverless latency is a Thing: Daniele Frasca (Seven.One Entertainment Group) & Luca Mezzalira (AWS)” https://www.youtube.com/watch?v=xQo9fb4p0eE
Serverless Latency

Challenges and lessons learned
Architect with AWS Serverless quotas and technical concepts in
mind
General architectural decisions
SQS vs SNS vs Kinesis vs EventBridge
Aurora (Serverless) vs DynamoDB vs Aurora
DSQL
Challenging Serverless observability

When to use SQS, SNS, Kinesis and EventBridge
https://www.serverlessguru.com/tips/sqs-vs-sns-vs-kinesis-vs-eventbridge

DynamoDB DynamoDB + DAX
Investment in
Knowledge
• Understanding of
NoSQL databases
• Understanding of
single-table design
principles
Same
Requires to put
Lambda into the VPC
to access the
database
DynamoDB

Aurora (Serverless v2)
+ RDS Proxy
Aurora (Serverless v2)
+ Data API
Investment in
Knowledge
Relational databases are
familiar to many
developers
Same
Engine Support MySQL and PostgreSQL MySQL and PostgreSQL
Code re-use Yes, only endpoint
changes
No, requires code re-write
Requires to put
Lambda into the
VPC to access the
database
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Concepts.Aurora_Fea_Regions_DB-eng.Feature.ServerlessV2.html
https://dev.to/aws-builders/data-api-for-amazon-aurora-serverless-v2-with-aws-sdk-for-java-part-1-introduction-and-set-up-of-the-sample-application-3g71
Aurora (Serverless v2 )

• Amazon Aurora DSQL the fastest serverless distributed SQL database
• It offers virtually unlimited scale, highest availability, and zero infrastructure
management.
• It can scale to meet any workload demand without database sharding
operational burden of patching, upgrades, maintenance downtime, and more.
• With its active-active distributed architecture, Aurora DSQL is designed for 99.99%
availability in single-Region configuration and 99.999% in multi-Region
configuration, with an innovative active-active, distributed architecture
Adding Aurora DSQL into the mix
https://aws.amazon.com/blogs/database/introducing-amazon-aurora-dsql/

Current architecture our image storage
solution

Serverless reduces the need for (readily
available) ops skills but increases the
demand for (less readily available)
distributed system design skills.
https://architectelevator.com/cloud/serverless-illusion/
Serverless challenges

79
Thank you
Amazon DevOps Guru for the Serverless Applications

Making sense of AWS Serverless operations- AWS User Group Nuremberg

Recommended

More Related Content

Similar to Making sense of AWS Serverless operations- AWS User Group Nuremberg (20)

More from Vadym Kazulkin (20)

Recently uploaded (20)

Making sense of AWS Serverless operations- AWS User Group Nuremberg

Editor's Notes