Performance of Microservice frameworks on different JVMs
Download as PPTX, PDF11 likes3,899 views
A lot is happening in world of JVMs lately. Oracle changed its support policy roadmap for the Oracle JDK. GraalVM has been open sourced. AdoptOpenJDK provides binaries and is supported by (among others) Azul Systems, IBM and Microsoft. Large software vendors provide their own supported OpenJDK distributions such as Amazon (Coretto), RedHat and SAP. Next to OpenJDK there are also different JVM implementations such as Eclipse OpenJ9, Azul Systems Zing and GraalVM (which allows creation of native images). Other variables include different versions of the JDK used and whether you are running the JDK directly on the OS or within a container. Next to that, JVMs support different garbage collection algorithms which influence your application behavior. There are many options for running your Java application and choosing the right ones matters! Performance is often an important factor to take into consideration when choosing your JVM. How do the different JVMs compare with respect to performance when running different Microservice implementations? Does a specific framework provide best performance on a specific JVM implementation? I've performed elaborate measures of (among other things) start-up times, response times, CPU usage, memory usage, garbage collection behavior for these different JVMs with several different frameworks such as Reactive Spring Boot, regular Spring Boot, MicroProfile, Quarkus, Vert.x, Akka. During this presentation I will describe the test setup used and will show you some remarkable differences between the different JVM implementations and Microservice frameworks. Also differences between running a JAR or a native image are shown and the effects of running inside a container. This will help choosing the JVM with the right characteristics for your specific use-case!
![Java versions
What happens when migrating from Java 8 to Java 11?
• Java 8 and 11 behave very similarly for every framework
JDK 11 is slightly slower than 8
• OpenJ9 benefits most from going to JDK 11
Especially for Spring Boot and Akka
Responsetime[ms]
Java version](https://image.slidesharecdn.com/performanceofmicroserviceframeworksondifferentjvmsdistribute-190531075612/85/Performance-of-Microservice-frameworks-on-different-JVMs-29-320.jpg)
Ad
More Related Content
What's hot (20)
Similar to Performance of Microservice frameworks on different JVMs (20)
Ad
More from Maarten Smeets (15)
Ad
Recently uploaded (20)
![Get & Download Wondershare Filmora Crack Latest [2025]](https://cdn.slidesharecdn.com/ss_thumbnails/revolutionizingresidentialwi-fi-250422112639-60fb726f-250429170801-59e1b240-thumbnail.jpg?width=560&fit=bounds)
![Download Wondershare Filmora Crack [2025] With Latest](https://cdn.slidesharecdn.com/ss_thumbnails/neo4j-howkgsareshapingthefutureofgenerativeaiatawssummitlondonapril2024-240426125209-2d9db05d-250419-250428115407-a04afffa-thumbnail.jpg?width=560&fit=bounds)
![Pixologic ZBrush Crack Plus Activation Key [Latest 2025] New Version](https://cdn.slidesharecdn.com/ss_thumbnails/fashionevolution2-250322112409-f76abaa7-250428124909-b51264ff-250504160528-fc2bb1c5-thumbnail.jpg?width=560&fit=bounds)
Performance of Microservice frameworks on different JVMs
- 1. Performance of Microservice frameworks on different JVMs CJIB / Maarten Smeets
- 2. Performance of Microservice frameworks on different JVMs
- 3. Agenda 1. Introduction 2. Microservice frameworks 3. Test setup 4. Results 5. Recommendations
- 4. Who am I? Who is Maarten? • Software architect at AMIS / Conclusion • Several certifications SOA, BPM, MCS, Java, SQL, PL/SQL, Mule, AWS, etc • Enthusiastic blogger http://javaoraclesoa.blogspot.com @MaartenSmeetsNL https://nl.linkedin.com/in/smeetsm [email protected]
- 5. What is the CJIB • The Central Judicial Collection Agency part of the Ministry of Justice and Security in the Netherlands • The CJIB is responsible for collecting a range of different fines, such as traffic fines and punitive orders. • Works together with EU Member States when it comes to collecting fines. • Plays a key enforcement role in decisions relating to criminal matters, such as • court rulings • decisions made by one of the Public Prosecution Service’s public prosecutors • Located in Leeuwarden, Friesland Where do I work?
- 6. Boete voor verkeersovertreding 9.223.477 Leges kostenveroordeling 3.846 CJIB: Kengetallen 2017 Door rechter opgelegde boete 57.900 Schadevergoedingsmaatregel 13.563 Hulp bij problematische schuldsituatie Transactievoorstel 4.575Principale vrijheidsstraffen 13.485 Bestuursrechtelijke premies 2.081.270 OM-afdoening 284.642 Coördinatie van taakstraffen 36.630 Inkomende Europese boetes 1.038 Uitgaande Europese boetes 49.766 Ontnemingsmaatregel 1.690 Voorwaardelijke invrijheidstelling 1.043 Bestuurlijke boetes 40.608 Toezicht 15.021 Omgezette taakstraffen 7.657 Jeugdtoezicht 5.258
- 7. The CJIB and Java • 1400 people. ICT department of around 325 people. 100 Java developers • 30 teams using Scrum and SAFe. Tight integration between business and IT • Solid CI/CD pipelines and release train • Automated testing using Cucumber, Gherkin • Code quality checks using SonarQube • Bamboo, Puppet, Git, Maven, Vault • Running on Redhat 7, OpenJDK 8 with Spring Boot Microservices on Jetty • Innovation lab • Blockchain • Machine learning CJIB ICT
- 8. Disclaimer • The performance tests mentioned in this presentation were conducted with intention to obtain information on what performance differences can be expected from running various frameworks in various JVMs using various garbage collection algorithms. A best effort has been made to conduct an unbiased test. Still, performance depends on many parameters such as hardware, specifics of the framework implementation, the usage of different back-ends, concurrency, versions of libraries, OS, virtualization and various other factors. I cannot provide any guarantees that the same values will be achieved with other than tested configurations. The use of these results is at your own risk. I shall in no case accept liability for any loss resulting from the use of the results or decisions based on them.
- 10. Microservice frameworks Spring Boot • Spring Boot makes it easy to create stand-alone, production-grade Spring based Applications that you can "just run". • An opinionated view of the Spring platform and third-party libraries so you can get started with minimum fuss. Most Spring Boot applications need very little Spring configuration. Application.java HelloController.java
- 11. Microservice frameworks Spring Boot Reactive / WebFlux • Spring WebFlux is fully non-blocking, supports Reactive Streams back pressure, and runs on such servers as Netty, Undertow, and Servlet 3.1+ containers. GreetingRouter.java GreetingHandler.java
- 12. Microservice frameworks Spring Fu • Spring Fu is an incubator for Kofu (Ko for Kotlin, fu for functional), which provides a Kotlin API to configure Spring Boot applications programmatically. Spring Fu allows for native compilation on GraalVM while ‘regular’ Spring does not Application.kt
- 13. Microservice frameworks Vert.x • Eclipse Vert.x is event driven and non blocking. This means your app can handle a lot of concurrency using a small number of kernel threads. Vert.x lets your app scale with minimal hardware. • You can use Vert.x with multiple languages including Java, JavaScript, Groovy, Ruby, Ceylon, Scala and Kotlin. • Vert.x is flexible and unopiniated Application.java RestServiceVerticle.java
- 14. Microservice frameworks Akka • Akka is a toolkit for building highly concurrent, distributed, and resilient message-driven applications for Java and Scala GreetingActor.javaQuickstartServer.java
- 15. Microservice frameworks Quarkus • A Kubernetes Native Java stack tailored for GraalVM & OpenJDK HotSpot crafted from the best of breed Java libraries and standards • Extensions configure, boot and integrate a framework or technology into your Quarkus application. They also provide the right information to GraalVM for your application to compile natively GreetingController.java
- 16. Microservice frameworks Microprofile • An open forum to optimize Enterprise Java for a microservices architecture by innovating across multiple implementations and collaborating on common areas of interest with a goal of standardization. • Specifies the least number of Java Enterprise specifications APIs required to build a microservice.
- 17. No framework • Minimal implementation using Java SE code only • No reflective code and no frameworks easy native compilation Application.java
- 18. JVMs differ • Licensing / support • Memory usage • Garbage collection algorithms • Start up, class loading • Other features JVMs
- 19. Test setup Framework versions used Hardware used • Intel Core i7-8700 • hexa-core • 12 threads • Max 4.6GHz • 32Gb DDR4L (2400MHz) OS used • Running Linux Mint 19.1 (Tessa) based on Ubuntu 18.04LTS (Bionic Beaver) • Docker server 18.06.1-ce client 18.09.2 Framework Version HTTP server Spring Boot 2.1.4 Tomcat Spring Fu 0.0.5 Reactor Netty WebFlux 2.1.4 Reactor Netty Akka 2.12 10.1.8 Akka HTTP Open Liberty 19.0.0.4+ Open Liberty Vert.x 3.7.0 Vert.x Quarkus 0.15.0 Netty My laptop
- 20. What did I do? • Create minimal but comparable implementations for every framework Java, Kotlin • Create a script to loop over JVMs, Microservice implementations, GC-algorithms Bash • Create multithreaded load generators and compare results Python, Node/JavaScript • Containerize the implementations; makes testing JVMs and resource isolation easy Docker • Summarize results (determine average, standard deviation, Prometheus results) Bash: awk, sed, curl • Run the setup under various conditions Mostly 15 minutes per framework per JVM per variable (weeks of data) • Visualize results Python: pandas, numpy, pyplot Test setup
- 21. Test setup Docker base image with JVM Microservice framework fat JAR Load generator (Python) Build and run container JVM + framework Load generation runtest.sh Loop over JVMs and frameworks Start processes, clean up and summarize results outputfile.txt groupedbarplot.py Data validation Data description Data visualization Generate and summarize data Validate and visualize results.txt (measures per JVM per framework) messages
- 22. Resource isolation User processes Load generators JVM process 1 32
- 23. Running in a container • The kernel is shared among processes • The disk is shared with the host • The load generators and the JVM might compete for resources • Running inside a container has a performance cost
- 24. • Images available on Docker Hub – OpenJDK – AdoptOpenJDK – Oracle GraalVM – Amazon Corretto – Eclipse OpenJ9 – Azul Zulu • Images not available on Docker Hub (due to license restrictions) – Oracle JDK – Azul Zing Running in a container
- 25. Containers Results • Hosting on Docker is slower than hosting locally • Docker to Docker is not faster than local to Docker • Everything outside a container is fastest
- 26. Microservice frameworks Which framework gives best response times • Akka gives worst performance. Vert.x best. • Reactive frameworks (Akka, Vert.x, WebFlux) do not outperform non-reactive frameworks (Microprofile, Quarkus, Spring Boot, Spring Fu) Spring Fu
- 29. Java versions What happens when migrating from Java 8 to Java 11? • Java 8 and 11 behave very similarly for every framework JDK 11 is slightly slower than 8 • OpenJ9 benefits most from going to JDK 11 Especially for Spring Boot and Akka Responsetime[ms] Java version
- 30. JVMs Which JVM performs best for which framework? • OpenJDK and Oracle JDK perform similarly for every framework (no consistent winner) • OpenJ9 does worst for every framework followed by Zing • For Vert.x the differences between JVMs are smallest Zing (JDK 8) does best here • Substrate VM (native compilation) for Quarkus gives worst performance for Quarkus +
- 31. Application startup Spring Boot, 2Gb heap, default GC
- 32. Application startup Startup time Quarkus per JVM Native compilation and startup • Native compilation (Substrate VM) greatly reduces start-up time • Oracle JDK is fastest to start of the JIT compilers followed closely by OpenJDK • Zing and OpenJ9 are relatively slow to start Have not looked at Zing Compile Stashing and ReadyNow! features • Quarkus starts approximately 10x faster than Spring Boot!
- 33. • OpenJ9 Balanced Divide memory in individually managed blocks. Good for NUMA • OpenJ9 Metronone Garbage collection occurs in small interruptible steps • OpenJ9 OptAvgPause Uses concurrent mark and sweep phases. Reduces pause times • OpenJ9 OptThruPut Optimizes on throughput but long pause times (app freezes) • Shenandoah GC (Java 12) A low pause time algorithm which does several tasks concurrently No increased pause times with a larger heap • ZGC (Java 12) Scalable (concurrent) low latency garbage collector • Zing C4 GC Continuously Concurrent Compacting Collector Pauseless garbage collection • OpenJDK G1GC (default Java 9+) Compact free memory space without lengthy pause times • OpenJDK Parallel (default Java 8) High-throughput GC which does not allow memory shrinking • OpenJDK ConcMarkSweep (EOL?) Designed for lower latency / pause times than other parallel collectors • OpenJDK Serial GC Single threaded GC which freezes the application during collection • OpenJ9 Generational Concurrent policy Minimize GC pause times without compromising throughput GC algorithms
- 34. GC algorithms How do GC algorithms influence response times (2Gb heap) • OpenJ9 did worst (with and without sharedclasses) Metronome does best for OpenJ9 • Every JVM (OpenJ9, Zing, OpenJDK) achieves similar performance for every available GC algorithm (at 2Gb heap!) • OpenJDK Serial GC did best OpenJDK OpenJ9 Zing
- 35. GC algorithms 20mb heap How do GC algorithms influence response times (20Mb heap) • GC algorithms influence the minimal amount of memory required to start the JVM OpenJ9 Metronome GC gave out of memory • When memory is limited • Do not use Shenandaoh (30ms) or parallel GC • OpenJ9 does better than AdoptOpenJDK • Azul Zing cheated! ‘Warning Maximum heap size rounded to 1024 MB’ Zing supports heap sizes from 1 GB to 8 TB • OpenJ9 with OptThruPu GC produces best performance on limited memory
- 36. GC algorithms: Prometheus metrics
- 37. Prometheus metrics When not to use Prometheus metrics • Prometheus metrics are not suitable to compare frameworks Every framework has a different implementation • Prometheus/Grafana are powerful to quickly look at many metrics and measures over time (dashboards) • Prometheus and Grafana are less useful if you want a single measure per test
- 38. Good to know Some challenges • Scoping. You can not test every situation for every framework. • Testing • Tests have long running times. 15 minutes per framework or GC algorithm per JVM • Writing a performance-test script which produces reproduceable results • GraalVM • 1.0 RC16 and 19.0.0 native image support contains breaking changes • Spring Framework 5.3 will have OOTB support • Is only available as Java 8 • Quarkus makes native images more easy! • Open Liberty 19.0.0.5+ will run on GraalVM. Earlier versions will not (reported the bug in 19.0.0.4 and it got quickly resolved)
- 39. Considerations Suggestions for further research • You should have used switch x or y with JVM z • You didn’t run statistic x or y to determine of the differences are significant • You only ran every test for 15 minutes (a couple of million requests) • You’re only looking at a minimal implementations. No one uses those. • You’re running the load test and JVM on the same machine • You have not looked at the differences in libraries in the container images • You should have included JVM x or Microservice framework y • You have only tested on hardware x. I’m using hardware y • You have not compared different servlet engines with the same application
- 40. • Low on memory: consider OpenJ9 • Is performance important? Don’t run in a container! • JDK 11 has slower start-up times and slightly worse performance than JDK 8 • OpenJDK variants (GraalVM, Corretto, AdoptOpenJDK, Zulu) and Oracle JDK perform pretty similarly. Does not matter much which one (when looking at performance) • Native images (Quarkus, Substrate VM) have way faster startup time but worse response times. Choose which one matters most to you Recommendations
- 41. Choices the CJIB made Framework: Spring Boot Quick for development and can run standalone Jetty servlet engine Efficient in memory usage and performance OpenJDK 8 on RedHat 7 (currently not in a container) CJIB already has RedHat licenses for support RedHat is steward for OpenJDK 8 and 11 Spring Boot runs well on OpenJDK Garbage Collection Default as long as no performance issues Choices the CJIB made
- 42. Want more answers? Suggestions • Sources: https://github.com/MaartenSmeets/jvmperformance • Get help from the JVM suppliers! Eclipse OpenJ9, Oracle GraalVM, Azul Systems got in touch and provided valuable feedback • Do your own tests. Your environment and application is unique. Results might differ
Editor's Notes
- #7: Thema: Het CJIB – Kengetallen Het innen van verkeersboetes is nog altijd een van de grootste taken van het CJIB. In 2017 inde het CJIB ruim 9,2 miljoen verkeersboetes. Naast verkeersboetes int het CJIB bijvoorbeeld ook strafrechtboetes, dit zijn door de rechter opgelegde boetes; bestuursrechtelijke premies, zoals het innen van de premie voor de zorgverzekering; of een bestuurlijke boete, bijvoorbeeld een boete die een bedrijf krijgt van de arbeidsinspectie als het niet voldoet aan veiligheidsnormen. Het CJIB kent inmiddels uiteenlopende taken op het gebied van Innen & Incasseren en Coördineren & Informeren. (Bron cijfers: CJIB, Management Informatie 23 februari 2018).