Tuning Java GC to resolve performance issues

Editor's Notes

• #2: [email protected]
• #3: Before we start let me introduce myself. My name is Sergey, and for the past 4 years I have been working in Epam for Barclays Investment Bank. I have started my career as a .NET developer, and have switched to Java more than 3 years ago. Our job in Barclays is to valuate risks for financial derivatives, and we use Java a lot to accomplish this goal.My presentation is going to consist of 2 parts. In the first half of the presentation I am going to provide you some boring theory, and in the second part we are going to have some coding fun.
• #4: There are several reasons why I decided to make this presentation. The first reason is some of my colleagues who have an opinion that they don’t have to know all this stuff. Their opinion is based on the argument that they are users of Java and don’t have to know how does it work. Some of them are saying “if I drive a car then I don’t have to know how it works under the hood”. Today I will try to prove why this is not the case for JVM, and why every decent developer has to know this. Now I have a question to the audience: please raise your hand those who know how many garbage collectors there are in java. … So the second reason is that many people that I know believe that there is only one garbage collector in the JVM which is essentially a black box for them. And the third reason is a real production issue that we had which could not be resolved without all this knowledge.
• #5: The problem we had in production is the following. We developed a standalone RESTful java application and successfully tested it in staging. However, after releasing it to production the client started getting ConnectionRefusedException randomly, which was not reproducible in other environments. Question to the audience: how would you start investigating this kind of a problem?
• #6: The issue has been identified after we had a look at the GC log where we found long full stop-the-world GC pauses around 40 seconds each. During each such pause all the application threads are stopped for 40 seconds, and application becomes completely unresponsive. As a result of that Socket.accept() method is not called on incoming connections and they line up in the operating system TCP queue. Since that queue has limited size it eventually builds up and all the subsequent incoming connections are refused. The mystery is unraveled!
• #7: Before we start looking at different garbage collectors I would like to make a brief recap of the heap structure in JVM. As you probably know the heap in Java is divided into young generation and old generation, and the reason for that is that most of the objects die young. The diagram you are seeing on my screen was taken from one of the articles by Oracle and represents the average lifetime statistics of the objects in bytes. As you can see, most of the objects live for a very short period of time and can be reclaimed immediately after they are out of scope. As an example you can think of iterators, builders and even streams which are all basically temporary objects.
• #8: They all go to the Eden area within young generation until Eden is full. When Eden becomes full, garbage collector chimes in, stops all the application threads, and removes most of the temporary objects. This is called minor collection. Those who survive are moved to one of the survival spaces. The other survival space remains empty. After each subsequent collection these survived objects are moved back and forth between two survival spaces. If they are still alive even after some number of cycles, they go to much bigger so called old or tenured generation, since they are most likely needed for a long time if not for the whole lifetime of the application. When old generation gets full, major collection takes place. There is also a small region within Eden called thread local allocation buffer, which is aimed to reduce contention for Eden between application threads, but we will discuss it later.
• #9: Therefore, there are two cycles that all the garbage collectors have. The minor GC cycle when young generation is collected and when all the application threads are stopped regardless of the garbage collector you use, and a major GC cycle which works differently for different garbage collectors.
• #10: This whole memory model is essentially a tradeoff, and it’s important to keep the right balance between the sizes of the young generation and the old generation. The bigger your young generation is, the less frequent minor GC cycles are experienced. The bigger old generation size is, the longer pauses will be during the full GC cycles. There are several flags that are aimed to control this balance between two generations. You can specify the absolute size of the young generation within the entire heap using these JVM parameters, which are basically lower bound, upper bound, and constant value for the young generation. Alternatively you can specify the ratio of the young generation to the old generation using NewRatio parameter. You can also disable automatic sizing of the generations via UseAdaptiveSizePolicy parameter. And sometimes it may be useful to see how JVM resizes generations in the log records that can be enabled using PrintAdaptiveSizePolicy parameter.
• #11: One of the remarkable things about the young generation is that it never gets fragmented due to its structure, which is not the case for the old generation. You can also tweak different parts of the young generation if needed. For example you can tweak the ratio between Eden and survival spaces using these parameters, as well as the number of times the objects are ping-ponged between two survival spaces. There are two flags that can circumvent that behavior at either extreme. If you know that objects that survive a young collection will always be around for a long time, you can specify AlwaysTenure. The opposite to that is NeverTenure. You can also monitor how these numbers change if you enable corresponding logging using PrintTenuringDistribution. Sometimes you may need to increase the size of Thread Local Allocation Buffer because it’s pretty small by default, and if you allocate large objects that can’t fit into TLAB they go directly to Eden or even to the old generation, which can significantly impact your application’s performance.
• #12: It also worth mentioning permanent generation which has been replaced with metaspace in Java 8. The purpose of this area is to hold class metadata and it behaves like a separate heap instance. The problem is that when you have a very excessive class loading and it appears to be too small, the JVM may decide to resize it which causes full garbage collection. This can be avoided by increasing permgen or metaspace size in advance to avoid unnecessary resizings.
• #13: Now after I have reminded you the structure of the heap, we can discuss the available garbage collectors. All the garbage collectors have the same three functions: Scan the object graph and find objects that are not in use anymore Reclaim these unused objects And compact the heap if it gets fragmented.
• #14: All the garbage collectors also divide into two categories: So called stop-the-world garbage collectors that stop all of the application threads while collecting the old generation And so called concurrent (not to be confused with parallel) garbage collectors that can collect the old generation while application threads are working without stopping them. The representatives of the long pause collectors are serial and throughput collector, the latter one is also called parallel collector. And there are two concurrent low pause garbage collectors: G1 and CMS.
• #15: The simplest one is the serial collector which is enabled by default on all machines that have one CPU or running 32-bit JVM on Windows. It can be also enabled explicitly using UseSerialGC parameter. It is usually a good choice for the heaps less than 100MB. As you might guessed it’s running in a single thread and is triggered once young or old generation is full or too fragmented so that no more objects can be allocated.
• #16: An enhancement of the serial collector is the throughput collector which is also called parallel collector. It is triggered in the same way as the serial collector, but the collection is done by multiple threads. By default it does only parallel minor GC, but since Java 7u4 major GC is done in parallel as well. Both minor and major parallel collections can be enabled using UseParallelGC and UseParallelOldGC JVM flags. This collector is used by default on all the multi-CPU Unix machines and all the 64bit JVMs. The number of threads can be tuned by ParallelGCThreads parameter.
• #17: There are three goals that Parallel Collector tries to achieve from performance prospective: The highest priority is given to the GC pauses. By default the maximum GC pause is unlimited, but if it’s specified then parallel collector will decrease heap size until the expected pause time is reached. Once the first goal is met, parallel collector tries to minimise the total amount of time the application spends in GC comparing to the time spent executing the code. By default this value is 1% and JVM tries to improve it by increasing heap size which leads to less frequent GCs. Once both the first and the second goals are met parallel collector tries to minimise the total memory footprint.
• #18: CMS garbage collector is different from the previous two because it can do major GC collection without stopping application threads. The price you have to pay for it is higher overall CPU usage. Therefore it makes sense to enable CMS only if you have extra CPU cycles and your application is not CPU greedy. CMS can only be enabled explicitly if you specify both UseParNewGC and UseConcMarkSweepGC JVM flags together.
• #19: G1 garbage collector is the newest one and is considered to be a long term replacement for CMS. It is intended to be used with heaps bigger than 4GB and provides minimal GC pauses because it compacts heap on the fly. You can enable it explicitly using UseG1GC flag. It was considered experimental until Java 7u4 and has some performance improvements in Java 8.
• #20: What makes G1 very different from the other three collectors is that it divides the whole heap into small regions so that both young and old generations are not contiguous chunks of the heap anymore. G1 is an abbreviation for Garbage First and its name is based on the way it works, because it first tries to collect those regions which mostly contain garbage. Also it moves data from one region to the other along the way, which eliminates heap fragmentation on fly.
• #21: G1 has four cycles: Young collection – which can be done in parallel but stops all the application threads anyway Background concurrent cycle – which is used to identify garbage without stopping all the application threads for a long time Mixed collection – collects young generation and a number of regions from the old generation depending on the pause time it can afford And Full GC – which usually happens because of the evacuation failure and should be avoided as much as possible because it stops all the application threads.
• #22: There are several reasons why CMS and G1 collectors may fall into Full GC: One of the reasons is too excessive memory allocation which fills up the heap too quickly before concurrent cycle completes In case of CMS Full GC happens when heap becomes too fragmented so that there is no contiguous piece for a new object to be allocated and heap needs to be compacted Another thing that can become completely devastating for your application is swapping. Because GC needs to scan the entire heap this can cause memory pages to be copied back and forth between memory and hard drive on each GC. As a general rule of thumb the amount of memory consumed by your java application should not exceed free physical memory on your machine. This should be addressed even more carefully if you are running multiple JVMs on the same machine and the total amount of memory consumed exceeds the actual physical memory. The frequency of concurrent mode or evacuation failures can be decreased if you initiate concurrent cycles sooner so that they can complete before the heap fills up. The default threshold for CMS is 92% and for G1 is 45%. You can also increase the number of background threads, and increase the old generation size as well as whole heap size.
• #23: You can control your heap size by specifying the absolute initial and maximum size in bytes, but sometimes it’s useful to specify which fraction of the entire memory should be given to the heap. You can even override the physical memory available on your machine with MaxRAM parameter. Just for the reference I have also included the table with default values for different environments.
• #24: Question to the audience: have you ever seen this line in your code? This is almost always a bad idea because it causes full GC even if it’s not needed. It is only useful for very specific cases, for example before taking a heap dump, because you don’t usually want garbage to sneak in. Moreover, this line of code can be completely ignored by specifying DisableExplicitGC flag.
• #25: As a general rule of thumb you should always have GC logging enabled in your java applications. The impact it makes on application performance is almost negligible, but the benefit it gives you in case of production issues outweighs all the overheads. PrintGC and PrintGCDetails should be always enabled, and the others allow you to specify things such as log file path, log file rotation, log file size, and so on.
• #26: And the last thing you may hit occasionally is OutOfMemoryException. The thing is that it does not usually mean that there is no free space in your heap for an object to be allocated. Sometimes it happens when GC is just too excessive and does not really free too much space. It is triggered if all these three conditions are met 5 times in a row. It’s worth noting that on the fifth attempt GC cleans all the soft references as the last chance before throwing OutOfMemoryException, so this may free some of the memory so that the exception is not thrown, but the application still remains in a bad state when most of the CPU time is spent in GC.
• #27: Hopefully you didn’t get too bored.