I see the the PS survivor space is almost full (98 %) most of the time for my application. I don't know what is PS survivor space . Is this normal ?
What should be done in such scenarios ?
First, see e.g. here : What is a survivor space?
Usually, there are 2 survivor spaces in the YoungGeneration part of the heap (e.g. for the Hostpot VM ). They are there to allow objects to mature before promoting them to the Old Generation. Because its more expensive to cleanup the old generation.
Collect some statistics to see if the survivor spaces are really full most of the time. You should see that one is always empty while the other one is being populated. See e.g. this question for collecting GC stats.
Once you have the data, look for:
survivor space overflow - this occurs when the survivor space is too small to allow the objects to mature between YoungHeap collections and the objects are overflowing to the OldGeneration without having time to mature (and die before being promoted).
also, monitor tenuring distribution with -XX:+PrintTenuringDistribution. To see how fast are the objects maturing.
UPDATE: Read the Hotspot Memory Management Whitepaper and see the section Serial Collector, there is a nice explanations of the Survivor spaces:
Note: If the To space becomes full, the live objects from Eden or From that have not been
copied to it are tenured, regardless of how many young generation collections they have survived. Any
objects remaining in Eden or the From space after live objects have been copied are, by definition, not live, and they do not need to be examined.
Related
As part of the question in Java 11 GC logging I am struggling to understand what the numbers actually mean.
For example:
[2020-07-14T10:01:14.791-0400][gc ] GC(353) Pause Young (Normal) (G1 Evacuation Pause) 163M->16M(248M) 1.689ms
[2020-07-14T10:01:14.790-0400][gc,heap ] GC(353) Eden regions: 147->0(147)
[2020-07-14T10:01:14.790-0400][gc,heap ] GC(353) Survivor regions: 1->1(19)
[2020-07-14T10:01:14.790-0400][gc,heap ] GC(353) Old regions: 16->16
[2020-07-14T10:01:14.790-0400][gc,heap ] GC(353) Humongous regions: 1->1
I know that 147->0 is before/after collection, but what is the unit here and for the ones below? As I see it, is that the whole young generation is reduced from 163M to 16M , it also looks like this happens almost entirely within the Eden regions - so the objects already went out of scope before even moving to the survivor space?
what is the unit here
A region. Region size varies based on heap size or an explicit setting.
it also looks like this happens almost entirely within the Eden regions - so the objects already went out of scope before even moving to the survivor space?
Most of them, a small amount might still trickle into later generations but on the other hand those regions may also contain now-dead objects that can be collected so it's mostly in equilibrium with only a very small flow towards the old generation. This kind of behavior is what makes generational collectors so efficient.
jmap can know size of each generation, but I want to monitor my java process real time.
With jmx, MemoryMXBean.getHeapMemoryUsage().getUsed() can get the total heap size. But I cannot find any method to get:
Size of New (Young) and Tenured (Old) generations;
Size of Eden and each Survivor in New generations.
You can use ManagementFactory.getMemoryPoolMXBeans(). It shows all the memory regions. Depending on the GC you used, the names differ. pool.getCollectionUsage().getInit() gives the initial size of a pool. pool.getName() is for example 'G1 Eden Space' or 'G1 Old Gen' or 'G1 Survivor Space' if you use G1 GC.
pool.getUsage().getUsed() returns as well the amount of used memory for that region.
I am using JDK1.6.0_16 JVM for the java application that is hosted on an Linux Intel procesor 80 cores machine.
while starting the Java application I have only two options configured
-Xms2048m -Xmx8000m in the JVM Options (after java command).
I see that PS Old Gen is calculated as 5.21G and PS Eden is calcuated 2.6G but the PS Survivor space is 25MB.
I have exactly same JVM in production and in that PS Survivor Space size is shown as 888MB. I am seeing these sizes in java mission control Memory tab.
The cache size (output of /proc/cpuinfo) is showing 24656 in both UAT and production boxes.
Dont think it will make any difference for JVM but still mentioning that the there was very low load on machine at the time of starting JVM.
Can you please advise what parameters does JVM consider for calculating the PS Survivor Space size?
From oracle gc tuning article 1 and article 2 :
Survivor Space Sizing
You can use the parameter SurvivorRatio can be used to tune the size of the survivor spaces, but this is often not important for performance. For example, -XX:SurvivorRatio=6 sets the ratio between eden and a survivor space to 1:6.
In other words, each survivor space will be one-sixth the size of eden, and thus one-eighth the size of the young generation (not one-seventh, because there are two survivor spaces).
If survivor spaces are too small, copying collection overflows directly into the tenured generation. If survivor spaces are too large, they will be uselessly empty.
The NewSize and MaxNewSize parameters control the new generation’s minimum and maximum size. Regulate the new generation size by setting these parameters equal. The bigger the younger generation, the less often minor collections occur.
NewRatio: The size of the young generation relative to the old generation is controlled by NewRatio. For example, setting -XX:NewRatio=3 means that the ratio between the old and young generation is 1:3, the combined size of eden and the survivor spaces will be fourth of the heap.
As correctly quoted by Peter Lawrey, setting survivor depends on type of your application. From gc tuning article by Oracle, here are the guidelines.
First decide the maximum heap size you can afford to give the virtual machine. Then plot your performance metric against young generation sizes to find the best setting
If the total heap size is fixed, then increasing the young generation size requires reducing the tenured generation size. Keep the tenured generation large enough to hold all the live data used by the application at any given time, plus some amount of slack space (10 to 20% or more).
Subject to the previously stated constraint on the tenured generation: Grant plenty of memory to the young generation and increase the young generation size as you increase the number of processors, because allocation can be parallelized. The default is calculated from NewRatio and the -Xmx setting
Can you please advise what parameters does JVM consider for calculating the PS Survivor Space size?
It must large enough to never actually fill up after a collection of the Eden space, otherwise you will get Full GCs which is undesirable.
What is an optimal Survivor space size depends on your application. I suggest you test you application under realistic loads with a larger Eden and Survivor space than you imagine useful and see how much of that space ever gets used and add 50% to 100% based on what you see is used.
The machine has 256G of physical memory out of which ~200G
The default heap size is 32 GB and I suggest you use this default unless you have a good reason to reduce it.
-XX:SurvivorRatio=1
This is usually a bad idea and having a high survivor ratio like 8 is usually better.
Setting the value to 8 didnt have any effect
Most likely you have a low allocation rate. I usually set a high Young space, alike -Xmn8g or even -Xmn24g but whether this is a good/bad idea depends on your application.
Please feel free to correct me if I am wrong. In JVM heap, there are two generations, old and young. When doing full GC, in old generation, there are heavy operations like compact spaces and fixing the hole, which will make JVM hang. And I find in young generation, a light weighted GC is applied, and there are another area called Eden involved in young generation from my search results. However, after search a lot of documents, I still have two confusions about GC in young generation,
In young generation, it seems GC does not work in the way which old generation GC works (i.e. old generation GC compact and fixing the hole)? If so, how did GC in young generation works?
What is Eden space and how this space is utilized in young generation? Appreciate if any document for a newbie could be recommended.
This is the single, most important diagram you have to memorize and understand:
(source: oracle.com)
It comes from Java SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning, one stop place to learn everything about GC internals. But to address your immediate questions:
Allocating new objects using new operator (almost) always happens in Eden space. But Eden is actually a stack. When you create new object needing N bytes, single pointer advances by N bytes on that stack and that's it. Allocating is that fast, no searching for free spot, compacting, whatever.
Of course this stack is not infinite, at some point we'll reach its end, triggering minor GC. Also most likely multiple objects are already garbage. So what JVM does in minor GC is the following:
traverse graph of objects starting from GC roots
copy all objects reachable from GC roots to one of survivor spaces (no gaps, we know all of them and this is a single process)
wipe out eden space (basically just moving this stack pointer back to 0)
In subsequent minor collections there are additional steps:
one of survivor spaces is examined as well. Live objects from both eden and one of survivor spaces are copied to second survivor space. This means there is always exactly one free survivor space.
So how are objects ending in tenured generation? First young objects are copied to one of survivor spaces. Then they are copied to the other and again and again. Once given object jumps back and forth too many times (configurable, 8 by default), it is promoted to tenured space.
Major GC runs when tenured space is full.
I want to monitor Full GC frequency in JMX. A MBean exposes GC count.
(cf. http://download.oracle.com/javase/1.5.0/docs/api/java/lang/management/GarbageCollectorMXBean.html - java.lang:type=GarbageCollector,name=).
The problem is that MBean does not distinguish between minor and full gc.
Does someone have an idea ?
Thanks.
Arnault
I'm not completely sure about this but I assume that the garbage collector that controls all the memory pools (at least the one for Old Gen) is the one used for major gc. e.g.: I have a JVM running with these 2 collectors:
PS MarkSweep
MemoryPoolNames: PS Eden Space, PS Survivor Space, PS Old Gen, PS Perm Gen
CollectionCount: 68
PS Scavenge
MemoryPoolNames: PS Eden Space, PS Survivor Space
CollectionCount: 2690
Taking this into account I would say, PS Scavenge is used for minor gc and PS MarkSweep for major gc.
UPDATE (based on #ajeanson comment, thanks for your feedback btw):
Effectively, the example I put in there was taken from the information exposed in the MXBeans of the JVM I was using. As you mentioned, these are GC algorithms, and the name the MXBean for the GC is using is based on the algorithm the GC is using. I've been looking for some more information about this; in this article http://download.oracle.com/javase/6/docs/technotes/guides/management/jconsole.html, reads the following:
The Java HotSpot VM defines two
generations: the young generation
(sometimes called the "nursery") and
the old generation. The young
generation consists of an "Eden space"
and two "survivor spaces." The VM
initially assigns all objects to the
Eden space, and most objects die
there. When it performs a minor GC,
the VM moves any remaining objects
from the Eden space to one of the
survivor spaces. The VM moves objects
that live long enough in the survivor
spaces to the "tenured" space in the
old generation. When the tenured
generation fills up, there is a full
GC that is often much slower because
it involves all live objects. The
permanent generation holds all the
reflective data of the virtual machine
itself, such as class and method
objects.
Taking a look at the collectionCount property on the MXBeans, in the case of my "PS MarkSweep" collector (the one managing the Old Generation pool), the collection count seems to increase only when I get a full GC in the verbose output. I might be wrong and maybe in some cases this Collector performs also minor GC, but I would need to run more tests to be totally sure about this.
Please, let me know if someone finds out something else or you have some more specific information about this issue as I'm quite interested in it.
it does ... have a look to the names e.g. ParNew, ConcurrentMarkSweep, .. etc.
some names are for minor gc, some for full gc,
Was looking for the same information and found out after reading https://docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html#BABFAFAE for JAVA 8 that some collectors can be used for both minor/full GCs (such as G1 or SerialGC) but some other collectors are for only minor or full GCs (such as ParNewGC, ConcMarkSweepGC).
And when you use the G1 for example, the two collectors used are quite explicit with their names and the one for full gc is the G1 Old Generation.
But, because the MXBean is missing the information about being minor or full, either:
you know the GC in use for your app and code accordingly your monitoring method knowing the collector names
or you start having like a map of all possibilities for your selected JVM version
I will, in my case, just print the collector name along with the time and count value and let the person reading those data make the analysis. In my case, the data will be graphed (Grafana)
Not sure if the newest JDK improve this...