I have not been able to find the type of GC that System.gc() recommends the JVM to do. Here, it is said that "the System.gc() method ... forces major collections", but this post implies a full gc is requested.
Could anyone clarify or point to documentation that spells this out? In other words, is it a major or full gc that is requested?
That depends on the JVM you're using.
Assuming you're using hotspot the behavior varies based on flags passed to it. By default it triggers a full stop-the-world GC, which will show up as the gc cause [Full GC (System.gc)] in the logs. With DisableExplicitGC it won't invoke any GC at all. If G1 or CMS are used then ExplicitGCInvokesConcurrent will change that behavior to initiate a concurrent old gen collection instead.
The major and minor terminology is not very useful anymore since GC cycles have become more nuanced.
If in doubt, enable GC logging and see for yourself.
Related
If my nodejs memory is reached upto 1.5GB and then i am not applying any load on it and the system is idle for 30 minutes then also garbage collector is not freeing the memory
It's impossible to say anything specific, like "why gc is not collecting the garbage" as you ask in the comments, when you say nothing about what your program is doing or how the code looks like.
I can only point you to good explanation of how GC works in Node and explain how to run GC manually to see if that helps. When you run node with the --expose-gc flag, then you can use:
global.gc();
in your code. You can try to run that code in a timeout, on a regular interval or at any other specific moment and see if that frees your memory. If it does, that would mean that the GC was indeed not running and that was the problem. If that doesn't free your memory that could mean that the problems is not with the GC not running, but rather not being able to free anything.
Memory not being freed after manual GC invocation would mean that you have some memory leak or that you use so much memory that cannot be freed by the GC.
If the memory is freed after running GC manually it could mean that it is not running by itself, maybe because you are doing a very long, blocking operation that doesn't give the event loop any chance to run. For example having a long running for or while loop could give such an effect.
Not knowing anything about your code or what it does, it's not possible to give you any more specific solution to your problem.
If you want to know how and when the GC in Node works, then there is some good documentation online.
There is a nice article by StrongLoop about how the GC works:
Node.js Performance Tip of the Week: Managing Garbage Collection
Also this article by Daniel Khan is worth reading:
Understanding Garbage Collection and hunting Memory Leaks in Node.js
It's possible that the GC is running but it can't free any memory because you have some memory leak. Without seeing an actual code or even an explanation of what it does it's realy impossible to say more.
Recently, I have installed https://github.com/lloyd/node-memwatch for development, to investigate how GC interacts with my program.
I have binded the event "stat", the arthor states that the event is triggered when GC is performed.
I've found that when the script with high load. The "stat" events are not triggered. I am not sure whether it implies GC is not performed, but it is a sign that GC may not have triggered.
In my production server, the loading is even a lot higher throughout the day. I am quite sure that GC has no chance to perform. The memory usage has no chance to decrease. It is just like memory leak.
Is my observation correct? Is GC not able to perform when there are high load continuously?
If so, should I use the exposed GC interface to force GC?
Is GC blocking? Should I perform GC more frequently so that GC will not block for a long time for each GC?
I know manual GC is not a good idea (There is someone opposing the idea of manual GC in node.js, but I cannot find the link for reference), but I am seeing that the memory usage is increasing continuously. It really needs to be solved.
There are 3 types of GC events in V8
kGCTypeMarkSweepCompact
kGCTypeScavenge
kGCTypeAll
V8 runs the scavenge event quite often, but only on newly created objects. During a heavy load the other types of GC may occur infrequently.
You can try running the NodeFly agent which uses nodefly-gcinfo module to track ongoing memory usage.
You can also call nodefly-gcinfo directly which has a callback that runs each time a GC event occurs:
require('nodefly-gcinfo').onGC(function(usage, type, flags){
console.log("GC Event Occurred");
console.log("Heap After GC:",usage, type, flags);
});
What's the Gambit-C's GC mechanism? I'm curious about this for making interactive app. I want to know whether it can avoid burst GC operation or not.
According to these threads:
https://mercure.iro.umontreal.ca/pipermail/gambit-list/2005-December/000521.html
https://mercure.iro.umontreal.ca/pipermail/gambit-list/2008-September/002645.html
Gambit has traditional stop-the-world GC at least until September 2008. People in thread recommended using pre-allocated object pooling to avoid GC operation itself. I couldn't find out about current implementation.
*It's hard to agree with the conversation. Because I can't pool object not written by myself and finally full-GC will happen at sometime by accumulated small/non-pooled temporary objects. But the method mentioned by #Gregory may help to avoid this problem. However, I wish incremental GC added to Gambit :)
According to http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Debugging#Garbage_collection_threshold gambit has some controls:
Garbage collection threshold
Pay attention to the runtime options h (maximum heapsize in kilobytes) and l (livepercent). See the reference manual for more information. Setting livepercent to five means that garbage collection will take place at the time that there are nineteen times more memory allocated for objects that should be garbage collected, than there is memory allocated for objects that should not. The reason the livepercent option is there, is to give a way to control how sparing/generous the garbage collector should be about memory consumption, vs. how heavy/light it should be in CPU load.
You can always force garbage collection by (##gc).
If you force garbage collection after some small number of operations, or schedule it near continuously, or set the livepercent to like 90 then presumably the gc will run frequently and not do very much on each run. This is likely to be more expensive overall, but avoid bursts of expense. You can then fairly easily budget for that expense to make the service fast despite.
As far as I know GC is used only when JVM needs more memory, but I'm not sure about it. So, please, someone suggest an answer to this question.
The garbage collection algorithm of Java is as I understand, pretty complex and not as straightforward. Also, there is more than algorithm available for GC, which can be chosen at VM launchtime with an argument passed to the JVM.
There's a FAQ about garbage collection here: http://www.oracle.com/technetwork/java/faq-140837.html
Oracle has also published an article "Tuning Garbage Collection with the 5.0 Java[tm] Virtual Machine" which contains deep insights into garbage collection, and will probably help you understand the matter better: http://www.oracle.com/technetwork/java/gc-tuning-5-138395.html
The JVM and java specs don't say anything about when garbage collection occurs, so its entirely up to the JVM implementors what policies they wish to use. Each JVM should probably have some documention about how it handles GC.
In general though, most JVMs will trigger GC when a memory allocation pushes the total amount of allocated memory above some threshold. There may be mulitple levels of gc (full vs partial/generational) that occur at different thresholds.
Garbage Collection is deliberately vaguely described in the Java Language Specification, to give the JVM implementors best working conditions to provide good garbage collectors.
Hence garbage collectors and their behaviour are extremely vendor dependent.
The simplest but least fun is the one that stops the whole program when needing to clean. Others are more sophisticated and run quietly along your program cleaning up as you go more or less aggressively.
The most fun way to investigate garbage collection is to run jvisualvm in the Sun 6 JDK. It allows you to see many, many internal things many relevant to garbage collection.
https://visualvm.dev.java.net/monitor_tab.html (but the newest version has plenty more)
In the older days garbage collector were empirical in nature. At some set interval or based on certain condition they would kick in and examine each of the object.
Modern days collectors are smarter in the sense that they differentiate based on the fact that objects are different lifespan. The objects are differentiated between young generation objects and tenured generation objects.
Memory is managed in pools according to generation. Whenever the young generation memory pool is filled, a minor collection happens. The surviving objects are moved to tenured generation memory pool. When the tenured generation memory pool gets filled a major collection happens. A third generation is kept which is known as permanent generation and may contain objects defining classes and methods.
I am trying to profile a specific page of my ASP.NET site to optimize memory usage, but the nature of .NET as a Garbage Collected language is making it tough to get a true picture of what how memory is used and released in the program.
Is there a perfmon counter or other method for profiling that will allow me to see not only how much memory is allocated, but also how much has been released by the program and is just waiting for garbage collection?
Actually nothing in the machine really knows what is waiting for garbage collection: garbage collection is precisely the process of figuring that out and releasing the memory corresponding to dead objects. At best, the GC will have that information only on some very specific instants in its cycle. The detection and release parts are often interleaved (this depends on the GC technology) so it is possible that the GC never has a full count of what could be freed.
For most GC, obtaining such an information is computationally expensive. If you are ready to spend a bit of CPU time on it (it will not be transparent to the application) then you can use GC.Collect() to force the GC to run, immediately followed by a call to GC.GetTotalMemory() to know how much memory has survived the GC. Note that forcing the GC could induce a noticeable pause, and may also decrease overall performance.
This is the "homemade" method; for a more serious analysis, try a dedicated profiler.
The best way that I have been able to profile memory is to use ANTS Profiler from RedGate. You can view a snapshot, what stage of the lifecycle it is in and more. Including actual object values.