I have a memory leak on my nodejs application. In order to resume the purpose of the application, it's an api called by an iOS application and a backoffice to administrate some content.
The application is in production and we experience some memory leak due to utilisation.
The memory on the server is always going up and never going down.
I try to analyze the problem using node-heapdump.
First of all, i see a big difference between the heap size of the snapshot given by node-heapdump and the size taken by the app in the memory (heap size ~ 30Mb and RAM size ~ 100Mb), where that difference came from ?
Then i see an increment of the heap size just by refreshing a home page who does not return anything.
Is anyone has an idea of where my problem could be ?
For information i use nodejs version 0.10.x and expressjs 4.0.0
Thanks in advance guys.
EDIT
I install memwatch-next and the leak event is raised.
The error i have is this one :
warning: possible EventEmitter memory leak detected. 11 leak listeners
added. Use emitter.setMaxListeners() to increase limit.
I try to set the defaultMaxListeners but when i stress the application the leak event is raised after sometime.
Does anyone knows what that error means ?
have a look at memwatch-next
I had similar issues with the memwatch package and switched to memwatch-next and it installed without the node-gyp error and produced worked. As for the difference between the RSS and the heapdump , I am in the same boat as you are.
Try to find memory leakage leak and stat from https://www.npmjs.com/package/memwatch.
Hope it would help.
I think you need this tool: easy-monitor
Might I recommend you try running the application with the --inspect argument, this will then allow you to attach Chrome dev tools and take memory snapshots. From here take one at startup one during testing and then one after you have finished testing the application (no more requests to the application but must still be running.) This will allow you to see what exactly is causing the growth in memory.
From here you will be able to see what is causing the growth and hopefully an indication as to where the leak is.
Related
I am using the node.js 6.11.3 ,cluster module, express 4.14
Seeing memory leaks slowly over a period of 1 weeks.
attached is a screenshot of the heap dumps in chrome dev tools. Cant tell the reason for the leak.click for heap dump
Unfortunately, nobody can answer where your leak comes from without accessing the entire application and environment. The real question is how do you debug a node memory leak.
First its important to try to understand how memory leaks occur in Node. How is it possible if Node has built in garbage collection? Well, variables are marked as garbage when no longer referenced. If you have code (closures etc) that still hold reference to variables, they are not collected. That is just one example. There are also dependencies that can cause memory leaks which can confuse you into thinking you have issues in your own code. And maybe you do, in the way you use the dependency.
Bottom line, its best to get familiar with this debugging process yourself so that you can understand the issues leading to the leak. Best of luck. Here is one article that is helpful.
https://www.alexkras.com/simple-guide-to-finding-a-javascript-memory-leak-in-node-js/
I'm using meteor to make scrapping engine and I have to do a HTTP GET request and this send me an xml but this xml is bigger than 400 ko.
I get a exception "out of memory".
result =Meteor.http.get 'http://SomeUrl.com'
FATAL ERROR: JS Allocation failed - process out of memory
There is a way to increase memory limit of a variable ?
I'm developing on Windows and had the same error. In my case, was caused by a flood of console.log statements. I disabled the log statements, and works fine again.
if you are Developing on windows
find meteor.bat in
/APPData/Local/.meteor/packages/meteor-tool/<build-tool-version>/
edit the last line of the batch file which calls the node.exe and change to
"%~dp0\dev_bundle\bin\node.exe" --max-old-space-size=2048 "%~dp0\tools\main.js" %*
Hope this helps
It is possible to increase the memory available to your node application that is spawned using meteor.
I did not have success using the --max-old-space-size flag in the instance of node called in the meteor script nor in trying to change that in the script in meteor-tool as suggested by gatolgaj
However setting the environment variable NODE_OPTIONS="--max-old-space-size=8192" did work for me.
I saw it mentioned in this thread: https://groups.google.com/forum/#!topic/meteor-talk/C5oVNqm16MY
You need to increase the amount of memory on your server, e.g. by enabling swap memory. To see how, assuming you're on Linux, you can f.ex. read DigitalOcean's guide on enabling swap memory on Ubuntu 14.04.
I don't know of any way to handle the case where Node runs out of memory, except perhaps you could separate the GET request into a child process so that the whole server doesn't crash in case you run out of memory.
To increase Node's memory limit, you could use Node's --max_old_space_size option.
Same here on Windows 10 using Meteor 1.1.0.3:
C:\Users\Cees.Timmerman\AppData\Local\.meteor\packages\meteor-tool\1.1.4\mt-os.windows.x86_32\tools\fiber-helpers.js:162
}).run();
^
FATAL ERROR: Evacuation Allocation failed - process out of memory
Resolved by setting console log level to "warning" instead of "debug" in settings.json used internally by a logger package like Winston 2.1.0 (var level = Meteor.settings.log_level).
I know this question is solved and a bit old, but I would like to share my experience. After some research, I just updated my Meteor version. It seems they are recently taking more care about Out Of Memory Errors. So I will encourage you to update to new Meteor versions.
We have a quite nasty memory leak going on in a loopback (node.js) app, but it does not seem to happen locally, only on Heroku.
It steadily increases memory usage without any requests, and I fired up 10 000 requests locally without seeing a similar pattern.
I currently have no good ideas for how to debug this further.
It turns out disabling New Relic fixed the issue. We had log level set to debug to figure out another issue, and suddenly all hell broke loose. They do indeed have a notice on their documentation about this.
I believe there is a blog post from Strongloop dealing with memory leak profiling here. It goes over installing heap dump and how to use Chrome dev tools to collect heap snapshots on the client side using the JavaScript console built into the browser. Analysis of the heap can be done within this same console as well.
I'm writing a simple cms in Node.js, Express and MongoDB. I'm planning to run a different Node.js process for every site. The problem is that after startup the process takes about 90m of RAM and for me it's too big (eight site take all server RAM). This memory is taken after the first connection to the site and other connections don't affect the memory.
Is there a guideline or a list of "best practices" to optimize this memory usage? I'm trying to track where the memory is allocated with process.memoryUsage() or a similar function but it's not simple to do this.
Is not a problem of memory leaks or something similar because the memory usage doesn't grow up after the first connection, so probably the optimization could be in loading less modules or do something differently...
The links below may help you to understand and detect memory leaks (if they do exist):
Debugging memory leaks in node.js
Detecting Memory Leaks in Node.js Applications
Tracking Down Memory Leaks in Node.js
These SO questions may also be useful:
How to monitor the memory usage of Node.js?
Node.js Memory Leak Hunting
Here is a quick fix, a node.js lib that will restart the any node process once it reaches a certain size.
https://github.com/DoryZi/memory_limiter
Set the --max_old_space_size CLI flag to control the maximum heap size. There's a post that describes Running a node.js app in a low-memory environment
tl;dr; Try setting this value, in megabytes, to about 80% of the maximum memory footprint you want node to try to remain under. e.g. to run app.js and keep it under 500MB RAM used
node --max_old_space_size=400 app.js
This setting is also described in the Node JS CLI documentation
My Node.Js application occasionally fails with a "Segmentation fault" and I am at a loss as to how to diagnose the cause. As mentioned below I have dramatically reduced their frequency by raising maxlisteners but not made the problem go away.
The application runs on a BeagleBone Black under Node v0.8.22 and uses Socket.io to communicate realtime data to browser pages displayed on the BBB's LCD display. It collects data from a sensor connected via I2C using the korevec/node-i2c library. I have, however, isolated that library and still have failures.
The failures generally occur when I have streamed data for a while to the client though they will occasionally/rarely happen at other times as well. This is not surprising as my app uses socket.io to communicate on almost all pages but the streaming page is at much higher volume.
I am getting the below message:
(node) warning: possible EventEmitter memory leak detected. 11
listeners added. Use emitter.setMaxListeners() to increase limit.
Trace
and have been doing so since day 1. I was also seeing the symptoms of a memory leak but since I raised the MaxListeners memory usage stays constant. Failure rate after making this change went down dramatically but has not gone away completely. I am using socket.io on top of http and raised MaxListeners for both socket.it and for http.
How does one go about diagnosing this problem? Is the memory leak error related? I can post code but there is quite a bit of it and I am not sure what parts are most relevant.
Thanks for any help,
Cheers, Will