I have some serious bunch of asynchronous operations running, but NodeJS process is just not exiting when supposedly all have been done. Can I somehow find out what keeps it running? Can I see heap stack of running process somehow? Or can you give me tips what are the most usual causes of such idlings?
I don't have any kind of server running there, but I am using async.nextTick quite extensively which basically uses setImmediate. I am not sure if this can somehow get stuck. Also there are no connections to any kind of database or remote server. It's just process that does some work on file system.
Maybe there is some recursive loop, but I have tried using node-inspector and paused execution after it was stuck and it didn't showed me any point in code where it would hanging.
Take a look at process._getActiveHandles() and process._getActiveRequests()
Related
Recently I'm having an issue with my server.
My node server stops, and forever does not restart it.
In my forever log I see this line:
error: Forever detected script was killed by signal: SIGKILL
The server itself does not throw an error. In fact the server seems to run without any glitches and then a random SIGKILL is executed.
I don't know if it's AWS shutting down my server, or if it's an issue with forever, or perhaps the node server itself.
Searching Google does not provide much insight.
I thought this might be related to a cpu spike or a memory usage spike, but both seem to be low (but maybe there's a spike for a split second that I don't recognize).
Is this an issue anyone has encountered before?
Any idea how can I fix it?
Well..
Why the problem occurred is still mystery but I was able to resolve it by reducing the queue for my queries on my MongoDB.
While both mongo and node were not using a lot of RAM this seems to be the cause of the issue since by reducing the number of queries, the problem disappeared.
What exactly triggered the SIGKILL is still a mystery, yet I thought this information maybe useful for other users.
For me it had to do with the way mongoose was setup and interacting with the application code.
I was able to fix by creating a connection using the answer from here: Mongoose Connection, creating my schema definitions and only exporting the models to be used.
I hope this is helpful to someone
Can you help me?
I tried jmeter testing with nodejs server but after some 5000 requests node server doesn't respond, So I have to restart the server to make it work again. is there any way to make it work again without restarting the server?
What you are asking is a way to treat the symptoms without treating the cause. I know of no way to "make it work again" but if we can find the cause of your problem we can fix it and remove the symptoms. It is difficult to comment on what exactly is happening without more information/code, but two things come to mind.
You may be performing some very heavy computations and accidentally blocking your event loop. This article discusses it in more detail.
You may have a memory leak which is crashing node. This is easy to check by watching the memory usage of node on windows task manager or a mac/linux equivalent. If the memory keeps increasing and never falls, Node may reach its max memory limit and crash. The only way I know of to fix this is to run the node garbage collector manually. This article talks about it. This is of course a temporary solution, you should fix the memory leak if you find one.
Those are the only two I can think of. If you want more help, I'll need to see your code.
you can use 'forever': https://github.com/foreverjs/forever
but I think you know why the script fails.
Good luck
I was wondering if you could terminate just the thread on which the node application is executing but NOT the process.
Now, I know this sounds strange, because node IS single-threaded, BUT I'm working on an .NET application that hosts node in-process. And when I terminate node with process.exit() the whole application gets closed, which is a behavior I don't want. I just want the node thread to get terminated.
Honestly I'm so desperate I even tried, getting a list of all application threads prior to creating the thread on which node is executing, and then another list after it's created, then removing all threads that were present prior to starting node, keeping a reference to the remaining thread, thinking it was the node thread. As you could expect this did not turn out so well.
I'm using EdgeJS to host node, if that makes any difference. It does not have a built in functionality to terminate itself, unfortunately.
Oh and server.close() doesn't work for me, for some reason.
I'm hosting a video streaming server in my node code, if that info can help you in any way.
Thanks for all the help in advance!
The node thread needs to co-operate by clearing everything that is running on the event loop and then it terminates naturally by returning. So just stop listening for requests, close handles etc. You can use process._getActiveRequests and process._getActiveHandles to see what is keeping the event loop alive.
You can also abruptly interrupt the node thread just by calling OS apis but this will leak a lot of garbage until you exit the process so you cannot start/restart node a lot of times before you need to exit the process anyway to free the leaked resources.
I have a relatively simple chat-type application running on Node.js and Socket.io. The node server streams chat data from a Minecraft server and then streams this to any clients connected on the website using Socket.io. A working demo of the system can be found here: standardsurvival.com/chat.
It works decently fine for the most part, but every once in a while the node server stops responding and active connections die shortly thereafter. The process will start consuming 100% CPU during this time but memory always stays relatively constant, so I'm doubting any sort of memory leak is involved.
It's been super frustrating as I haven't been able to reproduce the issue consistently enough to figure out what the problem is, and I don't know where to look. I've been setting up loops and commenting out various parts of the pipeline between the node server and website to try and pinpoint what may be causing it. No luck as of yet.
The code behind this system can be found here and here.
Any ideas?
Well I ended up figuring out what the problem was. A library I'm using was opening net.Sockets for standard HTTP requests to the Minecraft server, but was never actually closing them. Apparently the "end" event was never called when the request finished. So eventually all available file handles for the process were being used up and causing new requests to outright fail, which made the server appear to stop responding. I would have found this out sooner if I logged this error. Lesson learned.
I added a timeout to all sockets to fix this at least temporarily. Now the server has been running for days without a single issue :)
I have a Node.js app that loads some data from Mysql into Redis when the app starts. It has been working fine up until we modified the data in Mysql.
Now it is just exiting with a Killed message.
I am trying to pinpoint the problem but is is hard to debug using the node-inspector as the problem doesn't appear when running in --debug.
I don't think my problem is in the data itself because it works on my local machine but doesn't work on my production box.
My question is, what causes the Killed message? Is it Node.js or is it in the Mysql driver or elsewhere?
Check your system logs for messages about Node being killed. Your Node application might be using excessive memory and getting killed by the Out-Of-Memory killer.
Not sure if Redis is what causes the Killed message but that was the cause of my problem.
I was sending to much data to multi because I originally thought that was the way to use pipelining (which is automatic).