I am running some application in NodeJS that retrieves data from a database and serve at port 3000.
It is running fine most of the time but sometimes it just errors out (it could be too many connections or network issue or some sql injection etc.). This happens every 2-3 days randomly.
While I am figuring out root cause - what is the best way to have NodeJS always 'ON' - meaning if it stops running - some process kicks it and it starts again?
You can use forever as stated by the previous answer, but as I've worked with forever before, I'd suggest using the pm2 process manager. It has, on top of what you need, some more advanced functionalities and is well-documented.
Try this:
forever start -w <filename>
-w is for watch. it watches the file changes and restarts if any changes found.
Also, if the script is unexpectedly stopped, it will try to restart it.
forever list for details of the process.
you will see the PID, filename, log filename.
in the log file, you can see the log of your script.
Hope it helps. :)
Related
There are a lot of pieces so I don't expect anyone to be able to answer this without seeing every configuration. But maybe people can tell me how to look for diagnostics or kind of how the major pieces fit together so that I can understand what I'm doing wrong.
I have a Tencent CVM instance running Ubuntu Server.
I also have a domain name pointing to the ip address of that server.
I start an nginx service to listen to port 1337 and pass requests to example.com/parse
I have mongodb running inside of a docker container, listening on port 27017.
Inside of index.js, I have the databaseURI set as 'mongodb://localhost:27017/dev' and the SERVER_URL set as 'https://example.com/parse'
When it's time to deploy the Parse Server instance, I use screen inside of my current ssh session, run npm start, and then detach the screen, and then kill my ssh session by closing the terminal.
Finally, I run the parse dashboard on my local machine with serverURL 'https://example.com/parse'
And everything works great. I add items to the database via the test page that comes with the Parse Server repo. I add items to the database via cloudcode calls from Python. I add and delete classes and objects via the dashboard. The behavior is exactly like I should expect.
And it continues that way for anywhere between 12-72 hours.
But after a few days of normal operation, it will happen that I open parse dashboard and everything is gone. I can start adding things again and everything works right, but nothing persists for more than 72 hours.
There's a lot I don't understand about the anatomy of this, so I figured maybe using screen and then detaching and closing the terminal causes some process to get killed and that's my mistake. But when I run top, I can see everything. I can see npm start is running. I can see mongo is running. When I docker ps to check the container, it's still there doing fine. The nginx service is still running.
Can anyone suggest a way for me to start diagnosing what the problem is? I feel like it's not any of the configs because if that was the problem, it wouldn't work fine for so long. I feel like it must be how I'm deploying it is causing something to reset or causing some process that's supposed to be always running to die.
Edit: For posterity I'll summarize the below as a solution in case you've come here struggling with the same issue. #Joe pointed me to db.setProfilingLevel(), level 2 with the slowms=0 option for max verbosity. Those logs are written to the file indicated within mongodb.conf. Docker doesn't persist storage by default, so you should have a named volume. The syntax is $docker volume create <volume_name>. And the syntax to attach the volume when you create the container is to add a v flag like -v <volume_name>:. And finally, I was running mongodb in a container because that's the workflow I saw in tutorials. But it's solving a problem I didn't have and it was simpler to start mongodb as a service without a container.
For some reason I often notice after having my terminal open for a while, I can't see the output from my nodejs server anymore. I know it's running, though. How do I re-open the node output info? (By output info, I mean "console.log" in nodejs code. I'm running on linux.
For anyone looking for an answer to this, my research conclusion is that it's an artifact of nodemon that exists. You have to physically go into the port and kill it, which I've spent an enormous amount of time doing. Not very helpful if you were running this in production and you couldn't see output anymore. But it is what it is.
I'm trying to create a simple Twitter bot to learn some Node.js skills.
It works fine on my local computer. I start the script with node bot.js and then close it with Ctrl + C.
I've uploaded the files to a server (Krystal hosting). I've ssh'd into the server and then used $ source /home/[username]/nodevenv/twitterbot/10/bin/activate. Which I think puts me into a Node environment (I'm not really clear what is happening here).
From here I can run node bot.js. My Twitter bot runs fine and I can leave the terminal. What I've realised now is that I don't know how to stop this script.
Can someone explain how I should be doing this? Is there a command I can enter to stop the original bot.js process? Since looking into this it looks like perhaps I should have used something like pm2 process manager. Is this correct?
Any pointers would be much appreciated.
Thanks,
B
You can kill it externally by nuking the process from an OS command line or in an OS GUI (exact procedure varies by OS). Ctrl-C from the shell is one version of this, but it can be done without the command shell that it was started in too by nuking the process directly.
Or, you can add a control port (a simple little http server running on a port only accessible locally) that accepts commands that let you do all sorts of things with the server such as extract statistics, shut it down, change the configuration, tell it to clear caches so content updates take effect immediately, etc... Shutting down the server this way allows for a more orderly shut-down from code within the server. You can even stop accepting incoming connections, but wait for existing http connections to complete before shutting down, close databases normally, etc...
Or, you can use a monitoring program such as PM2 or forever that in addition to restarting the server automatically if it should ever crash, they also offer commands for shutting it down too (which will just send it certain signals kind of like Ctrl-C does).
In my project I have several servers which run NodeJS applications using PM2, those were not created by me. I am not that familiar with the PM2. Now I need to start a new server, which is simply a CRON process that queries an ElasticSearch instance.
There are no routes or anything in it, just a CRON with some logging.
Here is my dilemma. I have played with PM2 and I become somewhat familiar with what is it, and what it does. But the question is how shall I run it?
The previous projects do have PM2 config.json with many parameters, and they are started in cluster mode (handled with Nginx), and when I start them I see all process's becoming daemons. But in my case I don't need that. I just need it to run as a single service.
In other words if I use the configuration file to run the PM2, I see it spawned in cluster mode, and it creates chaos as my CRON is fired many times. I don't need that. If I start it in Fork mode, it also spawns instances, but all of them die, except one (due to which they are using same port). I also don't need that.
I just need single service.
I managed to run the my CRON app.js with the singe line, simple as:
PM2 start app.js. It runs in single thread, and I can see it's info with PM2 status. All fine.
If I run it with the single line(as in my case), is it considered ok? Based in my knowledge if I use config.json, it will always run it in fork or cluster.
Is it ok to run it in single line, or do I need still to use a config.json file.
If you only need one process to be run, as is the case, you're doing the right thing.
how can I automatically restart my node js server every hour automatically. There is a bug that only shows up after 1 hours and I want to be able to run it through the night. I'm already using forever.
I like pm2 for running my node apps but when it comes to keeping things alive then I rely on monit. But If it really crashes every hour then you are better off trying to fix that bug.
You could try forever. I've not used it myself, but seems to do what you want.