pm2 restart all keeps restarting again and again when run from script - node.js

I have a script(bash) which monitors pm2 memory usage and restarts the process if pm2 process takes memory more than 2 GB. But when i run the command (pm2 restart all) from the script, all processes in pm2 keep restarting again and again.
I am running this script via a crontab.
I want to be able to successfully restart all apps without continous restart.

pm2 already has an option for max memory which is max_memory_restart: "3G" which will restart the process if it's taking more than 3G for example.
Source: http://pm2.keymetrics.io/docs/usage/process-management/#max-memory-restart

Related

How to keep alive node server permanently?

I have tried so many ways to keeping node server alive on Linux environment but nothing has worked. Sometime the server runs only 4-5 hours and sometime it runs 10-12 hours and after that server goes shut down automatically.
I have tried forever start, pm2, nodemon but nothing has worked.
I have also tried shell script with forever start for running it but that also not worked.
Applications that are running under PM2 will be restarted automatically if the application crashes or is killed, but an additional step needs to be taken to get the application to launch on system startup (boot or reboot). Luckily, PM2 provides an easy way to do this, the startup subcommand.
The startup subcommand generates and configures a startup script to launch PM2 and its managed processes on server boots:
$ pm2 startup systemd
Run the command that was generated (similar to the highlighted output above, but with your username instead of sammy) to set PM2 up to start on boot (use the command from your own output):
sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u sammy --hp /home/sammy
check here for details https://www.digitalocean.com/community/tutorials/how-to-set-up-a-node-js-application-for-production-on-ubuntu-16-04
You can increase the size of memory restar- check this: pm2 process crashed on server. it gives an error
Try using process manager for making the application run all the time. Here is the link for Pm2 . It will restart you application once it crashes also automatically
Use a NPM package called nodemon
npm install -g nodemon
nodemon index.js
If the application fails or crashes for any reason it will restart
Read more at
https://www.npmjs.com/package/nodemon

pm2 hangs and unable to restart process due to high numbers of request

For the purpose of load testing i created an API which request around 2000 bookings at a time and after hitting this API, server(AWS EC2 instance) reaches to the more than 20% of CPU consumption and consume almost all the memory(900/991M) of system. After that i was trying to restart node app with pm2 restart but it's not working and even "pm2 ls" and "pm2 log" commands are also not working. What can i do now to run my application again ?
List of command which i already run on server.
pm2 restart pid && pm2 log pid
pm2 ls
pm2 log pid
Try to kill the current process of pm2:
ps -ef | grep pm2
kill -9 <PID_OF_PM2>
And then restart pm2 again.
Try to Upgrade to the latest pm2 version if the above not working after killing the pm2 process, using this command:
npm install pm2#latest -g
PM2 hangs when your project contains millions of files and it is watching all those files. Make sure to exclude the non-essential files from watch using ignore_watch.

Why pm2 deleting process automatically?

I am running 3 process in pm2. But after 2-3 month my process are getting deleted from pm2 list. Its like we just installed pm2.
I want to know why this is happening and how to rectify it.
Are you sure that the PM2 daemon is still alive when you're doing the pm2 list ? If you run out of memory, chance are that your OS kill the pm2 daemon and by the way killing your application since they are process child of pm2 daemon.

Amazon EC2 NodeJS server stops itself after 2 days even after using sudo nohup

I have my app running on http://talkwithstranger.com/ and I have deployed it on AWS EC2. I use this command
sudo nohup node index.js &
To continue running my Node JS server even if I close my terminal and exit my SSH.
However, after 2 days everytime I wake up and I find out that the node server itself stops automatically. I checked the running processes by using
ps -ef
and my node script is not there.
Google Chrome say site DNS not found, because NodeJS Express is not running of course to serve my html file, but why it stops itself?
What is causing this unexpected shutdown of my server after every 2 days? I have to manually run nohup again to run it again.
Does nohup has a time to expire or something ?
You should run node.js using a service / process manager. You can use something basic such as forever or supervisord but I would actually advise you to take a look at PM2.
It can do a lot of things - one of them being that it manages your process, makes sure it keeps running, restarts it when it fails, manages the logs, etc. You can also have it autostart when you restart the server.
It becomes really powerful in combination with https://pm2.io, because this enables you to monitor your server's metrics such as CPU and memory remotely and see whether exceptions happened, and much more (such as even remotely updating the software by pulling from git). However, they no longer offer a free plan unfortunately - their plans now start at $79/month, which is a pity. But don't worry, the PM2 application itself is still free and open source, only the monitoring costs money.
Basic usage of PM2:
npm install -g pm2
...to install PM2.
pm2 start my_script.js
Starts a script and lets it run in background.
pm2 status
Shows the status of any running scripts.
pm2 restart all
Restarts all running scripts.
pm2 kill
Stops all scripts and completely shuts down the PM2 daemon.
pm2 monit
Monitors CPU/RAM usage and shows it.
pm2 logs
Shows the last 20 output and error log lines and starts streaming live logs to the console. The logs are stored in the folder ~/.pm2/logs.
Using PM2, your script will not stop - at most, it will restart. And if it does you will be able to more easily understand why because you can easily access logs and watch what happenes with memory usage, etc.
Extra tips:
To avoid filling up the harddisk with logfiles, I recommend installing the module pm2-logrotate:
pm2 install pm2-logrotate
To automatically launch PM2 with the same script on startup when the server starts, you can first save the current configuration:
pm2 save
...and then use the following command to install a startup script - follow the instructions displayed, which will be different based on the exact OS you are using:
pm2 startup
To use PM2 in a more advanced way with multiple processes, custom environment variables, etc., take a look at ecosystem files.
You can try forever.Install using the following command.
npm install -g forever
Then just start forever:
forever start index.js
Another better option for production use is pm2.You can install pm2 with below command
npm install -g pm2
# or
yarn global add pm2
start server
pm2 start index.js
The best thing is you can achieve load balancing with pm2(utilize all available CPU)
pm2 start index.js -i max
For more info, you can visit pm2 documentation page.

Can the pm2 node module restarts the app after crash automatically

I have a Node.js app ready which is workable, but has known and unknown bugs which crash the app. In such cases it would be nice if pm2 can restart the node app. Is this feature already available in pm2?
Yes, it does this by default. For more information see Restart strategies.
If the app repeatedly fails to start over a short period of time, pm2 may cease restarting. See configuration, min_uptime and max_restarts.
Also, check this new excellent option:
--exp-backoff-restart-delay=100
pm2 will restart the crashed app after 100 milliseconds (0.1 seconds), then step-by-step increase restart-delay to 15 seconds.
To make app restart when it crashes you have to use one of PM2's restart strategies.
There is something called "Exponential Backoff Restart Delay" which PM2 explains as:
Instead of restarting your application like crazy when exceptions happens (e.g. database is down), the exponential backoff restart will increase incrementaly the time between restarts.
You can set it using the CLI like this:
pm2 start app.js --exp-backoff-restart-delay=100
There are other restart methods also, which are mentioned here.
This may help:
# Generate Startup Script
$ pm2 startup
# Freeze your process list across server restart
$ pm2 save
# Remove Startup Script
$ pm2 unstartup
More details here

Resources