I have a pm2 server which, with no apparent reason, restarts every time someone is uploading a file to the server, no matter the size of the file.
I tried increasing the --max-memory-restart to 2048M
What should I do?
Di you see the logs on the server? With pm2 you can see all the errors in the log file that are happening. Try to review the logs file and try to be more specific next time.
Related
In my PM2 instance running on Windows server having 4 cores CPU and that many clusters/instances, file changes are caught and servers get restarted but the changes do not get reflected. Instead it continues to show the old updates/features.
But when I tried doing the following it starts reflecting the new updates.
$ pm2 stop all
$ pm2 kill
$ pm2 start... (one by one)
This does not happen always but happens sometime only. How to fix this?
I've faced this problem in AWS deployment. Then I come up with a solution after long research and this worked for me as a charm.
After git pull just stop your server and then start it. After that
pm2 start index.js
pm2 reload index
Now your changes are reflected.
I'm trying to figure out why my app is crashing.
When I look through my PM2 log file (www-out-0.log) I can see it restarting, but can't tell if it was because I typed in pm2 restart all, or because my app crashed and PM2 restarted it.
How can I make it print out this important information?
I have a website with Nginx installed as a reserve proxy for an ExpressJS server (proxies to port 3001). This uses Node and ReactJS for my frontend application.
This is simply a testing website currently, and isn't known or used by any users. I have this installed on a Digital Ocean Droplet with Ubuntu.
Every morning when I wake up, I load my website and see 502 Bad Gateway. The problem is, I don't know how to find out how this happened. I have PM2 installed which should automatically restart my ExpressJS server but it hasn't done so, and when I run pm2 list, my application is still showing online:
When I run pm2 logs, I get the following error (I am running this as an Administrator):
So I'll run pm2 restart all to restart the app, but then I don't see any crash information. However on this occasion when taking this screenshot, there were a couple of unusual requests. /robots.txt, /sitemap.xml and /.well-known/security.txt, but nothing indicating a crash:
When I look at my Nginx error.log file, all I can see is the following:
There is, however, something obscure within my access.log ([09/Oct/2018:06:33:19 +0000]) but I have no idea what this means:
If I run curl localhost:3001 whilst the server is offline, I will receive a connection error message. This works fine after I run pm2 restart all.
I'm completely stuck with this and even the smallest bit of help would be appreciated greatly, even if it's just to tell me I'm barking up the wrong tree completely and need to look elsewhere - thank you.
I think you should check this github thread, it seems like it could help you.
Basically, after few hours, a Nodejs server stop functioning, and the poor nginx can not forward its requests, as the service listening to the forward port is dead. So it triggers a 502 error.
It was all due to a memory leak, that leads to a massive garbage collection, then to the server to crash. Check your memory consumption, you could have some surprises. And try to debug your app code, a piece (dependency) at the time.
Updated answer:
So, i will add another branch to my question as it seems it has not helped you so far.
You could try to get rid of pm2, and use systemd to manage your app life cycle.
Create a service file
sudo vim /lib/systemd/system/appname.service
this is a simple file i used myself for a random ExpressJS app:
[Unit]
Description=YourApp Site Server
[Service]
ExecStart=/home/user/appname/index.js
Restart=always
Environment=PATH=/usr/bin:/usr/local/bin
Environment=NODE_ENV=production
WorkingDirectory=/home/user/appname
[Install]
WantedBy=multi-user.target
Note that it will try to restart if it fails somehow Restart=always
Manage it with systemd
Register the new service with:
sudo systemctl daemon-reload
Now start your app from systemd with:
sudo systemctl start appname
from now on you should be able to manage your app life cycle with the usual systemd commands.
You could add stdout and stderr to syslog to understand what you app is doing
StandardOutput=syslog
StandardError=syslog
Hope it helps more
You cannot say when exactly NodeJS will crash, or will do big GC, or will stun for other reason.
Easiest way to cover such issues is to do health check and restart an app. This should not be an issue when working with cluster.
Please look at health check module implementation, you may try to use it, or write some simple shell script to do the check
I have a pm2 process named app that was used to test the configuration.
I noticed app was starting when the system rebooted, and it was causing errors with the real application.
I ran:
pm2 delete app
then I ran:
pm2 list
and it didn't show app.
When I reboot my system, the app is still there and it is running. I attempted to find information on where the config file is online, and there is no information other than creating a template config file. Where should the config file that pm2 reads on startup be located on an Ubuntu system, or why isn't delete working as I intend? Is there another method or command I can use to remove a pm2 process or am I looking at this incorrectly?
If you use "pm2 delete {appname}" to delete the last app
pm2 delete app
when you run
pm2 save
It will show
[PM2] Saving current process list...
[PM2] Nothing to save !!!
[PM2] In this case we keep old dump file. To clear dump file you can delete it manually !
Which means, actually, the last app information is still not deleted.
The solution is to create a new dump file.
pm2 cleardump
Then, the app will be deleted permanently.
You can check the pm2 file to see what's actually saved into dump file.
/home/ubuntu/.pm2/dump.pm2
Saving some time for those who may or may not use pm2 regularly and struggle when searching for an answer on this:
You have to save the edits for pm2:
pm2 save
after making any changes. Unlike most Unix style settings interfaces, pm2 requires you to save your changes from the running version to the config file.
Hope this helps someone even though it is simple! I don't use PM2 all the time and it was really frustrating to not find any answers on SO or anywhere else that referred to the need to use pm2 save when deleting a process.
I have a sailsjs app on AWS EC2, which I have been running till now using forever. I have two adantages using forever:
1) Perpetuality: I can use the CLI forever start app.js or forever restart app.js and then app starts running and keeps on running till I stop it with the command forever stop app.js. So, the app does not stop even when I close my terminal. The process keeps on running.
2) Runtime Log: I have a .forever directory that has a log file, while on real time records the server logs, and when I check the log using tail -f file_name.log, I get to see run time logs.
However there is a disadvantage: Every time I upload a new/modified server file, I have to restart the app manually. To get rid of this, I am switching from forever to nodemon.
From the documentation provided by Nodemon, I cant figure out how can I replicate the two advantages, as mentioned above, from Nodemon too. Will be a great help if anyone can guide me on how to start my nodejs app using nodemon so that it can keep running even after closing the terminal on my side, and how to watch runtime log of server.
Just my two cents.
I use nodemon daily while developing and I dont think its something you want to use in place of something like forever. Nodemon is used when developing, the software will detect when there has been a file change and restart the server but for deployment it should not be considered.
There is no need to change either because forever has this use case handled with the --w or --watchDirectory comand that will watch for file changes(It can be found here on their readme).