PM2 resurrect not working on reboot - dump.pm2 being emptied - node.js

I'm trying to configure PM2 to daemonize a simple node.js server on centos. The init.d script executes, and there are no errors, but pm2 list returns an empty list, and the server is not starting.
Originally, I was attempting to start the process as a different user (for those interested, this can be done with modifications to pm2-init.sh). But due to the complication, I decided to first try to get it to run as root.
The strange symptom of the problem is that the dump.pm2 file is being emptied by some process, thus the pm2 resurrect has nothing to do. I set up pm2 to run on reboot as follows
$ pm2 startup centos
$ pm2 start /path/to/my/server.js
$ pm2 save
At this point, dump.pm2 had a large JSON object in it and the server is running. Then, running:
$ reboot
... wait and log back in
$ pm2 list
It was empty again, and so was the dump.pm2!
I'm not sure what process is clearing this file. I've tried reinstalling and re-running the startup script to no avail. Any help is greatly appreciated.

As a temporary workaround, I set the dump file as immutable:
$ chattr +i /path/to/my/dump.pm2
This worked. Although not ideal, it is what I will use until I can determine why the dump.pm2 file is being emptied.

Related

PM2 Flush not clearing logs

I run a Ubuntu server which is suddenly full because of pm2 logs taking 16GB. I tried pm2 flush. But this only clears the folder which took 4GB.
As root the ".pm2" folder is cleared but not logs folder. As my own user the folder is cleared but under the wrong username.
Here is showed running pm2 flush in root and admin user.
How do i clear it?
you should try pm2 flush <app_name> it will remove the default output and error logs from .pm2/logs
the above command does not flush the merge logs for the specified app_name
Hope I'm not too late. Alright, long story short.
The problem was within the code, the arguments passed aren't implemented correctly to deal with whatever logical comparison or verification they wanted. So as a result, the logs aren't being flushed.
I created an PR, which you can access here. I think it will take some time for the maintainers to review the pull. It may not even get merged at all.
Anyways, if you want to apply and test the updated code to your system. You need to find where your PM2 package files are located.
To do this, just run the commands according to the package manager that you used to install pm2.
Note: If you installed pm2 using sudo, you must ran the commands with sudo too.
yarn => yarn global dir
npm => npm root -g
After that, cd into the directory path that the command returned.
Then cd into <..PATH..>/node_modules/pm2/lib/API. You should see a file named LogManagement.js. Use sudo cp to make a copy of that file. So if there's any problem, you can still turn back.
Following, go to this github link, and replace the entire LogManagement.js with the copied code.
Done! You should be able to run pm2 flush "My App" or something like pm2 flush 1 as usual.

Supervisord loading old config

I'm using Supervisord to queue my Laravel jobs. Today I updated the config files and did the usual supervisorctl reread and supervisorctl update. Nevertheless, i didn't get any updated config files.
whereis supervisord results in supervisord: /usr/bin/supervisord /etc/supervisord.conf
whereis supervisor results in `` (empty response)
The /etc/supervisord.conf has the following
[include]
files=/var/www/vhosts/myfirstdomain/myfolder/ms_super.conf
files=/var/www/vhosts/myseconddomain/myfolder/th_super.conf
The path is correct and the files are setup exactly the same. I changed my PHP code for the worker and had to restart the service.
My struggle is that it's not updating, and I tried reload and restart all, it keeps running the old config file. I believe it's stored somewhere as cache or something.
My package manager is yum. I really appreciate your help.

How to properly run NodeJs on Windows Server in production

I am having difficult times, trying to make my NodeJS scripts run on windows server 2012. Or more precisely, to make it robust.
I have installed PM2, whic his great, also added service for windows startup which works fine, but now I found biggest issue I can't solve.
When windows server user start pm2 start, directly on server or through ssh, when logging out, all pm2 scripts are gone.
I've tried to look into pm2-windows-service but that seems inconsistent, when I restart service, it works fine, but sometimes I need to manually reload o restart only 1 script and then whole list of pm2 scripts gets somehow detached or attached to user, so when I log out from server it's all gone again.
I can't find solution to have watcher/autorestart on scripts, and make them run as a service regardless of user being logged in/out.
There must be solution for running multiple nodejs scripts on windows ?
Once you launch your NodeJS processes, you should run pm2 save command. That way, your stated scripts are saved.
Your attached/dettached problems may be due a relative path configuration in the service env variables. Try moving to an absolute path.
Note: PM2 recommendation is to launch commands with an admin-privileged user; otherwise inconsistences may appear.
Note (2): Bare in mind that PM2 on Windows has some issues. For example, if you restart your server, most of the times processes dissapear from pm2 ls (they noted that Windows does not have the feature to restore saved instances). If you saved them, pm2 resurrect will restore them.

How to make a node application on Digitalocean run constantly?

I recently got a node application up and running on digitalocean's droplet. Everything is working properly, if I go forever on the server javascript file I can navigate to the site and it works.
If I close the terminal window in which I am ssh-ing, the site is no longer available, which is expected of course. But my question is, how can I run the forever as a daemon of sorts?
I could not find the answer anywhere, which is really strange...hope somebody here might be able to help me
Generally forever should continue running if you set it up as a background task and don't close it. Unfortunately if the server needs to restart (which can happen often), you need something to get it started again.
An alternative to forever is pm2, pretty much the same thing. There is a tutorial on how to set up your production server here. Here is an excerpt from the site for just the pm2 part, as I assume you got everything else sorted:
Install PM2
Now we will install PM2, which is a process manager for Node.js applications. PM2 provides an easy way to manage and daemonize applications (run them as a service).
We will use Node Packaged Modules (NPM), which is basically a package manager for Node modules that installs with Node.js, to install PM2 on our app server. Use this command to install PM2:
sudo npm install pm2 -g
Manage Application with PM2
PM2 is simple and easy to use. We will cover a few basic uses of PM2.
Start Application
The first thing you will want to do is use the pm2 start command to run your application, hello.js, in the background:
pm2 start hello.js
This also adds your application to PM2's process list, which is outputted every time you start an application
PM2 automatically assigns an App name (based on the filename, without the .js extension) and a PM2 id. PM2 also maintains other information, such as the PID of the process, its current status, and memory usage.
Applications that are running under PM2 will be restarted automatically if the application crashes or is killed, but an additional step needs to be taken to get the application to launch on system startup (boot or reboot). Luckily, PM2 provides an easy way to do this, the startup subcommand.
The startup subcommand generates and configures a startup script to launch PM2 and its managed processes on server boots. You must also specify the platform you are running on, which is ubuntu, in our case:
pm2 startup ubuntu
The last line of the resulting output will include a command (that must be run with superuser privileges) that you must run:
Output:
[PM2] You have to run this command as root
[PM2] Execute the following command :
[PM2] sudo env PATH=$PATH:/usr/local/bin pm2 startup ubuntu -u sammy
Run the command that was generated (similar to the highlighted output above) to set PM2 up to start on boot (use the command from your own output):
sudo env PATH=$PATH:/usr/local/bin pm2 startup ubuntu -u sammy

Keeping node.js running

During development it would be great to have node.js restarted automatically when changes occur in the applications directory. When the application is in production, node could be start just like normally. When the server is rebooted with or without intention, node needs obviously to be restarted.
I've heard about and tried both forever and supervisor. Supervisor works good, but I can't make it to start during boot. Forever on the other hand ignores any changes in my directory.
Are there alternatives that are available that would do all this for me?
Thanks
Here's what I ended up doing:
Below code was put in /etc/init/myApplication.conf:
# Enter below
respawn
console none
start on runlevel [2345]
stop on runlevel [06]
script
sudo always /var/www/backend/app.js > /var/www/backend/nodelog.log
end script
Always checks for changes in my node directory while this script ensures that always is started on boot.
I've recently started using Foreman to manage my apps. You can export from Foreman to Upstart which should solve your problem with handling reboots, etc. in a production environment (although I haven't done this myself yet.)
For automatically updating files on change, check out always. (Then run always from your local Procfile through Foreman - that's my setup.)
I don't think you are likely to find anything that automatically starts node on boot, but depending on your OS, this isn't very difficult. Could you tell us what your OS is?

Resources