Unable to restart Node.js app after PM2 delete - node.js

I usually do pm2 stop to stop my application and it works fine.
However, when I just tried the pm2 delete on my app and starting it again, it doesn't seem to work anymore.
This is the pm2 error log I retrieved. No code was changed so I think it might be some ECONNRESET error from abrupt shutdown? (there wasn't any cleanup code)
Also tried killing the pid of pm2 and mongod when I find it using ps -ef and also pm2 kill and restarting of mongod to no avail.
Error: Invalid mongodb uri. Must begin with "mongodb://"
Received: NaN
at muri (/home/ubuntu/portal/node_modules/muri/lib/index.js:28:11)
at NativeConnection.Connection.openUri (/home/ubuntu/portal/node_modules/mongoose/lib/connection.js:713:18)
at Mongoose.createConnection (/home/ubuntu/portal/node_modules/mongoose/lib/index.js:198:17)
at Object.<anonymous> (/home/ubuntu/portal/models/stopwords.js:11:19)
at Module._compile (module.js:573:30)
at Object.Module._extensions..js (module.js:584:10)
at Module.load (module.js:507:32)
at tryModuleLoad (module.js:470:12)
at Function.Module._load (module.js:462:3)
at Module.require (module.js:517:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/ubuntu/portal/helpers/database/helper_mongodb.js:2:19)
at Module._compile (module.js:573:30)
at Object.Module._extensions..js (module.js:584:10)
at Module.load (module.js:507:32)
at tryModuleLoad (module.js:470:12)
at Function.Module._load (module.js:462:3)
at Module.require (module.js:517:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/ubuntu/portal/controllers/inf_controller.js:2:19)
at Module._compile (module.js:573:30)
at Object.Module._extensions..js (module.js:584:10)
Code for stopwords.js:11:19 is
let db = mongoose.createConnection(process.env.MONGODB_URI + process.env.MONGODB_HELPER_DB, { useMongoClient: true })

As we commented, and as you did, I also solved my problems using pm2
+ grunt to run my express.js app. Will try to explain here the solution to close the question.
The way to start and stop pm2 with different apps and without trouble
was with an ecosystem file where I configured my node apps.(centOS apache)
my ecosystem file
-bash-4.2$ pm2 ecosystem
File /var/www/vhosts/domain.com/ecosystem.config.js
[...]
-bash-4.2$ cat ecosystem.config.js
module.exports = {
/**
* Application configuration section
* http://pm2.keymetrics.io/docs/usage/application-declaration/
*/
apps : [
// First application
{
name : 'DomainProd',
cwd : '/var/www/vhosts/domain.com/httpdocs/App',
script : 'start.sh'
},
{
name : 'DomainTest',
cwd : '/var/www/vhosts/domain.com/testing.domain.com/httpdocs/App',
script : 'start-test.sh'
}
]
};
the start.sh script
-bash-4.2$ cat /var/www/vhosts/domain.com/httpdocs/App/start.sh
/usr/bin/grunt serve:prod
-bash-4.2$
The serve task on my Gruntfile
return grunt.task.run(['concurrent:cleanandbuild', 'build', 'env:all', 'env:prod', 'express:prod', 'wait', 'express-keepalive']);
The run example:
-bash-4.2$ pm2 status
┌────────────┬────┬──────┬───────┬────────┬─────────┬────────┬─────┬────────────┬──────────┐
│ App name │ id │ mode │ pid │ status │ restart │ uptime │ cpu │ mem │ watching │
├────────────┼────┼──────┼───────┼────────┼─────────┼────────┼─────┼────────────┼──────────┤
│ DomainProd │ 0 │ fork │ 17992 │ online │ 63 │ 20D │ 0% │ 264.1 MB │ disabled │
│ DomainTest │ 1 │ fork │ 25526 │ online │ 295 │ 113m │ 0% │ 1.2 MB │ disabled │
└────────────┴────┴──────┴───────┴────────┴─────────┴────────┴─────┴────────────┴──────────┘
Use `pm2 show <id|name>` to get more details about an app
-bash-4.2$ pm2 stop 1
[PM2] Applying action stopProcessId on app [1](ids: 1)
[PM2] [Domainest](1) ✓
┌────────────┬────┬──────┬───────┬─────────┬─────────┬────────┬─────┬────────────┬──────────┐
│ App name │ id │ mode │ pid │ status │ restart │ uptime │ cpu │ mem │ watching │
├────────────┼────┼──────┼───────┼─────────┼─────────┼────────┼─────┼────────────┼──────────┤
│ DomainProd │ 0 │ fork │ 17992 │ online │ 63 │ 20D │ 0% │ 264.1 MB │ disabled │
│ DomainTest │ 1 │ fork │ 0 │ stopped │ 295 │ 0 │ 0% │ 0 B │ disabled │
└────────────┴────┴──────┴───────┴─────────┴─────────┴────────┴─────┴────────────┴──────────┘
Use `pm2 show <id|name>` to get more details about an app
-bash-4.2$ pm2 start 1
[PM2] Applying action restartProcessId on app [1](ids: 1)
[PM2] [DomainTest](1) ✓
[PM2] Process successfully started
┌────────────┬────┬──────┬───────┬────────┬─────────┬────────┬─────┬────────────┬──────────┐
│ App name │ id │ mode │ pid │ status │ restart │ uptime │ cpu │ mem │ watching │
├────────────┼────┼──────┼───────┼────────┼─────────┼────────┼─────┼────────────┼──────────┤
│ DomainProd │ 0 │ fork │ 17992 │ online │ 63 │ 20D │ 0% │ 264.1 MB │ disabled │
│ DomainTest │ 1 │ fork │ 29816 │ online │ 295 │ 0s │ 0% │ 1.2 MB │ disabled │
└────────────┴────┴──────┴───────┴────────┴─────────┴────────┴─────┴────────────┴──────────┘
Use `pm2 show <id|name>` to get more details about an app
-bash-4.2$ pm2 logs 1
[TAILING] Tailing last 15 lines for [1] process (change the value with --lines option)
/var/www/vhosts/domain.com/.pm2/logs/DomainTest-out-1.log last 15 lines:
1|DomainTe | Execution Time (2017-10-02 19:33:24 UTC+2)
1|DomainTe | loading tasks 579ms ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 92%
1|DomainTe | clean:server 47ms ▇▇▇▇ 7%
1|DomainTe | Total 632ms
1|DomainTe |
1|DomainTe | Running "clean:dist" (clean) task
1|DomainTe | >> 5 paths cleaned.
1|DomainTe |
1|DomainTe | Done, without errors.
1|DomainTe |
1|DomainTe |
1|DomainTe | Execution Time (2017-10-02 19:33:24 UTC+2)
1|DomainTe | loading tasks 570ms ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 55%
1|DomainTe | clean:dist 456ms ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 44%
1|DomainTe | Total 1s
[STREAMING] Now streaming realtime logs for [1] process
Hope it helps anyone arriving here.

Related

Cannot run node js app on nginx server because address is already in use

I am running a node js app on nginx/digital ocean .
When i try to run pm2 start app.js i get this
┌─────┬────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │
├─────┼────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0 │ app │ default │ 1.2.0 │ fork │ 0 │ 0 │ 30 │ errored │ 0% │ 0b │ root │ disabled │
│ 1 │ app │ default │ 1.2.0 │ fork │ 49014 │ 0s │ 0 │ online │ 0% │ 15.1mb │ root │ disabled │
└─────┴────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
**
when i try "node app.js" looks like this**
node:events:491
throw er; // Unhandled 'error' event
^
Error: listen EADDRINUSE: address already in use :::4000
at Server.setupListenHandle [as _listen2] (node:net:1733:16)
at listenInCluster (node:net:1781:12)
at Server.listen (node:net:1869:7)
at Function.listen (/var/www/delivery.emms.io/html/node_modules/express/lib/application.js:618:24)
at Object.<anonymous> (/var/www/delivery.emms.io/html/app.js:114:5)
at Module._compile (node:internal/modules/cjs/loader:1218:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1272:10)
at Module.load (node:internal/modules/cjs/loader:1081:32)
at Module._load (node:internal/modules/cjs/loader:922:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
Emitted 'error' event on Server instance at:
at emitErrorNT (node:net:1760:8)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
code: 'EADDRINUSE',
errno: -98,
syscall: 'listen',
address: '::',
port: 4000
}
i tried getting the PID and using kill -9 but no success
any ideas?

Using PM2 with highcharts-export-server does not work

In the documentation for highcharts-export-server : it says that you should use PM2 to run it as a server but I just can't get this to work on my Windows machine.
I have no problem to run it from the command line :
highcharts-export-server --enableServer 1
However when I run :
pm2 start highcharts-export-server -- --enableServer 1
I get :
[PM2] Applying action restartProcessId on app [highcharts-export-server](ids: [ 0 ])
[PM2] [highcharts-export-server](0) ✓
[PM2] Process successfully started
┌─────┬─────────────────────────────┬─────────────┬─────────┬─────────┬──────────┬─────── ─┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │
├─────┼─────────────────────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0 │ highcharts-export-server │ default │ N/A │ fork │ 13916 │ 0 │ 50 │ stopped │ 0% │ 0b │ <ME> │ disabled │
└─────┴─────────────────────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
It doesn't work and I get this error multiple times in highcharts-export-server-error.log :
C:\USERS\<ME>\APPDATA\ROAMING\NPM\HIGHCHARTS-EXPORT-SERVER.CMD:1
#ECHO off
^
SyntaxError: Invalid or unexpected token
at Object.compileFunction (node:vm:352:18)
at wrapSafe (node:internal/modules/cjs/loader:1033:15)
at Module._compile (node:internal/modules/cjs/loader:1069:27)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
at Module.load (node:internal/modules/cjs/loader:981:32)
at Function.Module._load (node:internal/modules/cjs/loader:822:12)
at Object.<anonymous> (C:\Users\lars\AppData\Roaming\npm\node_modules\pm2\lib\ProcessContainerFork.js:33:23)
at Module._compile (node:internal/modules/cjs/loader:1105:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
at Module.load (node:internal/modules/cjs/loader:981:32)
I don't undertand what I'm doing wrong here because this works flawlessly on my Mac machine

PM2 Catching Errored State Signal

I am trying to catch the process before it goes into the state of errored. The process I am running is erroring and restarting correctly. After 15 attempts of restarting it will go into a state of errored, as shown for the process with an ID of 0 below.
┌─────┬─────────────────────────────────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │
├─────┼─────────────────────────────────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 1 │ a58a1e0d-3a6f-4512-8b83-4dcfd2f9e408 │ default │ 1.0.0 │ fork │ 3139 │ 8s │ 0 │ online │ 0% │ 61.6mb │ warren │ disabled │
│ 0 │ e95ff617-4800-4059-906b-2cde63bcb4b6 │ default │ 1.0.0 │ fork │ 0 │ 0 │ 15 │ errored │ 0% │ 0b │ warren │ disabled │
└─────┴─────────────────────────────────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
Before it goes into a state of errored what signal (if any) is sent to the process?
For example when I issue a pm2 stop <PROCESS_NAME> I can can intercept the SIGINT message and log something to my log file as in the example below.
process.on('SIGINT', function() {
logger.info("I HAVE BEEN KILLED")
})
I need something like this but the signal sent to the process when it switches to an errored state is listened for.
You can just use the local library of pm2 to catch the state of errored processes from logs.
npm install pm2 on your project directory
Go into the file where process is initiated
Add the following command into the file, it will add an event listener your process and will catch if error occured
const pm2 = require('pm2')
pm2.launchBus(function(err, bus) {
bus.on('log:err', function(e) {
//When a task throws errors
});
});

PM2 ignoring ecosystem file

I have the following file
ecosystem.js:
module.exports = {
apps: [
{
name: 'my-app',
cwd: '/test,
script: './myapp.js',
instances: 'max', // match the number of CPUs on the machine
exec_mode: 'cluster', // Run multiple child processes
args: 'start',
env: {
NODE_ENV: 'production'
}
},
],
};
I expect to see a cluster of node processes running. But it seems to start it in fork mode, ignoring my settings entirely.
I start like this:
pm2 start ecosystem.js
Output:
Starting ecosystem.js in fork_mode (1 instance)
─────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │
├─────┼─────────────────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0 │ ecosystem │ default │ 1.0.0 │ fork │ 156879 │ 0s │ 0 │ online │ 0% │ 14.3mb │ -… │ disabled │
└─────┴─────────────────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
What could be causing this?
Perhaps the missing trailing ' closing quote on cwd.
And if not that, I just stumbled on Problem running express app with pm2 using ecosystem config file. which solved the issue for me.
File name must end with .config.js.

verdaccio fail to start having an error "Cannot find module"

I am getting this error, verdaccio just crashed and no longer working after a server restart. When I try to pm2 start verdaccio , I get the following error
Error: Cannot find module '/home/ec2-user/verdaccio'
at Function.Module._resolveFilename (module.js:470:15)
at Function.Module._load (module.js:418:25)
at Object.<anonymous> (/usr/lib/node_modules/pm2/lib/ProcessContainerFork.js:53:21)
at Module._compile (module.js:571:32)
at Object.Module._extensions..js (module.js:580:10)
at Module.load (module.js:488:32)
at tryModuleLoad (module.js:447:12)
at Function.Module._load (module.js:439:3)
at Module.runMain (module.js:605:10)
at run (bootstrap_node.js:427:7)
npm config ls
; cli configs
scope = ""
user-agent = "npm/4.2.0 node/v7.10.0 linux x64"
; userconfig /home/ec2-user/.npmrc
http-proxy = null
https-proxy = null
registry = "https://registry.npmjs.org/"
strict-ssl = false
; globalconfig /usr/etc/npmrc
registry = "https://registry.npmjs.org/"
; node bin location = /usr/bin/node
; cwd = /home/ec2-user/verdaccio
; HOME = /home/ec2-user
; "npm config ls -l" to show all defaults.
pm2 show verdaccio
│ name │ verdaccio │
│ restarts │ 255 │
│ uptime │ 0 │
│ script path │ /home/ec2-user/verdaccio │
│ script args │ N/A │
│ error log path │ /home/ec2-user/.pm2/logs/verdaccio-error-0.log │
│ out log path │ /home/ec2-user/.pm2/logs/verdaccio-out-0.log │
│ pid path │ /home/ec2-user/.pm2/pids/verdaccio-0.pid │
│ interpreter │ node │
│ interpreter args │ N/A │
│ script id │ 0 │
│ exec cwd │ /home/ec2-user │
│ exec mode │ fork_mode │
│ node.js version │ 7.10.0 │
│ watch & reload │ ✘ │
│ unstable restarts │ 0 │
│ created at │ N/A
`
My config file is as follows:
storage: ./storage
auth:
htpasswd:
file: ./htpasswd
uplinks:
npmjs:
url: https://registry.npmjs.org/
packages:
'av-*':
allow_access: $all
allow_publish: $all
'*':
allow_access: $all
allow_publish: $all
proxy: npmjs
'#*/*':
allow_access: $all
allow_publish: $authenticated
proxy: npmjs
https:
key: /home/ec2-user/verdaccio/verdaccio-keynew.pem
cert: /home/ec2-user/verdaccio/verdaccio-certnew.pem
ca: /home/ec2-user/verdaccio/server.ca
logs:
- {type: stdout, format: pretty, level: http}
#- {type: file, path: verdaccio.log, level: info}
listen:
- https://devnpm2:4873
max_body_size: 300mb
It can't find the verdaccio executable, to locate it you can use the which command.
To do this you should try:
pm2 start `which verdaccio`

Resources