Systemd node app failing to start - node.js

I have a node app, that I'm trying to add as a service, the app works fine if I go to the directory where it's installed and do node start.js however when I do systemctl start app it just hangs for a few mins then journalctl -u darknet shows:
Dec 24 01:46:33 Skynet systemd[1]: Started darknet.
Dec 24 01:46:33 Skynet systemd[1]: Starting darknet...
Dec 24 01:46:34 Skynet darknet[32246]: module.js:434
Dec 24 01:46:34 Skynet darknet[32246]: return process.dlopen(module, path._makeLong(filename));
Dec 24 01:46:34 Skynet darknet[32246]: ^
Dec 24 01:46:34 Skynet darknet[32246]: Error: Module version mismatch. Expected 46, got 51.
Dec 24 01:46:34 Skynet darknet[32246]: at Error (native)
Dec 24 01:46:34 Skynet darknet[32246]: at Object.Module._extensions..node (module.js:434:18)
Dec 24 01:46:34 Skynet darknet[32246]: at Module.load (module.js:343:32)
Dec 24 01:46:34 Skynet darknet[32246]: at Function.Module._load (module.js:300:12)
Dec 24 01:46:34 Skynet darknet[32246]: at Module.require (module.js:353:17)
Dec 24 01:46:34 Skynet systemd[1]: darknet.service: main process exited, code=exited, status=1/FAILURE
Dec 24 01:46:34 Skynet systemd[1]: Unit darknet.service entered failed state.
Dec 24 01:46:34 Skynet systemd[1]: darknet.service failed.
This is my .service file:
[Unit]
Description=darknet
After=network.target
[Service]
ExecStart=/usr/bin/node /home/botty/Darknet/start.js
Restart=always
RestartSec=180
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=darknet
User=botty
Group=botty
Environment=NODE_ENV=production
WorkingDirectory=/home/botty/Darknet
[Install]
WantedBy=multi-user.target
I tried running the process both as regular user and root and it works fine .. but no matter what I try it gives errors if started with systemd.

I had this problem. I wasn't using the same node version in systemd as when running locally.
$ which node
/usr/local/bin/node
Replacing /usr/bin/node with /usr/local/bin/node (or the equivalent in which node, such as when using nvm) in ExecStart fixed it for me.

Contrary to a popular misconception, systemd is very different from traditional Unix concept of init scripts. It is explained in much more detail here and here. I have noticed that a lot of people here are asking about problems with running Node with systemd and they rarely get any solutions. My recommendation would be to use traditional SysV init scripts or Upstart that also works with no surprises whenever you see that you cannot do something with systemd. I have never seen situations where a Node application couldn't be easily run with Upstart or SysV init scripts but I've seen problems with systemd. I've heard that it is especially problematic with clustering and other more complicated deployment scenarios. Just follow the Unix philosophy and use simple tools that do one thing and do it well. Init scripts are not that complicated.
See also:
create systemd nodejs express-generator

Related

Linux service "Failed with result 'start-limit-hit'." error while trying to run jar file from automated script?

I have some scripts to restart a java jar service in linux machine taken from this post
and here is my service script:
[Unit]
Description=demo restarter
After=network.target
[Service]
Type=oneshot
ExecStart=systemctl stop demo.service
StartLimitIntervalSec=0
StartLimitBurst=0
ExecStartPost=systemctl start demo.service
[Install]
WantedBy=multi-user.target
If I manually add or remove file from the directory it works fine. But my main purpose is deploying jar file from bitbucket to droplet linux vm and if the file(jar) comes from bitbucket then I get the error:
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: Failed to start DEMOOOOOO Spring Boot application service.
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: demo.service: Start request repeated too quickly.
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: demo.service: Failed with result 'start-limit-hit'.
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: Failed to start DEMOOOOOO Spring Boot application service.
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: demo.service: Start request repeated too quickly.
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: demo.service: Failed with result 'start-limit-hit'.
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: Failed to start DEMOOOOOO Spring Boot application service.
I did
StartLimitIntervalSec=0
StartLimitBurst=0
as suggested in some posts but still I am getting demo.service: Failed with result 'start-limit-hit'. error.
How can I solve this problem ? (I am suspicious about that when I deploy the jar from bitbucket it is overriding the original jar and maybe it makes problem??)
And here is the service file:
[Unit]
Description=DEMOOOOOO Spring Boot application service
After=network.target
[Service]
#User=ubuntu
Type=simple
ExecStart=java -jar /root/artifacts2/demo-0.0.1-SNAPSHOT.jar
TimeoutStopSec=10
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
here path file:
[Path]
Unit=demo-watcher.service
PathModified=/root/artifacts2
[Install]
WantedBy=multi-user.target
I solved it.
I have changed PathModified= to PathChanged= in path script and the problem is solved.
sometimes this message can be misleading, if above fix doesnt work, try changing "User" and also try upgrading or downgrading java version. This can be due to a java version mismatch too..

Binary from app using winston package don't run as linux service

I have a very simple node js application that uses the npm Winston package.
The code is only a loop printing date using the Winston package The app runs just fine. I created a binary using
npm run build
or
npx pkg -t linux,macos,win . --out-path dist
and it also runs just fine. the problem is that when I run it as a service using the file
[Unit]
Description=Cryptween daemon service
After=systend-user-sessions.service
[Service]
Type=simple
WorkingDirectory=$CRYPTWEEN_BIN_DIR
ExecStart=$CRYPTWEEN_BIN_DIR/cryptween-daemon-linux
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=CttWBot_Console
[Install]
WantedBy=multi-user.target
I get the error
cryptween.service - Cryptween daemon service
Loaded: loaded (/etc/systemd/system/cryptween.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2022-03-24 10:54:45 -03; 1s ago
Process: 213784 ExecStart=/opt/cryptween/bin/cryptween-daemon-linux (code=exited, status=1/FAILURE)
Main PID: 213784 (code=exited, status=1/FAILURE)
mar 24 10:54:45 gintama CttWBot_Console[213784]: at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
mar 24 10:54:45 gintama CttWBot_Console[213784]: at Module.load (node:internal/modules/cjs/loader:981:32)
mar 24 10:54:45 gintama CttWBot_Console[213784]: at Function.Module._load (node:internal/modules/cjs/loader:822:12)
mar 24 10:54:45 gintama CttWBot_Console[213784]: at Module.require (node:internal/modules/cjs/loader:1005:19)
mar 24 10:54:45 gintama CttWBot_Console[213784]: at Module.require (pkg/prelude/bootstrap.js:1719:31)
mar 24 10:54:45 gintama CttWBot_Console[213784]: at require (node:internal/modules/cjs/helpers:102:18)
mar 24 10:54:45 gintama CttWBot_Console[213784]: at Object.<anonymous> (/snapshot/test-error-daemon/src/config/winston-logger.js:2:16)
mar 24 10:54:45 gintama CttWBot_Console[213784]: at Module._compile (pkg/prelude/bootstrap.js:1794:22)
mar 24 10:54:45 gintama systemd[1]: cryptween.service: Main process exited, code=exited, status=1/FAILURE
mar 24 10:54:45 gintama systemd[1]: cryptween.service: Failed with result 'exit-code'.
Here is the code:
code
I am using ubuntu 20.04
Thank you in advance

How can I start Docusaurus as a service on Linux (Ubuntu)

Hi guys I try to start Docusaurus V2 as a service on ubuntu headless. Currently I have a script in my Docusaurus folder as a temporarly solution. It works when I start it with ./start.sh:
/var/www/citro-docs-2/start.sh
#!/bin/bash
npm run serve
I tried to create a service like that:
/etc/systemd/system/docusaurus.service
[Unit]
Description=Docusaurus Service
[Service]
ExecStart=/var/www/citro-docs-2/start.sh
WorkingDirectory=/var/www/citro-docs-2
[Install]
WantedBy=multi-user.target
But when I start it with sudo systemctl start docusaurus.service and check the status I get these errors:
● docusaurus.service - Docusaurus Service
Loaded: loaded (/etc/systemd/system/docusaurus.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2021-08-25 11:53:20 UTC; 4s ago
Process: 3494 ExecStart=/var/www/citro-docs-2/start.sh (code=exited, status=1/FAILURE)
Main PID: 3494 (code=exited, status=1/FAILURE)
Aug 25 11:53:20 citro-docs2 start.sh[3521]: at require (node:internal/modules/cjs/helpers:94:18)
Aug 25 11:53:20 citro-docs2 start.sh[3521]: at Object.<anonymous> (/var/www/citro-docs-2/node_modules/#docusaurus/c>
Aug 25 11:53:20 citro-docs2 start.sh[3521]: at Module._compile (node:internal/modules/cjs/loader:1101:14)
Aug 25 11:53:20 citro-docs2 start.sh[3521]: at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153>
Aug 25 11:53:20 citro-docs2 start.sh[3521]: at Module.load (node:internal/modules/cjs/loader:981:32)
Aug 25 11:53:20 citro-docs2 start.sh[3521]: at Function.Module._load (node:internal/modules/cjs/loader:822:12)
Aug 25 11:53:20 citro-docs2 start.sh[3521]: at Function.executeUserEntryPoint [as runMain] (node:internal/modules/r>
Aug 25 11:53:20 citro-docs2 start.sh[3521]: at node:internal/main/run_main_module:17:47
Aug 25 11:53:20 citro-docs2 systemd[1]: docusaurus.service: Main process exited, code=exited, status=1/FAILURE
Aug 25 11:53:20 citro-docs2 systemd[1]: docusaurus.service: Failed with result 'exit-code'.
I found a solution where I can start the .js file directly: How do I run a node.js app as a background service? but I don't know if this is also possible for Docusaurus. At least I don't know which .js file I should start. It would be awesome if someone could help.
Yes, it is possible to run Docusaurus as a service on Ubuntu. It's important to mention that what you're trying to do here is quite normal for Node.js apps. After all, if you don't run your Node.js app as a service, then the service would stop running as soon as you close the shell you used to start the app.
However, it seems like most folks don't run Docusaurus on Ubuntu since it's "just a static site" and static sites don't really need a full blown server to run on. For example, you could leverage GitHub Pages to run your docs site.
That being said, Docusaurus is just a Node.js app. It's similar to all other Node.js apps, and you can certainly run it in Ubuntu if you wanted to.
Take a look at the docs on PM2.
The way I would start your app as a service is:
Install Node.js on your Ubuntu server (if it isn't already installed)
Install pm2 globally on your server via npm install -g pm2
Navigate to your app's directory and run npm install to install the dependencies of your app.
Now you can run pm2 start /var/www/citro-docs-2/start.sh --name myDocsSite
Your Docusaurus app should now be running. To stop it, you can run pm2 stop myDocsSite.

NODE APP: Systemd startup script not working?

Trying to create a start up script for my nodejs app which runs on port 3000.
Issue: The node-app.server script is not working and I think it's because ExecStart pathway is wrong. When I go to the server IP in Chrome nothing shows.
The node app was created with npm generator, and I normally use npm start to start the app. I've added path to bin/wwww, here:
[Unit]
Description=tweetMonster twtiter server - making your environment variables rad
Documentation=https://example.com
After=network.target
[Service]
Environment=NODE_PORT=3000
Type=simple
User=ubuntu
ExecStart=/home/ubuntu/twitter-server/bin/www.js
Restart=on-failure
[Install]
WantedBy=multi-user.target
My app is running on Ubuntu 18 at /home/ubuntu/twitter-server . And if do ls:
/twitter-server$ ls
app.js node_modules package.json routes
bin package-lock.json public views
Please help!
ERROR TERMINAL:
Nov 01 05:40:25 ip-172-31-22-207 systemd[1]: node-app.service: Main process exited, code=exited, status=203/EXEC
Nov 01 05:40:25 ip-172-31-22-207 systemd[1]: node-app.service: Failed with result 'exit-code'.
Nov 01 05:40:26 ip-172-31-22-207 systemd[1]: node-app.service: Service hold-off time over, scheduling restart.
Nov 01 05:40:26 ip-172-31-22-207 systemd[1]: node-app.service: Scheduled restart job, restart counter is at 5.
Nov 01 05:40:26 ip-172-31-22-207 systemd[1]: Stopped hello_env.js - making your environment variables rad.
Nov 01 05:40:26 ip-172-31-22-207 systemd[1]: node-app.service: Start request repeated too quickly.
Nov 01 05:40:26 ip-172-31-22-207 systemd[1]: node-app.service: Failed with result 'exit-code'.
Nov 01 05:40:26 ip-172-31-22-207 systemd[1]: Failed to start hello_env.js - making your environment variables rad.
Nov 01 06:20:11 ip-172-31-22-207 systemd[1]: /etc/systemd/system/node-app.service:10: Executable path is not absolute: "node /home/ubuntu/twitter-server/bin/www.js"
Nov 01 06:24:35 ip-172-31-22-207 systemd[1]: /etc/systemd/system/node-app.service:10: Executable path is not absolute: "node /home/ubuntu/twitter-server/bin/www.js"
root#ip-172-31-22-207:/etc/systemd/system#
There are 2 issues with your service config.
First, wrap the value of Environment with double quotes:
Environment="NODE_PORT=3000"
Second,
You need to use node to run the ExecStart script. /home/ubuntu/twitter-server/bin/www.js is no command in itself.
Do,
ExecStart=/bin/bash -c '$$(which node) /home/ubuntu/twitter-server/bin/www.js'
I recommend this package (service-systemd) for a simple program
Sometimes you just want an "old style" daemon for simple services.
Sometimes you have to deploy in small devices (like a RaspberryPi) and you can't use Docker and all the band.

CentOS 7 - boot order needs to be changed in order for sge to start automatically

It seems like sge tries start before lustre is mounted when the server boots, which brings an error to start automatically when it reboots.
Can somebody tell me how to change the order when it boots, so sge starts after lustre is mounted?
Error message from the log:
Aug 12 11:46:21 dragen1 systemd: Configuration file /usr/lib/systemd/system/sge_execd.service is marked executable. Please remove executable permission bits. Proceeding anyway.
Aug 12 11:46:40 dragen1 sge_execd: error: SGE_ROOT directory "/cm/shared/apps/sge/2011.11p1" doesn't exist
Aug 12 11:46:40 dragen1 systemd: sge_execd.service: control process exited, code=exited status=1
Aug 12 11:46:40 dragen1 systemd: Unit sge_execd.service entered failed state.
Aug 12 11:46:40 dragen1 systemd: sge_execd.service failed
I added in the following under [Unit] from the sge service
RequiresMountsFor=(Mount Point)
This fixed the problem.

Resources