Linux service "Failed with result 'start-limit-hit'." error while trying to run jar file from automated script? - linux

I have some scripts to restart a java jar service in linux machine taken from this post
and here is my service script:
[Unit]
Description=demo restarter
After=network.target
[Service]
Type=oneshot
ExecStart=systemctl stop demo.service
StartLimitIntervalSec=0
StartLimitBurst=0
ExecStartPost=systemctl start demo.service
[Install]
WantedBy=multi-user.target
If I manually add or remove file from the directory it works fine. But my main purpose is deploying jar file from bitbucket to droplet linux vm and if the file(jar) comes from bitbucket then I get the error:
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: Failed to start DEMOOOOOO Spring Boot application service.
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: demo.service: Start request repeated too quickly.
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: demo.service: Failed with result 'start-limit-hit'.
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: Failed to start DEMOOOOOO Spring Boot application service.
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: demo.service: Start request repeated too quickly.
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: demo.service: Failed with result 'start-limit-hit'.
Sep 08 00:09:06 ubuntu-1cpu systemd[1]: Failed to start DEMOOOOOO Spring Boot application service.
I did
StartLimitIntervalSec=0
StartLimitBurst=0
as suggested in some posts but still I am getting demo.service: Failed with result 'start-limit-hit'. error.
How can I solve this problem ? (I am suspicious about that when I deploy the jar from bitbucket it is overriding the original jar and maybe it makes problem??)
And here is the service file:
[Unit]
Description=DEMOOOOOO Spring Boot application service
After=network.target
[Service]
#User=ubuntu
Type=simple
ExecStart=java -jar /root/artifacts2/demo-0.0.1-SNAPSHOT.jar
TimeoutStopSec=10
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
here path file:
[Path]
Unit=demo-watcher.service
PathModified=/root/artifacts2
[Install]
WantedBy=multi-user.target

I solved it.
I have changed PathModified= to PathChanged= in path script and the problem is solved.

sometimes this message can be misleading, if above fix doesnt work, try changing "User" and also try upgrading or downgrading java version. This can be due to a java version mismatch too..

Related

Control python script like linux cli applicaitons

I have a python script which I want to control using linux commands.
e.g. like we control mysql: service mysql restart
How to achieve this kind of functionality.
I saw some bash scripts doing this but don't have any knowledge in bash scripting.
Thanks.
UPDATE:
I have a site_monitor.py script which I included in site_monitor.service in /etc/systemd/system/.
[Unit]
Description=Site Monitor Service
After=multi-user.target
[Service]
Type=simple
Restart=always
ExecStart=/usr/bin/python3 /home/hemantsah/WisdomLeaf/site_monitor/site_monitor.py
[Install]
WantedBy=multi-user.target
After doing systemctl daemon-reload.service, I started the script using systemctl start site_monitor.service
Listing all the services using systemctl list-units --type=service, I can see the service running, but it's not doing anything.
If I run my python script in terminal using python3 site_monitor.py, then it works.
I found just now if I start the service and check the status using sudo service site_monitor status,
I checked after starting the service, it was fine , checking after sometime again gave me this error:
hemantsah#pop-os:/etc/systemd/system$ sudo service site_monitor status
● site_monitor.service - Site Monitor Service
Loaded: loaded (/etc/systemd/system/site_monitor.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2021-11-18 10:47:30 IST; 16s ago
Process: 111989 ExecStart=/usr/bin/python3 /home/hemantsah/WisdomLeaf/site_monitor/site_monitor.py (code=exited, status=1/FAILURE)
Main PID: 111989 (code=exited, status=1/FAILURE)
Nov 18 10:47:30 pop-os systemd[1]: site_monitor.service: Scheduled restart job, restart counter is at 5.
Nov 18 10:47:30 pop-os systemd[1]: Stopped Site Monitor Service.
Nov 18 10:47:30 pop-os systemd[1]: site_monitor.service: Start request repeated too quickly.
Nov 18 10:47:30 pop-os systemd[1]: site_monitor.service: Failed with result 'exit-code'.
Nov 18 10:47:30 pop-os systemd[1]: Failed to start Site Monitor Service.
Nov 18 10:47:34 pop-os systemd[1]: site_monitor.service: Start request repeated too quickly.
Nov 18 10:47:34 pop-os systemd[1]: site_monitor.service: Failed with result 'exit-code'.
Nov 18 10:47:34 pop-os systemd[1]: Failed to start Site Monitor Service.
Running sudo journalctl -u site_monitor.service gave me following error:
Nov 18 10:10:37 pop-os systemd[1]: site_monitor.service: Scheduled restart job, restart counter is at 4.
Nov 18 10:10:37 pop-os systemd[1]: Stopped Site Monitor Service.
Nov 18 10:10:37 pop-os systemd[1]: Started Site Monitor Service.
Nov 18 10:10:37 pop-os python3[111023]: Traceback (most recent call last):
Nov 18 10:10:37 pop-os python3[111023]: File "/home/hemantsah/WisdomLeaf/site_monitor/site_monitor.py", line 3, in <module>
Nov 18 10:10:37 pop-os python3[111023]: from bs4 import BeautifulSoup
Nov 18 10:10:37 pop-os python3[111023]: ModuleNotFoundError: No module named 'bs4'
Nov 18 10:10:37 pop-os systemd[1]: site_monitor.service: Main process exited, code=exited, status=1/FAILURE
Nov 18 10:10:37 pop-os systemd[1]: site_monitor.service: Failed with result 'exit-code'.
I guess you already have systemd in your machine, but in case you don't, you can install it via package manager, e.g. apt:
sudo apt-get install systemd
You can then, create your own systemd service. To do so, just create a new file in /etc/systemd/systemd/, something like /etc/systemd/systemd/your_service_name.service. That file should look like this:
[Unit]
Description= My service
After=multi-user.target
[Service]
Type=simple
WorkingDirectory=/path/to/your/working/dir/
User=<user>
Restart=always
ExecStart=/usr/bin/python3 /path/to/your/script/<script_name>.py
[Install]
WantedBy=multi-user.target
Of course you can use different python 3 binaries instead of /usr/bin/python3. Also, the service configuration itself can be different, the example above is just a basic service structure.
After creating this file (with root permissions), you should reload the daemon with:
sudo systemctl daemon-reload
And if you want to keep your script enabled even if the server/machine restarts, run:
sudo systemctl enable your_service_name.service
Finally, you can start your service using the following:
sudo systemctl start your_service_name.service

Why is rhel8 aws systemd service throws No such file or directory error even when file exists?

I have a systemd service define on rhel8
[Unit]
Description=Apache Kafka - ZooKeeper
Documentation=http://docs.confluent.io/
After=network.target
[Service]
Type=simple
EnvironmentFile=/app/bin/confluent/etc/kafka/zookenv.properties
User=kafka
Group=kafka
ExecStart=/app/bin/confluent/bin/zookeeper-server-start /app/bin/confluent/etc/kafka/zookeeper.properties
TimeoutStopSec=180
Restart=no
[Install]
WantedBy=multi-user.target
when i start this service i get the below error in journalctl
Jul 09 12:00:51 10.204.142.111 systemd[1]: confluent-zookeeper.service: Failed to load environment files: No such file or directory
Jul 09 12:00:51 10.204.142.111 systemd[1]: confluent-zookeeper.service: Failed to run 'start' task: No such file or directory
Jul 09 12:00:51 10.204.142.111 systemd[1]: confluent-zookeeper.service: Failed with result 'resources'.
the environment file exists in the path and so does the start script and properties files.
this is on RHEL8 aws and trying this for the first time.
the component starts up fine when i run the start script manually from command line.
Check that the path and file for
EnvironmentFile=/app/bin/confluent/etc/kafka/zookenv.properties
are correct
In my case, I had my service file like this
ExecStart=/app/bin/confluent/bin/zookeeper-server-start
EnvironmentFile=/app/bin/confluent/etc/kafka/zookenv.properties
I changed it to
EnvironmentFile=/app/bin/confluent/etc/kafka/zookenv.properties
ExecStart=/app/bin/confluent/bin/zookeeper-server-start
Then I run,
systemctl daemon-reload &&
systemctl restart service-name.service`
Initially, I was using `systemctl start service-name.service` and I guess that didn't make systemd read the environment files properly

NODE APP: Systemd startup script not working?

Trying to create a start up script for my nodejs app which runs on port 3000.
Issue: The node-app.server script is not working and I think it's because ExecStart pathway is wrong. When I go to the server IP in Chrome nothing shows.
The node app was created with npm generator, and I normally use npm start to start the app. I've added path to bin/wwww, here:
[Unit]
Description=tweetMonster twtiter server - making your environment variables rad
Documentation=https://example.com
After=network.target
[Service]
Environment=NODE_PORT=3000
Type=simple
User=ubuntu
ExecStart=/home/ubuntu/twitter-server/bin/www.js
Restart=on-failure
[Install]
WantedBy=multi-user.target
My app is running on Ubuntu 18 at /home/ubuntu/twitter-server . And if do ls:
/twitter-server$ ls
app.js node_modules package.json routes
bin package-lock.json public views
Please help!
ERROR TERMINAL:
Nov 01 05:40:25 ip-172-31-22-207 systemd[1]: node-app.service: Main process exited, code=exited, status=203/EXEC
Nov 01 05:40:25 ip-172-31-22-207 systemd[1]: node-app.service: Failed with result 'exit-code'.
Nov 01 05:40:26 ip-172-31-22-207 systemd[1]: node-app.service: Service hold-off time over, scheduling restart.
Nov 01 05:40:26 ip-172-31-22-207 systemd[1]: node-app.service: Scheduled restart job, restart counter is at 5.
Nov 01 05:40:26 ip-172-31-22-207 systemd[1]: Stopped hello_env.js - making your environment variables rad.
Nov 01 05:40:26 ip-172-31-22-207 systemd[1]: node-app.service: Start request repeated too quickly.
Nov 01 05:40:26 ip-172-31-22-207 systemd[1]: node-app.service: Failed with result 'exit-code'.
Nov 01 05:40:26 ip-172-31-22-207 systemd[1]: Failed to start hello_env.js - making your environment variables rad.
Nov 01 06:20:11 ip-172-31-22-207 systemd[1]: /etc/systemd/system/node-app.service:10: Executable path is not absolute: "node /home/ubuntu/twitter-server/bin/www.js"
Nov 01 06:24:35 ip-172-31-22-207 systemd[1]: /etc/systemd/system/node-app.service:10: Executable path is not absolute: "node /home/ubuntu/twitter-server/bin/www.js"
root#ip-172-31-22-207:/etc/systemd/system#
There are 2 issues with your service config.
First, wrap the value of Environment with double quotes:
Environment="NODE_PORT=3000"
Second,
You need to use node to run the ExecStart script. /home/ubuntu/twitter-server/bin/www.js is no command in itself.
Do,
ExecStart=/bin/bash -c '$$(which node) /home/ubuntu/twitter-server/bin/www.js'
I recommend this package (service-systemd) for a simple program
Sometimes you just want an "old style" daemon for simple services.
Sometimes you have to deploy in small devices (like a RaspberryPi) and you can't use Docker and all the band.

Creating a Systemd service Error

I've used golang to create a small program that runs silently in the background, and can run this program by typing ./my_app
It is located in $HOME/my_app/my_app
I went ahead and wrote a systemd my_app.service file:
[Unit]
Description="My Test Service"
[Service]
Type=simple
ExecStart= ~/my_app/my_app
[Install]
WantedBy=multi-user.target
However, when i run sudo systemctl start my_app.service, it errors out and returns
Failed to start my_app.service: Unit my_app.service is not loaded properly: Invalid argument.
See system logs and 'systemctl status my_app.service' for details.
systemctl status my_app.service returns
Apr 07 10:15:47 DEV-Rohan systemd[1]: Starting My Test App...
Apr 07 10:15:47 DEV-Rohan systemd[1]: Started My Test App.
Apr 07 10:36:34 DEV-Rohan systemd[1]: [/etc/systemd/system/my_app.service:6] Executable path is not abs
Apr 07 10:36:34 DEV-Rohan systemd[1]: my_app.service: Service lacks both ExecStart= and ExecStop= setti
Apr 07 11:00:39 DEV-Rohan systemd[1]: [/etc/systemd/system/my_app.service:6] Executable path is not abs
Apr 07 11:00:39 DEV-Rohan systemd[1]: my_app.service: Service lacks both ExecStart= and ExecStop= setti

Systemd Service for jar file gets "operation timed out" error after few minues or stay in "activating mode"

the service unit is:
[Unit]
Description=test
After=syslog.target
After=network.target
[Service]
Type=forking
ExecStart=/bin/java -jar /home/ec2-user/test.jar
TimeoutSec=300
[Install]
WantedBy=multi-user.target
it starts fine for 1-4 minues. But later it fails:
tail /var/log/messages:
Feb 27 18:43:44 ip-172-31-40-48 systemd: Reloading.
Feb 27 18:44:06 ip-172-31-40-48 systemd: Starting test...
Feb 27 18:44:06 ip-172-31-40-48 java: 5.1.73
Feb 27 18:44:06 ip-172-31-40-48 java: Starting the internal [HTTP/1.1] server on port 8182
Feb 27 18:49:06 ip-172-31-40-48 systemd: test.service operation timed out.Terminating.
Feb 27 18:49:06 ip-172-31-40-48 systemd: test.service: control process exited, code=exited status=143
Feb 27 18:49:06 ip-172-31-40-48 systemd: Failed to start test.
Feb 27 18:49:06 ip-172-31-40-48 systemd: Unit test.service entered failed state.
systemctl status test.service (while restarting- stays in activating mode):
test.service - Setsnew
Loaded: loaded (/etc/systemd/system/test.service; enabled)
Active: activating (start) since Sun 2015-03-01 14:29:36 EST; 2min 30s ago
Control: 32462 (java)
CGroup: /system.slice/test.service
systemctl status test.service (after fail):
test.service - test
Loaded: loaded (/etc/systemd/system/test.service; enabled)
Active: failed (Result: exit-code) since Fri 2015-02-27 18:49:06 EST; 18min ago
Process: 27954 ExecStart=/bin/java -jar /home/ec2-user/test.jar (code=exited, status=143)
when running the jar in command line it works just fine.
tried changing the jar location because I thought it's a permissions problem
selinux is off
How can i fix this issue so I could start the jar on boot? there any alternatives? (RHEL7 do not include service command)
You made the service type forking, but this service does not fork. It just runs directly. Thus systemd waited five minutes for the program to daemonize itself, and it never did. The correct type for such a service is simple.
You also disabled SELinux, which is another problem you should resolve.

Resources