systemd service doesn't start silently. How to debug? - linux

I wrote a program (called whisky) which I now want to startup when booting the machine (a Raspberry Pi with which I'm creating an autonomous boat). So I created the file /lib/systemd/system/whisky.service:
[Unit]
Description=Whisky Boat Program
After=network.target
StartLimitIntervalSec=0
[Service]
ExecStart=/home/pi/whisky/run
KillMode=process
IgnoreSIGPIPE=true
Restart=always
RestartSec=3
User=root
Type=simple
[Install]
WantedBy=multi-user.target
I verified the file is correctly formatted for systemd using systemd-analyze verify whisky.service.
When I now run sudo systemctl start whisky I get no output (suggesting no errors).
sudo systemctl status whisky gives me the following output though:
* whisky.service - Whisky Boat Program
Loaded: loaded (/lib/systemd/system/whisky.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Fri 2020-03-20 15:03:35 CET; 792ms ago
Process: 8621 ExecStart=/home/pi/whisky/run (code=exited, status=203/EXEC)
Main PID: 8621 (code=exited, status=203/EXEC)
Mar 20 15:03:35 raspberrypi systemd[1]: whisky.service: Unit entered failed state.
Mar 20 15:03:35 raspberrypi systemd[1]: whisky.service: Failed with result 'exit-code'.
The file /home/pi/whisky/run is actually a bash script which in turn starts the program. To check whether systemd even starts that bash script I added a first line to it: mkdir /home/pi/RUNNING_FROM_SYSTEMD. The dir RUNNING_FROM_SYSTEMD is not created though, so it seems systemd doesn't even try to run the file /home/pi/whisky/run.
Does anybody know what I'm doing wrong here?

Check your run script. There should be shebang. If not, add #!/bin/bash at the top of your script and give absolute path like ExecStart=/bin/bash /home/pi/whisky/run
The error message (code=exited, status=203/EXEC) is often seen when the script itself or its interpreter cannot be executed.
Other possible reasons maybe:
wrong path to script
script not executable
no shebang (first line) or wrong path in shebang
internal files in your script might be missing access permissions.

Related

Problem with a Golang webapp and system service

I am trying to create a Golang server using the gin framework on ubuntu. It works fine when it is executed in the terminal after building it with go build and equally works well locally.
Systemd
Description=goapp
[Service]
Type=simple
Restart=always
RestartSec=5s
ExecStart=/home/.../goapp/main
[Install]
WantedBy=multi-user.target
I got this error
goapp.service - rediate
Loaded: loaded (/lib/systemd/system/goapp.service; disabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Thu 2022-09-29 08:14:10 UTC; 66ms ago
Process: 21628
ExecStart=/home/.../go/goapp/main (code=exited, status=2)
Main PID: 21628 (code=exited, status=2)
CPU: 9ms
Go is compiled language. You need to build your code into an executable binary file using go build command and then give path to binary file to systemd via ExecStart property in unit file.
See Go Documentation and specifically Compile and install the application section to find out more about how to compile your application.
In your example you have ExecStart=/home/.../goapp/main.go which is telling systemd to run source code file. That file is not executable and understood by operating system so it fails to execute and systemd unit fails because of that.
Adding a working directory to the systemd fix this error.
Description=goapp
[Service]
Type=simple
Restart=always
RestartSec=
WorkingDirectory=/home/.../goapp
ExecStart=/home/.../goapp/main
[Install]
WantedBy=multi-user.target

systemd doesn't run an application from bash script

I have a service that should run set of applications in background on my Yocto embedded Linux system. I don't like an idea to create a systemd startup script for each app so I just run them from a bash script as following:
The service:
startup.service
[Unit]
Description=applications startup script
After=network.target
[Service]
Type=simple
ExecStart=/opt/somedir/startup.sh
[Install]
WantedBy=multi-user.target
and the script
startup.sh
#!/bin/bash
echo "application startup script"
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/somedir
/opt/somedir/app1 &
/opt/somedir/app2 &
/opt/somedir/app3 &
But no application started. Checking the service status give me:
systemctl status startup
● startup.service - applications startup script
Loaded: loaded (/lib/systemd/system/startup.service; enabled; vendor preset: enabled)
Active: inactive (dead) since Thu 2021-03-25 10:33:16 UTC; 18min ago
Process: 428 ExecStart=/opt/somedir/startup.sh (code=exited, status=0/SUCCESS)
Main PID: 428 (code=exited, status=0/SUCCESS)
Mar 25 10:33:16 systemd[1]: Started application startup script.
Mar 25 10:33:16 startup.sh[428]: application startup script
Mar 25 10:33:16 systemd[1]: startup.service: Succeeded.
So the service executed on the system startup and executes the script. If I execute the script from the command line it starts the applications as expected. So what a reason that no application run?
Systemd will need to know how to run the script. Therefore either add:
#!/bin/bash
to the top line of the startup.sh script or change the ExecStart line in the systemd service file to:
ExecStart=/bin/bash -c /opt/somedir/startup.sh
Also, to ensure that the processes spawned remain persistent after being spawned, change:
Type=forking
systemd runs script startup.sh, and after that process ends, it assumes all is done so it kills off any remaining processes and the unit ends. The simplest solution is to add a wait at the end of startup.sh so that it only returns when the backgrounded processes have all ended.

Systemd service leaves out command in script

I am trying to start a service named pigpiod.service via systemd. It invokes a script with three commands. The second one is left out. Why is this?
/etc/systemd/system/pigpiod.service:
[Unit]
Description=Starts pigpiod
Before=touchscreen.service
[Service]
ExecStart=/home/sysop/pigpiod.sh
[Install]
WantedBy=multi-user.target
/home/sysop/pigpiod.sh:
#!/bin/sh
touch /home/sysop/before_pigpiod
/usr/bin/pigpiod
touch /home/sysop/after_pigpiod
When restarting the machine the two files get created in /home/sysop/, but pigpiod is not starting.
When starting the service manually via sudo systemctl start pigpiod the same happens.
When running sudo /home/sysop/pigpiod.sh manually pigpiod is actually starting!
This is the output of sudo systemctl status pigpiod -l right after boot:
● pigpiod.service - Starts pigpiod
Loaded: loaded (/etc/systemd/system/pigpiod.service; enabled)
Active: inactive (dead) since Sat 2017-09-16 20:02:03 UTC; 2min 29s ago
Process: 440 ExecStart=/home/sysop/pigpiod.sh (code=exited, status=0/SUCCESS)
Main PID: 440 (code=exited, status=0/SUCCESS)
Sep 16 20:02:02 kivypie systemd[1]: Starting Starts pigpiod...
Sep 16 20:02:02 kivypie systemd[1]: Started Starts pigpiod.
Why is it, that systemd skips the execution of /usr/bin/pigpiod, but manually running the script as root does not?
My system: Raspberry Pi Model 3B, Raspbian GNU/Linux 8 (jessie)
pigpiod forks without the -g option. So use Type = forking or use pigpiod -g
[Unit]
Description=Starts pigpiod
Before=touchscreen.service
[Service]
ExecStart=/home/sysop/pigpiod.sh
Type=forking
[Install]
WantedBy=multi-user.target

Systemd script fail

I want to run a script at system startup in a Debian 9 box. My script works when run standalone, but fails under systemd.
My script just copies a backup file from a remote server to the local machine:
#!/bin/sh
set -e
/usr/bin/sshpass -p "PASSWORD" /usr/bin/scp -p USER#10.0.0.2:ORIGINPATH/backupserver.zip DESTINATIONPATH/backupserver/
Just for privacy I replaced password, user, and paths above.
I wrote the following systemd service unit:
[Unit]
Description=backup script
[Service]
Type=oneshot
ExecStart=PATH/backup.sh
[Install]
WantedBy=default.target
Then I set permissions for the script:
chmod 744 PATH/backup.sh
And installed the service:
chmod 664 /etc/systemd/system/backup.service
systemctl daemon-reload
systemctl enable backup.service
When I reboot the script fails:
● backup.service - backup script
Loaded: loaded (/etc/systemd/system/backup.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sat 2017-05-13 13:39:54 -03; 47min ago
Main PID: 591 (code=exited, status=1/FAILURE)
Result of journalctl -xe:
mai 16 23:34:27 rodrigo-acer systemd[1]: backup.service: Main process exited, code=exited, status=6/NOTCONFIGURED
mai 16 23:34:27 rodrigo-acer systemd[1]: Failed to start backup script.
mai 16 23:34:27 rodrigo-acer systemd[1]: backup.service: Unit entered failed state.
mai 16 23:34:27 rodrigo-acer systemd[1]: backup.service: Failed with result 'exit-code'.
What could be wrong?
Solved guys. There was 2 problems:
1 - I had to change the service unit file to make the service run only after network was up. The unit section was changed to:
[Unit]
Description = World server backup
Wants = network-online.target
After = network.target network-online.target
2 - The root user did not have the remote host added to the known host list, unlike the ordinary user I used to test the script.
Failed with result 'exit-code' you could try this on your last line:
# REQUIRED FOR SYSTEMD: 0 means clean no error
exit 0
You may also need to add:
Type=forking
to the systemd entry similar to: https://serverfault.com/questions/751030/systemd-ignores-return-code-while-starting-service
If your service or script does not fork add a & at the end to run it in the background, and exit with 0 fast. Otherwise it will be like a startup that times out and takes forever / seems like frozen service.

systemd cannot run service after running commands

I tried to run systemd using the commands systemctl enable photogrid.service & systemctl start photogrid.service in ubuntu 16
The nodejs app itself can run as expected. The service is to ensure that application will auto-start when application crash or server reboot.
The service apparently did not start. So I key in systemctl status photogrid.service to see what happened, the below is what I got from the terminal.
● photogrid.service - Photogrid
Loaded: loaded (/lib/systemd/system/photogrid.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Wed 2016-11-09 04:35:36 UTC; 7s ago
Process: 27523 ExecStart=/usr/local/bin/node /home/ubuntu/photogrid/app.js (code=exited, status=203/EXEC)
Main PID: 27523 (code=exited, status=203/EXEC)
Nov 09 04:35:36 ip-172-31-34-151 systemd[1]: photogrid.service: Main process exited, code=exited, status=203/EXEC
Nov 09 04:35:36 ip-172-31-34-151 systemd[1]: photogrid.service: Unit entered failed state.
Nov 09 04:35:36 ip-172-31-34-151 systemd[1]: photogrid.service: Failed with result 'exit-code'.
This the script that I wrote for the service under the path /lib/systemd/system/photogrid.service
[Unit]
Description=Photogrid
[Service]
Type=simple
Restart=always
RestartSec=10
Environment=NODE_ENV=production
ExecStart=/usr/local/bin/node /home/ubuntu/photogrid/app.js
[Install]
WantedBy=multi-user.target
Basically under ExecStart make sure you point to the correct nodejs executable. For my case it was in a different folder and not /usr/local/bin/node, to check where is your node executable. (Assuming you confirm you have downloaded and install it correctly in linux) use command which node to give you path direction.

Resources