I've added a service (Seafile, in this case) that I want to have running at all times to systemd with a service file. It works great, but every time the unattended updates run, the service gets shutdown properly - but never restarted.
Here's what the service file looks like:
[Unit]
Description=Seafile
Requires=mysql.service
After=mysql.service
[Install]
WantedBy=multi-user.target
[Service]
Type=forking
User=root
Group=root
PermissionsStartOnly=true
ExecStart=/srv/start-seafile
TimeoutSec=600
Restart=on-failure
/srv/start-seafile looks like this:
#!/bin/bash
cd /srv/seafile/XXXX/seafile-server-latest && nohup ./seafile.sh start
Like I said - this works perfectly - systemd can enable / disable / start / stop the service, realize if it's started / running - so I must be doing something right.
# systemctl start seafile
# systemctl status seafile
* seafile.service - Seafile
Loaded: loaded (/etc/systemd/system/seafile.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2018-01-28 17:53:51 CET; 8s ago
Process: 6569 ExecStart=/srv/start-seafile (code=exited, status=0/SUCCESS)
CGroup: /system.slice/seafile.service
|-6598 /srv/seafile/XXXX/seafile-server-5.1.4/seafile/bin/seafile-controller -c /srv/seafile/XXXX/ccnet -d /volume1/Seafile -F /srv/seafile/XXXX/conf
|-6600 ccnet-server -F /srv/seafile/XXXX/conf -c /srv/seafile/XXXX/ccnet -f /srv/seafile/XXXX/logs/ccnet.log -d -P /srv/seafile/XXXX/pids/ccnet.pid
`-6602 seaf-server -F /srv/seafile/XXXX/conf -c /srv/seafile/XXXX/ccnet -d /volume1/Seafile -l /srv/seafile/XXXX/logs/seafile.log -P /srv/seafile/XXXX/pids/seaf-server.pid
Jan 28 17:53:48 XXXX systemd[1]: Starting Seafile...
Jan 28 17:53:48 XXXX start-seafile[6569]: [01/28/18 17:53:48] ../common/session.c(132): using config file /srv/seafile/XXXX/conf/ccnet.conf
Jan 28 17:53:48 XXXX start-seafile[6569]: Starting seafile server, please wait ...
Jan 28 17:53:51 XXXX start-seafile[6569]: Seafile server started
Jan 28 17:53:51 XXXX start-seafile[6569]: Done.
Jan 28 17:53:51 XXXX systemd[1]: Started Seafile.
However, every time the unattended updates come, this happens:
Jan 23 06:38:08 XXXX systemd[1]: Starting Daily apt activities...
Jan 23 06:40:13 XXXX systemd[1]: Reloading.
Jan 23 06:40:13 XXXX systemd[1]: Stopping Seahub...
Jan 23 06:40:14 XXXX systemd[1]: Stopped Seahub.
Jan 23 06:40:14 XXXX systemd[1]: Stopping Seafile...
Jan 23 06:40:15 XXXX systemd[1]: Stopped Seafile.
Jan 23 06:40:15 XXXX systemd[1]: Stopping MySQL Community Server...
Jan 23 06:40:16 XXXX systemd[1]: Stopped MySQL Community Server.
Jan 23 06:40:17 XXXX systemd[1]: Reloading.
Jan 23 06:40:17 XXXX systemd[1]: Stopped MySQL Community Server.
...
Jan 23 06:40:43 XXXX systemd[1]: Reloading.
Jan 23 06:40:44 XXXX systemd[1]: Stopped MySQL Community Server.
Jan 23 06:40:44 XXXX systemd[1]: Reloading.
Jan 23 06:40:44 XXXX systemd[1]: Reloading.
Jan 23 06:40:44 XXXX systemd[1]: Starting MySQL Community Server...
Jan 23 06:40:45 XXXX systemd[1]: Started MySQL Community Server.
Jan 23 06:40:47 XXXX systemd[1]: Reloading.
Jan 23 06:40:48 XXXX systemd[1]: Stopping MySQL Community Server...
Jan 23 06:40:49 XXXX systemd[1]: Stopped MySQL Community Server.
Jan 23 06:40:49 XXXX systemd[1]: Starting MySQL Community Server...
Jan 23 06:40:50 XXXX systemd[1]: Started MySQL Community Server.
(... continues with unrelated services )
So it realizes that Seafile needs to be stopped before MySQL, and does so, but no longer starts it after it restarts MySQL.
Does anyone have any experience as to what could be causing this? ie. under which circumstances systemd services will be stopped during an update, but not restarted?
Use Restart=always if you want it to run at all times. When mysql service update happens, this service does a clean stop and therefore systemd doesn't restart it. You have Restart=on-failure set, which only restarts if the stop has a return code other than 0.
Restart = always
RestartSec = 10
RestartSec
Configures the time to sleep before restarting a service (as configured with Restart=). Takes a unit-less value in seconds, or a time span value such as "5min 20s". Defaults to 100ms.
Related
I created /etc/systemd/system/veracrypt-automount-devices.service:
[Unit]
Description=VeraCrypt auto-mount device-hosted volumes
[Service]
Type=forking
ExecStartPre=/bin/sleep 300
ExecStart=/usr/bin/veracrypt --auto-mount=devices /media/veracrypt1
[Install]
WantedBy=multi-user.target
Then I did:
sudo systemctl daemon-reload
sudo systemctl enable veracrypt-automount-devices
sudo systemctl status veracrypt-automount-devices
● veracrypt-automount-devices.service - VeraCrypt auto-mount device-hosted volumes
Loaded: loaded (/etc/systemd/system/veracrypt-automount-devices.service; enabled; vendor preset: enabled)
Active: failed (Result: timeout) since Sat 2020-06-06 17:28:59 CEST; 8min ago
Process: 967 ExecStartPre=/bin/sleep 300 (code=killed, signal=TERM)
Jun 06 17:27:29 username-computername systemd[1]: Starting VeraCrypt auto-mount device-hosted volumes...
Jun 06 17:28:59 username-computername systemd[1]: veracrypt-automount-devices.service: Start-pre operation timed out. Terminating.
Jun 06 17:28:59 username-computername systemd[1]: veracrypt-automount-devices.service: Failed with result 'timeout'.
Jun 06 17:28:59 username-computername systemd[1]: Failed to start VeraCrypt auto-mount device-hosted volumes.
As you can see, it doesn't work.
If I grep syslog, here is what I find:
Jun 6 16:56:08 username-computername systemd[1]: veracrypt-automount-devices.service: Control process exited, code=exited status=1
Jun 6 16:56:08 username-computername veracrypt[969]: Enter password:
Jun 6 16:56:08 username-computername systemd[1]: veracrypt-automount-devices.service: Failed with result 'exit-code'.
Jun 6 17:28:59 username-computername systemd[1]: veracrypt-automount-devices.service: Start-pre operation timed out. Terminating.
Jun 6 17:28:59 username-computername systemd[1]: veracrypt-automount-devices.service: Failed with result 'timeout'.
Basically, what I want is to be asked for the password to decrypt the device-hosted volume after I logged in with my username and password in Linux Mint.
I found how to do it. I put the veracrypt command in ~/.profile to execute the program on login. See https://askubuntu.com/a/270050/787567.
I installed Postgres 12.3 from source code with steps(according to this):
./configure --with-openssl --with-systemd
make
sudo make install
If I start with pg_ctl from postgres user all works fine:
pg_ctl -D $PGDATA -l /path/to/logfile
Then I try to create a systemd service, as described here.
Steps:
Create file /etc/systemd/system/postgresql.service with content:
[Unit]
Description=PostgreSQL database server
Documentation=man:postgres(1)
[Service]
Type=notify
User=postgres
ExecStart=/usr/local/pgsql/bin/postgres -D /path/to/pgdata
ExecReload=/bin/kill -HUP $MAINPID
KillMode=mixed
KillSignal=SIGINT
TimeoutSec=0
[Install]
WantedBy=multi-user.target
sudo systemctl enable postgresql.service
Then I reboot my machine.
After restart Postgres unavaliable. Some logs:
sudo systemctl status postgresql.service
postgresql.service - PostgreSQL database server
Loaded: loaded (/etc/systemd/system/postgresql.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2020-06-05 03:23:32 MSK; 37s ago
Docs: man:postgres(1)
Process: 724 ExecStart=/usr/local/pgsql/bin/postgres -D /path/to/pgdata (code=exited, status=1/FAILURE)
Main PID: 724 (code=exited, status=1/FAILURE)
Jun 05 03:23:31 ctsvc systemd[1]: Starting PostgreSQL database server...
Jun 05 03:23:32 ctsvc systemd[1]: postgresql.service: Main process exited, code=exited, status=1/FAILURE
Jun 05 03:23:32 ctsvc systemd[1]: Failed to start PostgreSQL database server.
Jun 05 03:23:32 ctsvc systemd[1]: postgresql.service: Unit entered failed state.
Jun 05 03:23:32 ctsvc systemd[1]: postgresql.service: Failed with result 'exit-code'.
journalctl -xe | grep postgres
-- Subject: Unit postgresql.service has begun start-up
-- Unit postgresql.service has begun starting up.
Jun 05 03:23:32 ctsvc postgres[724]: 2020-06-05 03:23:32.209 MSK [724] LOG: starting PostgreSQL 12.3 on armv7l-unknown-linux-gnueabihf, compiled by gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516, 32-bit
Jun 05 03:23:32 ctsvc postgres[724]: 2020-06-05 03:23:32.211 MSK [724] LOG: could not bind IPv4 address "172.17.17.42": Cannot assign requested address
Jun 05 03:23:32 ctsvc postgres[724]: 2020-06-05 03:23:32.211 MSK [724] HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
Jun 05 03:23:32 ctsvc postgres[724]: 2020-06-05 03:23:32.211 MSK [724] WARNING: could not create listen socket for "172.17.17.42"
Jun 05 03:23:32 ctsvc postgres[724]: 2020-06-05 03:23:32.211 MSK [724] FATAL: could not create any TCP/IP sockets
Jun 05 03:23:32 ctsvc postgres[724]: 2020-06-05 03:23:32.212 MSK [724] LOG: database system is shut down
Jun 05 03:23:32 ctsvc systemd[1]: postgresql.service: Main process exited, code=exited, status=1/FAILURE
-- Subject: Unit postgresql.service has failed
-- Unit postgresql.service has failed.
Jun 05 03:23:32 ctsvc systemd[1]: postgresql.service: Unit entered failed state.
Jun 05 03:23:32 ctsvc systemd[1]: postgresql.service: Failed with result 'exit-code'.
Jun 05 03:24:09 ctsvc sudo[1602]: user1 : TTY=pts/0 ; PWD=/home/user1 ; USER=root ; COMMAND=/bin/systemctl status postgresql.service
netstat -tnl | grep "5432" - shows nothing.
After that I can manualy run this service:
sudo systemctl status postgresql.service
● postgresql.service - PostgreSQL database server
Loaded: loaded (/etc/systemd/system/postgresql.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2020-06-05 03:30:57 MSK; 8s ago
Docs: man:postgres(1)
Main PID: 1681 (postgres)
Tasks: 8 (limit: 4915)
CGroup: /system.slice/postgresql.service
├─1681 /usr/local/pgsql/bin/postgres -D /path/to/pgdata
├─1683 postgres: checkpointer
├─1684 postgres: background writer
├─1685 postgres: walwriter
├─1686 postgres: autovacuum launcher
├─1687 postgres: stats collector
├─1688 postgres: logical replication launcher
└─1693 postgres: postgres postgres 172.17.17.40(53600) idle
Jun 05 03:30:56 ctsvc systemd[1]: Starting PostgreSQL database server...
Jun 05 03:30:57 ctsvc postgres[1681]: 2020-06-05 03:30:57.006 MSK [1681] LOG: starting PostgreSQL 12.3 on armv7l-unknown-linux-gnueabihf, compiled b
Jun 05 03:30:57 ctsvc postgres[1681]: 2020-06-05 03:30:57.007 MSK [1681] LOG: listening on IPv4 address "172.17.17.42", port 5432
Jun 05 03:30:57 ctsvc postgres[1681]: 2020-06-05 03:30:57.032 MSK [1681] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
Jun 05 03:30:57 ctsvc postgres[1681]: 2020-06-05 03:30:57.424 MSK [1682] LOG: database system was shut down at 2020-06-05 02:59:03 MSK
Jun 05 03:30:57 ctsvc postgres[1681]: 2020-06-05 03:30:57.725 MSK [1681] LOG: database system is ready to accept connections
Jun 05 03:30:57 ctsvc systemd[1]: Started PostgreSQL database server.
netstat -tnl | grep '5432'
tcp 0 0 172.17.17.42:5432 0.0.0.0:* LISTEN
In my postgresql.conf I have following:
# - Connection Settings -
listen_addresses = '172.17.17.42'
port = 5432
max_connections = 100
If it helps: Postgres runs on Cubietruck with Armbian.
uname -a
Linux ctsvc 4.19.62-sunxi #5.92 SMP Wed Jul 31 22:07:23 CEST 2019 armv7l GNU/Linux
In my system there are no more processes that try to bind this port at boot time. As far as I understand, with the service itself and Postgresql everything is fine. However, something strange happens during the launch, but I can’t understand how to find out the reason of this behavior.
Thanks in advance.
Finally my file /etc/systemd/system/postgresql.service looks like this:
[Unit]
Description=PostgreSQL database server
Documentation=man:postgres(1)
Wants=network-online.target
After=network.target network-online.target
[Service]
Type=notify
User=postgres
ExecStart=/usr/local/pgsql/bin/postgres -D /path/to/pgdata
ExecReload=/bin/kill -HUP $MAINPID
KillMode=mixed
KillSignal=SIGINT
TimeoutSec=0
[Install]
WantedBy=multi-user.target
Thanks to Laurenz Albe comment, I added following in Unit section:
Wants=network-online.target
After=network.target network-online.target
to make sure that network fully operational before PG start. After this PG running correctly after reboot.
I am running this systemd command but when I screen -ls I don't see the screen.
The status is active and seems well.
But it isn't actually running when I check.
This is the .service file
[Unit]
Description=webhookdaemon
[Service]
ExecStart=/bin/bash path/to/script
RemainAfterExit=yes
Type=forking
Restart=on-failure
RestartSec=30
[Install]
WantedBy=multi-user.target
Here is the script (path/to/script)
screen -S docker-hub-daemon -d -m npm run start --prefix /root/nodeserver/
Here is the status output
webookdaemon.service - webhookdaemon
Loaded: loaded (/etc/systemd/system/webookdaemon.service; enabled; vendor preset: enabled)
Active: active (exited) since Tue 2018-03-13 19:55:15 UTC; 57min ago
Main PID: 2144 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/webookdaemon.service
Mar 13 19:58:29 aggregate-terminal-logs-tor1-01 systemd[1]: Started webhookdaemon.
Mar 13 19:59:03 aggregate-terminal-logs-tor1-01 systemd[1]: Started webhookdaemon.
Mar 13 20:00:22 aggregate-terminal-logs-tor1-01 systemd[1]: Started webhookdaemon.
Mar 13 20:01:21 aggregate-terminal-logs-tor1-01 systemd[1]: Started webhookdaemon.
Mar 13 20:02:26 aggregate-terminal-logs-tor1-01 systemd[1]: Started webhookdaemon.
Mar 13 20:04:41 aggregate-terminal-logs-tor1-01 systemd[1]: Started webhookdaemon.
Mar 13 20:47:41 aggregate-terminal-logs-tor1-01 systemd[1]: Started webhookdaemon.
Mar 13 20:49:53 aggregate-terminal-logs-tor1-01 systemd[1]: Started webhookdaemon.
Mar 13 20:52:53 aggregate-terminal-logs-tor1-01 systemd[1]: Started webhookdaemon.
root#aggregate-terminal-logs-tor1-01:~#
You should not be using screen to manage services. Just use systemd directly.
Make sure that Type= is set match the behavior of the service you are lauching. I could not find references to docker-hub-daemon, so I'm not sure the appropriate value for it. See man systemd.service for the documentation for Type=.
Instead of using screen -ls to check the status of the service, use systemctl status webookdaemon.
You may also wish to update the spelling of this service to be webhoookdaemon to match the spelling in the description.
why my service always exited when running systemd?
if change dir manualy to var/www/enterprise/src/ and execute /var/www/env/bin/celery -A catalog.tasks beat , proccess work perfectly
root#debian-django-nginx:/var/www/enterprise/src# cat /etc/systemd/system/celerybeat.service
[Unit]
Description=celery beat
[Service]
WorkingDirectory=/var/www/enterprise/src/
Type=simple
ExecStart=/var/www/env/bin/celery -A catalog.tasks beat &
KillMode=process
TimeoutSec=300
[Install]
WantedBy=multi-user.target
root#debian-django-nginx:/var/www/enterprise/src# systemctl daemon-reload
root#debian-django-nginx:/var/www/enterprise/src# systemctl start celerybeat
root#debian-django-nginx:/var/www/enterprise/src# systemctl status celerybeat
● celerybeat.service - celery beat
Loaded: loaded (/etc/systemd/system/celerybeat.service; enabled)
Active: active (exited) since Tue 2017-02-28 15:01:52 +10; 1 weeks 0 days ago
Mar 07 14:11:55 debian-django-nginx systemd[1]: celerybeat.service lacks ExecStart setting. Refusing.
Mar 07 14:28:08 debian-django-nginx systemd[1]: Started celery beat.
Mar 07 14:39:34 debian-django-nginx systemd[1]: Started celery beat.
Mar 07 15:05:56 debian-django-nginx systemd[1]: Started celery beat.
Mar 07 15:08:46 debian-django-nginx systemd[1]: Started celery beat.
Mar 07 15:19:35 debian-django-nginx systemd[1]: Started celery beat.
Mar 07 15:33:38 debian-django-nginx systemd[1]: Started celery beat.
I am trying to run xsp4 as a systemd service in order to host WebAPI 2 application on Ubuntu 16.04. This is my service file:
[Unit]
Description=myapp xsp4 WebAPI 2
Wants=network.target
[Service]
Type=simple
User=www-data
Group=www-data
EnvironmentFile=/etc/myapp/env
ExecStart=/usr/bin/xsp4 --appconfigdir /etc/xsp4/ --root /var/www/myapp.com/server --address 127.0.0.1
Restart=on-failure
[Install]
WantedBy=multi-user.target
The command in ExecStart works fine when ran in shell (as root). When I start via systemd I get:
root#ubuntu-512mb-ams3-01:/etc/systemd/system# sctl status myapp.service
● myapp.service - myapp xsp4 WebAPI 2
Loaded: loaded (/etc/systemd/system/myapp.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Oct 07 14:32:00 ubuntu-512mb-ams3-01 systemd[1]: myapp.service: Start request repeated too quickly.
Oct 07 14:32:00 ubuntu-512mb-ams3-01 systemd[1]: Failed to start myapp xsp4 WebAPI 2.
Oct 07 14:59:12 ubuntu-512mb-ams3-01 systemd[1]: Stopped myapp xsp4 WebAPI 2.
Oct 07 14:59:16 ubuntu-512mb-ams3-01 systemd[1]: Started myapp xsp4 WebAPI 2.
Oct 07 14:59:19 ubuntu-512mb-ams3-01 xsp4[26490]: xsp4
Oct 07 14:59:19 ubuntu-512mb-ams3-01 xsp4[26490]: Listening on address: 127.0.0.1
Oct 07 14:59:19 ubuntu-512mb-ams3-01 xsp4[26490]: Root directory: /var/www/myapp.com/server
Oct 07 14:59:19 ubuntu-512mb-ams3-01 xsp4[26490]: Listening on port: 9000 (non-secure)
Oct 07 14:59:19 ubuntu-512mb-ams3-01 xsp4[26490]: Hit Return to stop the server.
Oct 07 14:59:19 ubuntu-512mb-ams3-01 xsp4[26490]: [0x7f24851e9700:] EXCEPTION handling: System.Threading.ThreadAbortException:
That System.Threading.ThreadAbortException does not occur when ran in shell. The actual error is "Start request repeated too quickly." because I configured mono to output all exceptions, even if caught - so this exception is not cause of exit.
Any advice ?
I will add the answer, because I myself was looking for.
[Service]
TimeoutStartSec=100
ExecStart=/usr/bin/xsp4 --appconfigdir /etc/xsp4/ --root /var/www/myapp.com/server --address 127.0.0.1 --nonstop