Start Airflow Webserver Automaticalll as a Service - linux

I currently would like to write a system configuration file that starts the airflow webserver. Here is my config file definition:
#airflow-webserver.service
[Unit]
Description=Airflow webserver daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service
[Service]
#EnvironmentFile=/etc/default/airflow
Environment="PATH=/root/airflow_venv/bin"
User=airflow
Group=airflow
Type=simple
ExecStart=/root/airflow_venv/bin/airflow webserver -p 8080 --pid /root/airflow/airflow-webserver.pid
Restart=on-failure
RestartSec=5s
PrivateTmp=true
[Install]
WantedBy=multi-user.target
However running "sudo service airflow-webserver status" it return:
● airflow-webserver.service - Airflow webserver daemon
Loaded: loaded (/etc/systemd/system/airflow-webserver.service; enabled; vendor preset: disabled)
Active: activating (auto-restart) (Result: exit-code) since Mon 2021-01-25 06:58:14 UTC; 3s ago
Process: 5913 ExecStart=/root/airflow_venv/bin/airflow webserver -p 8080 --pid /root/airflow/airflow-webserver.pid (code=exited, status=217/USER)
Main PID: 5913 (code=exited, status=217/USER)
CGroup: /system.slice/airflow-webserver.service
I already looked at How to enable a virtualenv in a systemd service unit? as it sounds like a similar issue. However, I could not find my solution there. The OS I am operating on is Amazon Linux 2 AMI.
Can somebody help?
Lazloo

For anyone who gets the same Error:
Main PID: 5913 (code=exited, status=217/USER)
this indicates that the user is incorrect. After adapting the user and group, this service worked for me.

Related

Linux service returns (code=exited, status=2)

I have created a Golang web app and built as an executable. The executable is running well when executed as ./main (main is the name of the executable) from its subdirectory.
I'm trying to create it as a service. I have a file goservice.service
[Unit]
Description=goservice
[Service]
Type=simple
Restart=always
RestartSec=5s
ExecStart=/home/nshankar/go/eperssona/main
[Install]
WantedBy=multi-user.target
When I see the status of the service, I get following response -
Loaded: loaded (/etc/systemd/system/goservice.service; enabled; preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Fri 2022-10-28 10:47:54 UTC; 498ms ago
Process: 13286 ExecStart=/home/nshankar/go/eperssona/main (code=exited, status=2)
Main PID: 13286 (code=exited, status=2)
CPU: 4ms
The service is listening on port 9002.
Why am I getting this error?

Failed to stop Apache Spark Master or Slave using Systemd

Perspectives
Actually I needs to configure two service files. One for Spark Master and another for Spark Slave (Worker) node. Please find the environment and service configuration as following:
Cofigurations
/opt/cli/spark-3.3.0-bin-hadoop3/etc/env
JAVA_HOME="/usr/lib/jvm/java-17-openjdk-amd64"
SPARK_HOME="/opt/cli/spark-3.3.0-bin-hadoop3"
PYSPARK_PYTHON="/usr/bin/python3"
/etc/systemd/system/spark-master.service
[Unit]
Description=Apache Spark Master
Wants=network-online.target
After=network-online.target
[Service]
User=spark
Group=spark
Type=forking
WorkingDirectory=/opt/cli/spark-3.3.0-bin-hadoop3/sbin
EnvironmentFile=/opt/cli/spark-3.3.0-bin-hadoop3/etc/env
ExecStartPost=/bin/bash -c "echo $MAINPID > /opt/cli/spark-3.3.0-bin-hadoop3/etc/spark-master.pid"
ExecStart=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/start-master.sh
ExecStop=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/stop-master.sh
[Install]
WantedBy=multi-user.target
/etc/systemd/system/spark-slave.service
[Unit]
Description=Apache Spark Slave
Wants=network-online.target
After=network-online.target
[Service]
User=spark
Group=spark
Type=forking
WorkingDirectory=/opt/cli/spark-3.3.0-bin-hadoop3/sbin
EnvironmentFile=/opt/cli/spark-3.3.0-bin-hadoop3/etc/env
ExecStartPost=/bin/bash -c "echo $MAINPID > /opt/cli/spark-3.3.0-bin-hadoop3/etc/spark-slave.pid"
ExecStart=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/start-slave.sh spark://spark.cdn.chorke.org:7077
ExecStop=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/stop-slave.sh
[Install]
WantedBy=multi-user.target
Outcome
It's started successfully but failed to stop successfully for some sorts of errors! Actually it's failed to stop Apache Spark Master or Slave using Systemd
Spark Master Stop Status
× spark-master.service - Apache Spark Master
Loaded: loaded (/etc/systemd/system/spark-master.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2022-09-26 18:43:39 +08; 8s ago
Docs: https://spark.apache.org/docs/3.3.0
Process: 488887 ExecStart=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/start-master.sh (code=exited, status=0/SUCCESS)
Process: 489000 ExecStartPost=/bin/bash -c echo $MAINPID > /opt/cli/spark-3.3.0-bin-hadoop3/etc/spark-master.pid (code=exited, status=0/SUCCESS)
Process: 489484 ExecStop=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/stop-master.sh (code=exited, status=0/SUCCESS)
Main PID: 488903 (code=exited, status=143)
CPU: 4.813s
Spark Slave Stop Status
× spark-slave.service - Apache Spark Slave
Loaded: loaded (/etc/systemd/system/spark-slave.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2022-09-26 18:38:22 +08; 15s ago
Docs: https://spark.apache.org/docs/3.3.0
Process: 489024 ExecStart=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/start-slave.sh spark://ns12-pc04:7077 (code=exited, status=0/SUCCESS)
Process: 489145 ExecStartPost=/bin/bash -c echo $MAINPID > /opt/cli/spark-3.3.0-bin-hadoop3/etc/spark-slave.pid (code=exited, status=0/SUCCESS)
Process: 489174 ExecStop=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/stop-slave.sh (code=exited, status=0/SUCCESS)
Main PID: 489040 (code=exited, status=143)
CPU: 4.306s
Expected Behavior
Your guide line would be appreciated to shutdown both Master & Slave node without any error.
Theoretical Solution
In this case you have to write your own script for manipulating the shutdown to force exit code 0 instead of 143. If you are idle enough like me then you can changeSuccessExitStatus from 0 to 143. By default systemd unit test looking forSuccessExitStatus code is 0. We need to change the default unit test behavior.
Practical Solution
/etc/systemd/system/spark-master.service
[Unit]
Description=Apache Spark Master
Wants=network-online.target
After=network-online.target
[Service]
User=spark
Group=spark
Type=forking
SuccessExitStatus=143
WorkingDirectory=/opt/cli/spark-3.3.0-bin-hadoop3/sbin
EnvironmentFile=/opt/cli/spark-3.3.0-bin-hadoop3/etc/env
ExecStartPost=/bin/bash -c "echo $MAINPID > /opt/cli/spark-3.3.0-bin-hadoop3/etc/spark-master.pid"
ExecStart=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/start-master.sh
ExecStop=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/stop-master.sh
[Install]
WantedBy=multi-user.target
/etc/systemd/system/spark-slave.service
[Unit]
Description=Apache Spark Slave
Wants=network-online.target
After=network-online.target
[Service]
User=spark
Group=spark
Type=forking
SuccessExitStatus=143
WorkingDirectory=/opt/cli/spark-3.3.0-bin-hadoop3/sbin
EnvironmentFile=/opt/cli/spark-3.3.0-bin-hadoop3/etc/env
ExecStartPost=/bin/bash -c "echo $MAINPID > /opt/cli/spark-3.3.0-bin-hadoop3/etc/spark-slave.pid"
ExecStart=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/start-slave.sh spark://spark.cdn.chorke.org:7077
ExecStop=/opt/cli/spark-3.3.0-bin-hadoop3/sbin/stop-slave.sh
[Install]
WantedBy=multi-user.target

status=203/EXEC Error when creating code server

I've created code server service on Ubuntu 18.04 nginx.
But gives error when run command :
systemctl status code-server
Error :
Warning: The unit file, source configuration file or drop-ins of code-server.service changed on disk. Run 'systemctl daemon-reload' to reload units.
● code-server.service - code-server
Loaded: loaded (/lib/systemd/system/code-server.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2022-09-02 17:45:29 CEST; 4min 53s ago
Process: 4749 ExecStart=/usr/bin/code-server --bind-addr 127.0.0.1:8080 --user-data-dir /var/lib/code-server --auth password (code=exited, status=203/EXEC)
Main PID: 4749 (code=exited, status=203/EXEC)
My /lib/systemd/system/code-server.service file as below
[Unit]
Description=code-server
After=nginx.service
[Service]
Type=simple
Environment=PASSWORD=xxxxxx
ExecStart=/usr/bin/code-server --bind-addr 127.0.0.1:8080 --user-data-dir /var/lib/code-server --auth password
Restart=always
[Install]
WantedBy=multi-user.target
I checked all topics, they say it's a missing path problem but I checked all paths and exists
root#xx:~# sudo mkdir /usr/lib/code-server
mkdir: cannot create directory ‘/usr/lib/code-server’: File exists
root#xx:~# sudo mkdir /var/lib/code-server
mkdir: cannot create directory ‘/var/lib/code-server’: File exists
root#xx:~#
I'm not a pro ubuntu user. What should I check ?

How do i install spark as a daemon

I start spark as master and slave on two machines with this guide:
https://www.tutorialkart.com/apache-spark/how-to-setup-an-apache-spark-cluster/
Then I make systemd .service for each of them but when I start them as a service they fail to start. Here is my systemctl status:
● sparkslave.service - Spark Slave
Loaded: loaded (/etc/systemd/system/sparkslave.service; enabled; ven
dor preset: enabled)
Active: inactive (dead) since Mon 2019-12-09 07:30:22 EST; 55s ago
Process: 31680 ExecStart=/usr/lib/spark/sbin/start-slave.sh spark://1
72.16.3.90:7077 (code=exited, status=0/SUCCESS)
Main PID: 31680 (code=exited, status=0/SUCCESS)
Dec 09 07:30:19 SparkSlave1 systemd[1]: Started Spark Slave.
Dec 09 07:30:19 SparkSlave1 start-slave.sh[31680]: starting org.apache.
spark.deploy.worker.Worker, logging to /usr/lib/spark/logs/spark-spark-
user-org.apache.spark.deploy.worker.Worker-1-SparkSlave1.out
And this is my sparkslave.service:
[Unit]
Description=Spark Slave
After=network.target
[Service]
User=spark-user
WorkingDirectory=/usr/lib/spark/sbin
ExecStart=/usr/lib/spark/sbin/start-slave.sh spark://172.16.3.90:7077
Restart=on-failure
RestartSec=10s
[Install]
WantedBy=multi-user.target
What is the problem?
Service type must change from simple to forking:
[Service]
Type=forking

Node.js, trying to add script as a service to centos gives error status=203/EXEC

I have a node.js server that I'd like to make a service so it wouldn't shut down after closing SSH-session.
I followed instructions in here, but it didn't go quite as planned because instead of the service starting up it gave me the following error:
Loaded: loaded (/etc/systemd/system/jasentiedot.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since ti 2018-01-23 10:38:08 EET; 59s ago
Process: 10525 ExecStart=/home/nodeuser/jasentiedot/server.js (code=exited, status=203/EXEC)
Main PID: 10525 (code=exited, status=203/EXEC)
Here is the service file at /etc/systemd/system/jasentiedot.service
[Unit]
Description=Servicename
[Service]
ExecStart=/home/nodeuser/jasentiedot/server.js
Restart=always
User=nobody
Group=nobody
Environment=PATH=/usr/bin:/usr/local/bin
Environment=NODE_ENV=production
WorkingDirectory=/home/nodeuser/jasentiedot
[Install]
WantedBy=multi-user.target
The script runs fine when running it manually, so there shouldn't be anything wrong with that.
Is there any better methods to make node.js script persistent in case this doesn't work out? I tried forever, but that didn't work.
You have to specify node xyz.js in Execstart. See this post: Node.js script failed to start with systemctl

Resources