systemd service works different then running the .sh file from user shell trying run a VNC startup script with noVNC - linux

Question

I want to run a .sh file with a systemd service. Let's explain it a little bit. If I go to cd /home/ubuntu I can run ./vnc_startup.sh. This file creates a VNC connection and starts noVNC. Then I go to the browser open the address and login. I can run every command like as example rosrun rviz rviz because I have installed ROS.
If I use this service, it will not work:
cat /etc/systemd/system/novnc.service
[Unit]
After=NetworkManager.service time-sync.target
[Service]
Type=forking
User=ubuntu
Group=ubuntu
WorkingDirectory=/home/ubuntu
TimeoutStartSec=infinity
TimeoutStopSec=infinity
ExecStartPre=/bin/rm -f /home/ubuntu/no_vnc_startup.log
ExecStartPre=/bin/rm -f /home/ubuntu/vnc_startup.log
ExecStartPre=/bin/rm -f /home/ubuntu/wm.log
ExecStartPre=/bin/rm -f /home/ubuntu/wm_startup.log
ExecStart=/bin/bash -c "source /etc/environment; /home/ubuntu/vnc_startup.sh"
ExecStopPost=/bin/rm -f /home/ubuntu/no_vnc_startup.log
ExecStopPost=/bin/rm -f /home/ubuntu/vnc_startup.log
ExecStopPost=/bin/rm -f /home/ubuntu/wm.log
ExecStopPost=/bin/rm -f /home/ubuntu/wm_startup.log
[Install]
WantedBy=multi-user.target
I also tried it with systemctl --user start novnc.service and put the file inside /usr/lib/systemd/user instead of sudo systemctl start novnc.service and /etc/systemd/system/novnc.service.
Following workaround will work in the noVNC environment: I can open a Terminal. I can see ubuntu#hostname:~$. So it seems to be the right user and I am in the right working directory. Before running as example rosrun rviz rviz I have to run sudo su ubuntu. And then it works. If I had run ./vnc_startup.sh instead of running this script with systemd it works directly without the workaround with sudo su ubuntu.
Hard to explain. I hope you can understand me.
systemctl --user show-environment
HOME=/home/ubuntu
LANG=de_DE
LOGNAME=ubuntu
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
SHELL=/bin/bash
USER=ubuntu
XDG_RUNTIME_DIR=/run/user/1001
The command printenv makes clear that maybe the DISPLAY=:0 is missing, so I tried it with adding
export DISPLAY=:0
xset q
And I also added Environment=XAUTHORITY=/home/ubuntu/.Xauthority:
[Unit]
After=NetworkManager.service time-sync.target
[Service]
Type=forking
User=ubuntu
Group=ubuntu
WorkingDirectory=/home/ubuntu
Environment=XAUTHORITY=/home/ubuntu/.Xauthority
TimeoutStartSec=infinity
TimeoutStopSec=infinity
ExecStartPre=/bin/rm -f /home/ubuntu/no_vnc_startup.log
ExecStartPre=/bin/rm -f /home/ubuntu/vnc_startup.log
ExecStartPre=/bin/rm -f /home/ubuntu/wm.log
ExecStartPre=/bin/rm -f /home/ubuntu/wm_startup.log
ExecStart=/bin/bash -c "source /etc/environment; export DISPLAY=:0; xset q; /home/ubuntu/vnc_startup.sh"
ExecStopPost=/bin/rm -f /home/ubuntu/no_vnc_startup.log
ExecStopPost=/bin/rm -f /home/ubuntu/vnc_startup.log
ExecStopPost=/bin/rm -f /home/ubuntu/wm.log
ExecStopPost=/bin/rm -f /home/ubuntu/wm_startup.log
[Install]
WantedBy=multi-user.target
Here my log files:
cat no_vnc_startup.log
New 'shlServer01:1 (ubuntu)' desktop is shlServer01:1
Starting applications specified in /home/ubuntu/.vnc/xstartup
Log file is /home/ubuntu/.vnc/shlServer01:1.log
r settings:
- Listen on :6901
- Flash security policy server
- Web server. Web root: /home/ubuntu/noVNC
- No SSL/TLS support (no cert file)
- proxying from :6901 to localhost:5901
Navigate to this URL:
http://shlServer01:6901/vnc.html?host=shlServer01&port=6901
Press Ctrl-C to exit
192.168.0.6 - - [15/Dec/2021 15:16:56] 192.168.0.6: Plain non-SSL (ws://) WebSocket connection
192.168.0.6 - - [15/Dec/2021 15:16:56] 192.168.0.6: Version hybi-13, base64: 'False'
192.168.0.6 - - [15/Dec/2021 15:16:56] 192.168.0.6: Path: '/websockify'
192.168.0.6 - - [15/Dec/2021 15:16:56] connecting to: localhost:5901
cat vnc_startup.log
Killing Xvnc process ID 63164
Xvnc process ID 63164 already killed
cat wm.log
/usr/bin/startxfce4: X server already running on display :0
xfce4-session: Cannot open display: .
▒xfce4-session --help▒ eingeben, um mehr ▒ber die Verwendung zu erfahren.
cat wm_startup.log
------------------ startup of Xfce4 window manager ------------------
No protocol specified
xset: unable to open display ":0"
No protocol specified
xset: unable to open display ":0"
No protocol specified
xset: unable to open display ":0"
No protocol specified
xrdb: Resource temporarily unavailable
xrdb: Can't open display ':0'
No protocol specified
No protocol specified
/usr/bin/startxfce4: X server already running on display :0
xfce4-session: Cannot open display: .
▒xfce4-session --help▒ eingeben, um mehr ▒ber die Verwendung zu erfahren.
What surprises me is that this is not the error. The error messages also come when I start the script from the terminal and then it works.
The vnc_startup.sh looks like following:
#!/bin/bash
### every exit != 0 fails the script
set -e
## print out help
help (){
echo "
OPTIONS:
-w, --wait (default) keeps the UI and the vncserver up until SIGINT or SIGTERM will received
-s, --skip skip the vnc startup and just execute the assigned command.
example: docker run consol/centos-xfce-vnc --skip bash
-d, --debug enables more detailed startup output
e.g. 'docker run consol/centos-xfce-vnc --debug bash'
-h, --help print out this help
Fore more information see: https://github.com/ConSol/docker-headless-vnc-container
"
}
if [[ $1 =~ -h|--help ]]; then
help
exit 0
fi
# should also source /home/ubuntu/generate_container_user
source /home/ubuntu/.bashrc
# add `--skip` to startup args, to skip the VNC startup procedure
if [[ $1 =~ -s|--skip ]]; then
echo -e "\n\n------------------ SKIP VNC STARTUP -----------------"
echo -e "\n\n------------------ EXECUTE COMMAND ------------------"
echo "Executing command: '${#:2}'"
exec "${#:2}"
fi
if [[ $1 =~ -d|--debug ]]; then
echo -e "\n\n------------------ DEBUG VNC STARTUP -----------------"
export DEBUG=true
fi
## correct forwarding of shutdown signal
cleanup () {
kill -s SIGTERM $!
exit 0
}
trap cleanup SIGINT SIGTERM
## write correct window size to chrome properties
/home/ubuntu/chrome-init.sh
## resolve_vnc_connection
VNC_IP=$(hostname -i)
## change vnc password
echo -e "\n------------------ change VNC password ------------------"
# first entry is control, second is view (if only one is valid for both)
mkdir -p "/home/ubuntu/.vnc"
PASSWD_PATH="/home/ubuntu/.vnc/passwd"
if [[ -f $PASSWD_PATH ]]; then
echo -e "\n--------- purging existing VNC password settings ---------"
rm -f $PASSWD_PATH
fi
if [[ $VNC_VIEW_ONLY == "true" ]]; then
echo "start VNC server in VIEW ONLY mode!"
#create random pw to prevent access
echo $(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 20) | vncpasswd -f > $PASSWD_PATH
fi
echo "ubuntu" | vncpasswd -f >> $PASSWD_PATH
chmod 600 $PASSWD_PATH
## start vncserver and noVNC webclient
echo -e "\n------------------ start noVNC ----------------------------"
if [[ $DEBUG == true ]]; then echo "/home/ubuntu/noVNC/utils/launch.sh --vnc localhost:5901 --listen 6901"; fi
/home/ubuntu/noVNC/utils/launch.sh --vnc localhost:5901 --listen 6901 &> /home/ubuntu/no_vnc_startup.log &
PID_SUB=$!
echo -e "\n------------------ start VNC server ------------------------"
echo "remove old vnc locks to be a reattachable container"
vncserver -kill :1 &> /home/ubuntu/vnc_startup.log \
|| rm -rfv /tmp/.X*-lock /tmp/.X11-unix &> /home/ubuntu/vnc_startup.log \
|| echo "no locks present"
echo -e "start vncserver with param: VNC_COL_DEPTH=24, VNC_RESOLUTION=1280x1024\n..."
if [[ $DEBUG == true ]]; then echo "vncserver :1 -depth 24 -geometry 1280x1024"; fi
vncserver :1 -depth 24 -geometry 1280x1024 &> /home/ubuntu/no_vnc_startup.log
echo -e "start window manager\n..."
/home/ubuntu/wm_startup.sh &> /home/ubuntu/wm_startup.log
## log connect options
echo -e "\n\n------------------ VNC environment started ------------------"
echo -e "\nVNCSERVER started on DISPLAY= :1 \n\t=> connect via VNC viewer with $VNC_IP:5901"
echo -e "\nnoVNC HTML client started:\n\t=> connect via http://$VNC_IP:6901/?password=...\n"
if [[ $DEBUG == true ]] || [[ $1 =~ -t|--tail-log ]]; then
echo -e "\n------------------ /home/ubuntu/.vnc/*:1.log ------------------"
# if option `-t` or `--tail-log` block the execution and tail the VNC log
tail -f /home/ubuntu/*.log /home/ubuntu/.vnc/*:1.log
fi
if [ -z "$1" ] || [[ $1 =~ -w|--wait ]]; then
wait $PID_SUB
else
# unknown option ==> call command
echo -e "\n\n------------------ EXECUTE COMMAND ------------------"
echo "Executing command: '$#'"
exec "$#"
fi
The wm_startup.sh looks like this:
#!/usr/bin/env bash
### every exit != 0 fails the script
set -e
echo -e "\n------------------ startup of Xfce4 window manager ------------------"
### disable screensaver and power management
xset -dpms &
xset s noblank &
xset s off &
/usr/bin/startxfce4 --replace > /home/ubuntu/wm.log &
sleep 1
cat /home/ubuntu/wm.log
And it should not be important but the launch.sh file fom noVNC looks like this:
#!/usr/bin/env bash
# Copyright 2016 Joel Martin
# Copyright 2016 Solly Ross
# Licensed under MPL 2.0 or any later version (see LICENSE.txt)
usage() {
if [ "$*" ]; then
echo "$*"
echo
fi
echo "Usage: ${NAME} [--listen PORT] [--vnc VNC_HOST:PORT] [--cert CERT] [--ssl-only]"
echo
echo "Starts the WebSockets proxy and a mini-webserver and "
echo "provides a cut-and-paste URL to go to."
echo
echo " --listen PORT Port for proxy/webserver to listen on"
echo " Default: 6080"
echo " --vnc VNC_HOST:PORT VNC server host:port proxy target"
echo " Default: localhost:5900"
echo " --cert CERT Path to combined cert/key file"
echo " Default: self.pem"
echo " --web WEB Path to web files (e.g. vnc.html)"
echo " Default: ./"
echo " --ssl-only Disable non-https connections."
echo " "
exit 2
}
NAME="$(basename $0)"
REAL_NAME="$(readlink -f $0)"
HERE="$(cd "$(dirname "$REAL_NAME")" && pwd)"
PORT="6080"
VNC_DEST="localhost:5900"
CERT=""
WEB=""
proxy_pid=""
SSLONLY=""
die() {
echo "$*"
exit 1
}
cleanup() {
trap - TERM QUIT INT EXIT
trap "true" CHLD # Ignore cleanup messages
echo
if [ -n "${proxy_pid}" ]; then
echo "Terminating WebSockets proxy (${proxy_pid})"
kill ${proxy_pid}
fi
}
# Process Arguments
# Arguments that only apply to chrooter itself
while [ "$*" ]; do
param=$1; shift; OPTARG=$1
case $param in
--listen) PORT="${OPTARG}"; shift ;;
--vnc) VNC_DEST="${OPTARG}"; shift ;;
--cert) CERT="${OPTARG}"; shift ;;
--web) WEB="${OPTARG}"; shift ;;
--ssl-only) SSLONLY="--ssl-only" ;;
-h|--help) usage ;;
-*) usage "Unknown chrooter option: ${param}" ;;
*) break ;;
esac
done
# Sanity checks
which netstat >/dev/null 2>&1 \
|| die "Must have netstat installed"
netstat -ltn | grep -qs ":${PORT} .*LISTEN" \
&& die "Port ${PORT} in use. Try --listen PORT"
trap "cleanup" TERM QUIT INT EXIT
# Find vnc.html
if [ -n "${WEB}" ]; then
if [ ! -e "${WEB}/vnc.html" ]; then
die "Could not find ${WEB}/vnc.html"
fi
elif [ -e "$(pwd)/vnc.html" ]; then
WEB=$(pwd)
elif [ -e "${HERE}/../vnc.html" ]; then
WEB=${HERE}/../
elif [ -e "${HERE}/vnc.html" ]; then
WEB=${HERE}
elif [ -e "${HERE}/../share/novnc/vnc.html" ]; then
WEB=${HERE}/../share/novnc/
else
die "Could not find vnc.html"
fi
# Find self.pem
if [ -n "${CERT}" ]; then
if [ ! -e "${CERT}" ]; then
die "Could not find ${CERT}"
fi
elif [ -e "$(pwd)/self.pem" ]; then
CERT="$(pwd)/self.pem"
elif [ -e "${HERE}/../self.pem" ]; then
CERT="${HERE}/../self.pem"
elif [ -e "${HERE}/self.pem" ]; then
CERT="${HERE}/self.pem"
else
echo "Warning: could not find self.pem"
fi
# try to find websockify (prefer local, try global, then download local)
if [[ -e ${HERE}/websockify ]]; then
WEBSOCKIFY=${HERE}/websockify/run
if [[ ! -x $WEBSOCKIFY ]]; then
echo "The path ${HERE}/websockify exists, but $WEBSOCKIFY either does not exist or is not executable."
echo "If you intended to use an installed websockify package, please remove ${HERE}/websockify."
exit 1
fi
echo "Using local websockify at $WEBSOCKIFY"
else
WEBSOCKIFY=$(which websockify 2>/dev/null)
if [[ $? -ne 0 ]]; then
echo "No installed websockify, attempting to clone websockify..."
WEBSOCKIFY=${HERE}/websockify/run
git clone https://github.com/novnc/websockify ${HERE}/websockify
if [[ ! -e $WEBSOCKIFY ]]; then
echo "Unable to locate ${HERE}/websockify/run after downloading"
exit 1
fi
echo "Using local websockify at $WEBSOCKIFY"
else
echo "Using installed websockify at $WEBSOCKIFY"
fi
fi
echo "Starting webserver and WebSockets proxy on port ${PORT}"
#${HERE}/websockify --web ${WEB} ${CERT:+--cert ${CERT}} ${PORT} ${VNC_DEST} &
${WEBSOCKIFY} ${SSLONLY} --web ${WEB} ${CERT:+--cert ${CERT}} ${PORT} ${VNC_DEST} &
proxy_pid="$!"
sleep 1
if ! ps -p ${proxy_pid} >/dev/null; then
proxy_pid=
echo "Failed to start WebSockets proxy"
exit 1
fi
echo -e "\n\nNavigate to this URL:\n"
if [ "x$SSLONLY" == "x" ]; then
echo -e " http://$(hostname):${PORT}/vnc.html?host=$(hostname)&port=${PORT}\n"
else
echo -e " https://$(hostname):${PORT}/vnc.html?host=$(hostname)&port=${PORT}\n"
fi
echo -e "Press Ctrl-C to exit\n\n"
wait ${proxy_pid}

Answer 1

Suggesting to simply your /etc/systemd/system/novnc.service service unit with a single script for ExecStart command and a single script for ExecStop
/etc/systemd/system/novnc.service
[Unit]
After=NetworkManager.service time-sync.target
[Service]
Type=forking
User=ubuntu
Group=ubuntu
WorkingDirectory=/home/ubuntu
Environment=XAUTHORITY=/home/ubuntu/.Xauthority
TimeoutStartSec=infinity
TimeoutStopSec=infinity
ExecStart=/bin/bash -c "/home/ubuntu/servic_vnc_startup.sh"
ExecStop=/bin/bash -c "/home/ubuntu/servic_vnc_shutdown.sh"
[Install]
WantedBy=multi-user.target
/home/ubuntu/servic_vnc_startup.sh
#!\bin\bash
source /home/ubuntu/.bash_profile
source /etc/environment
export DISPLAY=:0
xset q;
rm -f /home/ubuntu/{no_vnc_startup.log,vnc_startup.log,wm.log,wm_startup.log}
/home/ubuntu/vnc_startup.sh
/home/ubuntu/servic_vnc_shutdown.sh
#!\bin\bash
source /home/ubuntu/.bash_profile
source /etc/environment
export DISPLAY=:0
xset q;
pkill -9 -f "/home/ubuntu/vnc_startup.sh"
rm -f /home/ubuntu/{no_vnc_startup.log,vnc_startup.log,wm.log,wm_startup.log}
Debugging
Login as user ubuntu.
Run /home/ubuntu/servic_vnc_startup.sh from command line.
If fails, fix it till it is successful.
Then try running /home/ubuntu/servic_vnc_startup.sh as user noboby:
sudo -u nobody "/home/ubuntu/servic_vnc_startup.sh"
User nobody has no shell and no environment context, as is the systemd service /etc/systemd/system/novnc.service.
If user nobody can run /home/ubuntu/servic_vnc_startup.sh then the /etc/systemd/system/novnc.service service unit can do as well.
Do same testing pattern with /home/ubuntu/servic_vnc_shutdown.sh
Lessons to learn:
Simplify service unit as much as possible.
Pull all scripting to a single shell script. Debug single shell script.
Avoid debugging handling service unit once deployed. Instead debug and modify called script.
Use user nobody user, to debug script to run without environment context and without shell.