rsync daemon behaving eratically - linux

I'm running an rsync daemon (providing a mirror for the SaneSecurity signatures).
rsync is started like this (from runit):
/usr/bin/rsync -v --daemon --no-detach
And the config contains:
use chroot = no
munge symlinks = no
max connections = 200
timeout = 30
syslog facility = local5
transfer logging = no
log file = /var/log/rsync.log
reverse lookup = no
[sanesecurity]
comment = SaneSecurity ClamAV Mirror
path = /srv/mirror/sanesecurity
read only = yes
list = no
uid = nobody
gid = nogroup
But what I'm seeing is a lot of "lingering" rsync processes:
# ps auxwww|grep rsync
root 423 0.0 0.0 4244 1140 ? Ss Oct30 0:00 runsv rsync
root 2529 0.0 0.0 11156 2196 ? S 15:00 0:00 /usr/bin/rsync -v --daemon --no-detach
nobody 4788 0.0 0.0 20536 2860 ? S 15:10 0:00 /usr/bin/rsync -v --daemon --no-detach
nobody 5094 0.0 0.0 19604 2448 ? S 15:13 0:00 /usr/bin/rsync -v --daemon --no-detach
root 5304 0.0 0.0 11156 180 ? S 15:15 0:00 /usr/bin/rsync -v --daemon --no-detach
root 5435 0.0 0.0 11156 180 ? S 15:16 0:00 /usr/bin/rsync -v --daemon --no-detach
root 5797 0.0 0.0 11156 180 ? S 15:19 0:00 /usr/bin/rsync -v --daemon --no-detach
nobody 5913 0.0 0.0 20536 2860 ? S 15:20 0:00 /usr/bin/rsync -v --daemon --no-detach
nobody 6032 0.0 0.0 20536 2860 ? S 15:21 0:00 /usr/bin/rsync -v --daemon --no-detach
root 6207 0.0 0.0 11156 180 ? S 15:22 0:00 /usr/bin/rsync -v --daemon --no-detach
nobody 6292 0.0 0.0 20544 2744 ? S 15:23 0:00 /usr/bin/rsync -v --daemon --no-detach
root 6467 0.0 0.0 11156 180 ? S 15:25 0:00 /usr/bin/rsync -v --daemon --no-detach
root 6905 0.0 0.0 11156 180 ? S 15:29 0:00 /usr/bin/rsync -v --daemon --no-detach
(it's currently 15:30)
So there's processes (not even having dropped privileges!) hanging around since 15:10, 15:13 and the like.
And what are they doing?
Let's check:
# strace -p 5304
strace: Process 5304 attached
select(4, [3], NULL, [3], {25, 19185}^C
strace: Process 5304 detached
<detached ...>
# strace -p 5797
strace: Process 5797 attached
select(4, [3], NULL, [3], {48, 634487}^C
strace: Process 5797 detached
<detached ...>
This happended with both rsync from Ubuntu Xenial as well as installed from PPA (currently using rsync 3.1.2-1~ubuntu16.04.1york0 )

One process is created for each connection. Before a client selects the module the process does not know if it should drop privileges.
You can easily create such a process.
nc $host 873
You will notice that the connection will not be closed after 30s because the timeout is just a disk i/o timeout. The rsync client have a --contimeout option, but it seems that a server side option is missing.

In the end, I resorted to invoking rsync from (x)inetd instead of running it standalone.
service rsync
{
disable = no
socket_type = stream
wait = no
user = root
server = /usr/bin/timeout
server_args = -k 60s 60s /usr/bin/rsync --daemon
log_on_failure += USERID
flags = IPv6
}
As an additional twist, I wrapped the rsync invocation with timeout, adding another safeguard against long-running processes.

Related

How do I find the name of a process in linux?

I'm really struggling with how to find processes by name in linux. I'm sure it's probably something simple that I'm missing.
Are you looking for the command ps ?
Here an example
nabil#LAPTOP:~$ ps xua | grep python
rootwsl 327 0.0 0.1 29568 17880 ? Ss Jan30 0:02 /usr/bin/python3 /usr/bin/networkd-dispatcher --run-startup-triggers
rootwsl 411 0.0 0.1 108116 20740 ? Ssl Jan30 0:00 /usr/bin/python3 /usr/share/unattended-upgrades/unattended-upgrade-shutdown --wait-for-signal
nabil 106387 0.0 0.0 3444 736 pts/1 S+ 23:26 0:00 grep --color=auto python

Killing subprocess from inside a Docker container kills the entire container

On my Windows machine, I started a Docker container from docker compose. My entrypoint is a Go filewatcher that runs a task of a taskmanager on every filechange. The executed task builds and runs the Go program.
But before I can build and run the program again after filechanges I have to kill the previous running version. But every time I kill the app process, the container is also gone.
The goal is to kill only the svc1 process with PID 74 in this example. I tried pkill -9 svc1 and kill $(pgrep svc1). But every time the parent processes are killed too.
The commandline output from inside the container:
root#bf073c39e6a2:/app/cmd/svc1# ps -aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 2.5 0.0 104812 2940 ? Ssl 13:38 0:00 /go/bin/watcher
root 13 0.0 0.0 294316 7576 ? Sl 13:38 0:00 /go/bin/task de
root 74 0.0 0.0 219284 4908 ? Sl 13:38 0:00 /svc1
root 82 0.2 0.0 18184 3160 pts/0 Ss 13:38 0:00 /bin/bash
root 87 0.0 0.0 36632 2824 pts/0 R+ 13:38 0:00 ps -aux
root#bf073c39e6a2:/app/cmd/svc1# ps -afx
PID TTY STAT TIME COMMAND
82 pts/0 Ss 0:00 /bin/bash
88 pts/0 R+ 0:00 \_ ps -afx
1 ? Ssl 0:01 /go/bin/watcher -cmd /go/bin/task dev -startcmd
13 ? Sl 0:00 /go/bin/task dev
74 ? Sl 0:00 \_ /svc1
root#bf073c39e6a2:/app/cmd/svc1# pkill -9 svc1
root#bf073c39e6a2:/app/cmd/svc1
Switching to the containerlog:
task: Failed to run task "dev": exit status 255
2019/08/16 14:20:21 exit status 1
"dev" is the name of the task in the taskmanger.
The Dockerfile:
FROM golang:stretch
RUN go get -u -v github.com/radovskyb/watcher/... \
&& go get -u -v github.com/go-task/task/cmd/task
WORKDIR /app
COPY ./Taskfile.yml ./Taskfile.yml
ENTRYPOINT ["/go/bin/watcher", "-cmd", "/go/bin/task dev", "-startcmd"]
I expect only the process with the target PID is killed and not the parent process that spawned it it.
You can use process manager like "supervisord" and configure it to re-execute your script or the command even if you killed it's process which will keep your container up and running.

Gnome-terminal doesn't start any more after auto login to X

Tell me please is there any way to resolve this issue without installing a login manager please?
I've enabled the auto login for startx, following the steps from the beyond link:
How to make auto login work in Ubuntu? (no display manager)
Auto login is functioning now.
On the device is a command line installation from Minimal Lubuntu 16.10 mini.iso. without any desktop, only the Kernel and some restricted modules. The only environment installed is fluxbox.
After booting in Fluxbox, I can't open gnome-terminal at all, until I will not do the next steps. xterm can start.
ctrl+alt+del in the running Fluxbox, it will redirect me for a second-two in tty, but because auto login is enable it will redirect me back automatically from tty1 to Fluxbox. So, in order to remain in tty I will keep pressing continuously ctrl+c.
Now, being in tty I will
sudo -i
su myusername
startx
Being again in fluxbox, I can run the terminal normally.
Do you please have any clues please, why I can't open the terminal without doing the above?
Trying to start gnome-terminal from xterm when at the first login.
gnome-terminal
Error constructing proxy for org.gnome.Terminal:/org/gnome/Terminal/Factory0: Error calling StartServiceByName for org.gnome.Terminal: Timeout was reached
Excuse me, I am not sure that DBUS_SESSION_BUS_ADDRESS is active or not.
/var/log/Xorg.0.log
Output of env from xterm (after logging manually again to startx)
TERM=xterm
SHELL=/bin/bash
WINDOWID=8388621
XTERM_SHELL=/bin/bash
USER=root
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
SUDO_USER=xdpsx
SUDO_UID=1000
USERNAME=root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
MAIL=/var/mail/root
PWD=/home/xdpsx
LANG=en_US.UTF-8
XTERM_LOCALE=en_US.UTF-8
XTERM_VERSION=XTerm(324)
HOME=/root
SUDO_COMMAND=/bin/su
SHLVL=2
LOGNAME=root
LESSOPEN=| /usr/bin/lesspipe %s
DISPLAY=:0.0
SUDO_GID=1000
LESSCLOSE=/usr/bin/lesspipe %s %s
XAUTHORITY=/home/xdpsx/.Xauthority
COLORTERM=truecolor
_=/usr/bin/env
Output of ps aux | grep dbus from xterm (after logging manually again to startx)
message+ 668 0.0 0.0 6420 3936 ? Ss 19:11 0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
nobody 811 0.0 0.1 9316 4000 ? S 19:11 0:00 /usr/sbin/dnsmasq --no-resolv --keep-in-foreground --no-hosts --bind-interfaces --pid-file=/var/run/NetworkManager/dnsmasq.pid --listen-address=127.0.1.1 --cache-size=0 --conf-file=/dev/null --proxy-dnssec --enable-dbus=org.freedesktop.NetworkManager.dnsmasq --conf-dir=/etc/NetworkManager/dnsmasq.d
xdpsx 1375 0.0 0.0 6136 3460 ? Ss 19:11 0:00 /usr/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation
xdpsx 1381 0.0 0.0 6136 3316 ? S 19:11 0:00 /usr/bin/dbus-daemon --config-file=/usr/share/defaults/at-spi2/accessibility.conf --nofork --print-address 3
xdpsx 1505 0.0 0.0 7004 312 ? S 19:12 0:00 dbus-launch --autolaunch fedd8908d0d244c498876a97f5b34c28 --binary-syntax --close-stderr
xdpsx 1506 0.0 0.0 6136 3060 ? Ss 19:12 0:00 /usr/bin/dbus-daemon --fork --print-pid 5 --print-address 7 --session
xdpsx 1529 0.0 0.0 6136 3356 ? S 19:14 0:00 /usr/bin/dbus-daemon --config-file=/usr/share/defaults/at-spi2/accessibility.conf --nofork --print-address 3
root 1834 0.0 0.0 5144 828 pts/1 S+ 19:25 0:00 grep --color=auto dbus
Thank you.
I am seeing a similar problem since upgrading to Ubuntu 16.10. I discovered that I can fix the environment and start gnome-terminal like this:
dbus-update-activation-environment --systemd --all
gnome-terminal &
If that doesn't work, then you could try this:
dbus-launch gnome-terminal &

How does 'kill -STOP and kill -CONT' work?

I'm facing an issue.
We have a clean script using to clean old files, and sometimes we need stop it for and will start it again later. Like the below processes. We use kill -STOP $pid and kill -CONT $pid in check.sh to control the clean.sh, $pid is all the pids of clean.sh (at there, they are 23939, 25804):
root 4321 0.0 0.0 74876 1184 ? Ss 2015 0:25 crond
root 23547 0.0 0.0 102084 1604 ? S 2015 0:00 \_ crond
root 23571 0.0 0.0 8728 972 ? Ss 2015 0:00 \_ /bin/bash -c bash /home/test/sbin/check.sh >>/home/test/log/check.log 2>&1
root 23577 0.0 0.0 8732 1092 ? S 2015 0:00 \_ bash /home/test/sbin/check.sh
root 23939 0.0 0.0 8860 1192 ? S 2015 0:45 \_ bash /home/test/bin/clean.sh 30
root 25804 0.0 0.0 8860 620 ? S 2015 0:00 \_ bash /home/test/bin/clean.sh 30
root 25805 0.0 0.0 14432 284 ? T 2015 0:00 \_ ls -d ./455bb4cba6142427156d2b959b8b0986/120x60/ ./455bb4cba6142427156d2b959b8b0986/80x
root 25808 0.0 0.0 3816 432 ? S 2015 0:00 \_ wc -l
Once the check.sh stopped clean.sh, hours later, check.sh started clean.sh, but there is a strange thing, after a stop and continue, there is a child process 'ls -d ....', it's still stopping.
Could you tell me if it's caused by wrong use of the signal? And how can I modify it?
ok, same like my description is not clear, my bad English...
Not sure what's the reason, but there is a way to sovle it:
kill -CONT $pid
pkill -CONT -P $pid
This will continue the child process.

How to see a terminal output from a previously closed terminal

I connect to a remote server using SSH
I was compiling using cmake and then make, it's not common to have a progress percentage in compilation process, but this time it has. I was watching the compilation process until my internet connection failed, so puTTY closed the session and I had to connect again to my server. I though that all the progress was lost, but i first make sure by watching the processes list by ps aux command, and I noticed that the processes related to the compilation are still running:
1160 tty1 Ss+ 0:00 /sbin/mingetty tty1
2265 ? Ss 0:00 sshd: root#pts/1
2269 pts/1 Ss 0:00 -bash
2353 pts/1 S+ 0:00 make
2356 pts/1 S+ 0:00 make -f CMakeFiles/Makefile2 all
2952 ? S 0:00 pickup -l -t fifo -u
3085 ? Ss 0:00 sshd: root#pts/0
3089 pts/0 Ss 0:00 -bash
3500 pts/1 S+ 0:01 make -f src/compiler/CMakeFiles/hphp_analysis.dir/bui
3509 pts/1 S+ 0:00 /bin/sh -c cd /root/hiphop/hiphop-php/src/compiler &&
3510 pts/1 S+ 0:00 /usr/bin/g++44 -DNO_JEMALLOC=1 -DNO_TCMALLOC=1 -D_GNU
3511 pts/1 R+ 0:03 /usr/libexec/gcc/x86_64-redhat-linux6E/4.4.4/cc1plus
3512 pts/0 R+ 0:00 ps ax
I would like to know if is possible to watch the current progress of the compilation by watching the previously closed terminal output. Something similar like 'cat /dev/vcsa1' or something
As per the comment above, you should have used screen.
As it is, you could try to peek at the file descriptors used by sshd and the shell that you started, but I don't think this will get you very far.

Resources