BASH : How to make a script that make "tail -f" always logging the last file in a directory, live - linux

I'm basically trying to make debugging easier for other scripts of mine.
(Centos 7.6)
What I need is a script doing :
tail -f the last file entry in a directory
if a new file appears in this directory, it logs this new file, smoothly
if I send a SIGINT (Ctrl+C), it doesn't leave orphans
with the less possible add-ons for the maximum portability
This is my non working solution :
CURRENT_FILE=`ls -1t | head -n1`
tail -n 100 -f "$CURRENT_FILE" &
PID=$!
while [ true ];
do
#is there a new file in the directory ?
NEW_FILE=`ls -1t | head -n1`
if [[ "$CURRENT_FILE" != "$NEW_FILE" ]]; then
#yes, so kill last tail
kill -9 $PID
clear
#tail on the new file
CURRENT_FILE=$NEW_FILE
tail -n 100 -f "$CURRENT_FILE"
PID=$!
fi
sleep 1s
done
The problem with this solution is that when I'm sending SIGINT (Ctrl+C), what I normally do when exiting a "tail -f", it leaves an orphan child in the background.
I've searched solution with "trap" but I don't get it well, and it doesn't seem to work with an eternal process like "tail -f".
I'll be glad to here your thoughts about that and get into advanced bash programming.

You can trap whenever the script exits and kill the process then. You don't need -9 to kill your tail though, that's overkill.
You can also use inotify to tell you when something happens in the directory instead of sleeping and rechecking. Here's a basic building block. inotify has a lot of events you can wait for. You can add detection if the file was moved/renamed so you don't have to restart the tail in those cases etc.
#!/bin/bash
killpid() {
if [[ -n $PID ]]; then
kill $PID
PID=""
fi
}
trap killpid EXIT
DIR="."
CURRENT_FILE="$(ls -1t "$DIR" | head -n1)"
tailit() {
echo "::: $CURRENT_FILE :::"
tail -n 100 -f "$CURRENT_FILE" &
PID=$!
}
tailit
# wait for any file to be created, modified or deleted
while EVENT=$(inotifywait -q -e create,modify,delete "$DIR"); do
# extract event
ev=$(sed -E "s/^${DIR}\/ (\S+) .+$/\1/" <<< "$EVENT")
# extract the affected file
NEW_FILE=${EVENT#${DIR}/ $ev }
case $ev in
MODIFY)
# start tailing the file if we aren't tailing it already
if [[ $NEW_FILE != $CURRENT_FILE ]]; then
killpid
CURRENT_FILE="$NEW_FILE"
tailit
fi
;;
CREATE)
# a new file, tail it
killpid
CURRENT_FILE="$NEW_FILE"
tailit
;;
DELETE)
# stop tailing if the file we are tailing was deleted
if [[ $NEW_FILE == $CURRENT_FILE ]]; then
echo "::: $CURRENT_FILE removed :::"
CURRENT_FILE=""
killpid
fi
;;
esac
done

You can use trap solution at the beginning of your shell.
#! /bin/bash
trap ctrl_c INT
function ctrl_c() {
if [[ -n "$PID" ]]; then
kill -9 $PID
fi
exit 0
}
CURRENT_FILE=`ls -1t | head -n1`
tail -n 100 -f "$CURRENT_FILE" &
PID=$!
while [ true ];
do
#is there a new file in the directory ?
NEW_FILE=`ls -1t | head -n1`
if [[ "$CURRENT_FILE" != "$NEW_FILE" ]]; then
#yes, so kill last tail
kill -9 $PID
clear
#tail on the new file
CURRENT_FILE=$NEW_FILE
tail -n 100 -f "$CURRENT_FILE" &
PID=$!
fi
sleep 1s
done

Related

Binary operator expected error while running a while loop in bash

TL:DR
Check if a given PID is running, if yes kill the process.
count=0
while [[ "$count" -le 3 && ps -p $pid > /dev/null ]];
do
kill -9 $pid
count=$(( $count + 1 )):
done
To this I am getting an error as:
line 8: [: -p: binary operator expected
I am aware there are several similar questions, I already tried their solutions but it doesn't seem to work.
The while loop is logically incorrect, as #kvantour mentioned. Here is the script. Note that it will let you know if it could not kill the process, so you can investigate the root cause. The script gets PID as its first argument (e.g. $./kill-pid.sh 1234) Note that this works for bash ver. 4.1+:
#!/usr/bin/env bash
if ps -p $1 > /dev/null
then
output=$(kill -9 $1 2>&1)
if [ $? -ne 0 ]
then
echo "Process $1 cannot be killed. Reason:"
echo "$output"
# This line is added per OP request, to try to re-run the kill command if it failed for the first time.
# kill -9 $1
fi
fi

Getting PID of launched process

For some monitoring purposes, for a given list of files (that change path daily) I want to launch a tail -f in the background searching for specific strings
How do I properly get the exact pid of the tail commands generated by the for loop, so I could kill them later ?
Here's what I'm trying to do which I think gives me the pid of the script and not the tail process.
for f in $(find file...)
do
tail -f $f | while read line
do case $line in
*string_to_search*) echo " $line" | mutt -s "string detected in file : $1" mail#mail.com;
;;
esac
done &
echo $! >> pids.txt
done

tail the last modified file and monitor for new files in bash

I can tail -f the last modified file using ls --sort=time | head -1 | xargs tail -f but I'm looking for something that would continuously run the first part as well, i.e. if a new files is created while the tail command is running it switches to tailing the new file. I thought about using watch command but that doesn't seem to work nicely with tail -f
It seems you need something like this:
#!/bin/bash
TAILPID=0
WATCHFILE=""
trap 'kill $(jobs -p)' EXIT # Makes sure we clean our mess (background processes) on exit
while true
do
NEWFILE=`ls --sort=time | head -n 1`
if [ "$NEWFILE" != "$WATCHFILE" ]; then
echo "New file has been modified"
echo "Now watching: $NEWFILE";
WATCHFILE=$NEWFILE
if [ $TAILPID -ne 0 ]; then
# Kill old tail
kill $TAILPID
wait $! &> /dev/null # supress "Terminated" message
fi
tail -f $NEWFILE &
TAILPID=$! # Storing tail PID so we could kill it later
fi
sleep 1
done

bash lsof : get pid from one tty to another one

How to get the pid in tty1 of the process launched in tty2 ?
Context :
Trying to write a bash one-liner to kill a process generating a file when this file exceeds a pre defined max size. (The one-liner is not operating yet as it is as need to embed this into a loop).
During testing, the point is that lsof does not return any PID in the terminal tty1 despite the pid exists in the tty2 where the command is run.
tty1: generating the file and monitoring changes
MAX_SIZE_Ko=10001;file=test_lsof;dd if=/dev/zero of=$file bs=1k count=800;inotifywait $file;SIZE_Ko=$(du -s $file | cut -f1); [[ "$SIZE_Ko" -gt "$MAX_SIZE" ]] && ( PID=$(lsof $file | tail -n1 | awk -F" " '{ print $2 }') ; [[ ! -z $PID ]] && kill -9 $PID || echo "no running PID modifying $file" )
tty2 : increasing the file size
for (( 1; 1; 1));do echo -e "foobar\n" >> test_lsof; echo $(( i++ ))" - pid="$$; done
As mentioned in the other answer, the file is opened only for a short time, so the odds of your lsof catching it are low.
However, you can change that:
exec 5>test_lsof
for (( 1; 1; 1)); do
echo -e "foobar\n" >&5
echo $(( i++ ))" - pid="$$
done
This uses advanced shell redirection - the exec line opens a file descriptor, the >&5 redirects output from the command to that file descriptor.
If you do that, the shell will be visible to lsof.
The problem is that the process in tty2 opens the file only for a split second to append the string. Unless you run lsof in the same split second, you won't catch it.
One way to deal with this is to use inotify-tools. The program inotifywait allows you to wait until the file is opened and the run lsof, e.g. inotifywait $file; lsof $file.

need a restart server script in 1 hour if not stopped

I am working on a remote servers network setup.
What I need is a script that will rename the "/etc/network/interfaces" file and then restart the computer. The renaming I got but what I don't get is how i can terminate this script in case I don't need it.
See if everything works out fine I like to issue a stop command that will terminate this script, so that the server doesn't restart.
So here is what I got so far. the issues are:
It doesn't return the prompt
The stop command doesn't work. It doesn't get the pid file for some reason. It returns "rm: missing operand" although the echo tells me that the pid file is called "start.pid" and it is present in the /tmp folder
Any ideas?
#! /bin/sh
PATH=/sbin:/usr/sbin:/bin:/usr/bin
. /lib/lsb/init-functions
case "$1" in
start)
;;
export PIDFILE=/var/run/${1}.pid
ps -fe | grep ${1} | head -n1 | cut -d" " -f 6 > ${PIDFILE}
sleep 30 #3600
log_action_msg "WARNING: Will in 60 sec rename /etc/network/interfaces and then restart"
sleep 30# 60
SUFFIX=$(date +%s)
#cp /etc/network/interfaces /etc/network/interfaces.$SUFFIX
cp /tmp/interfaces /etc/network/interfaces.$SUFFIX
sleep 1
#cp /etc/network/interfaces.org /tmp/interfaces
cp /tmp/interfaces.org /tmp/interfaces
sleep 1
#reboot -d -f -i
;;
stop)
if [ -f ${PIDFILE} ]; then
rm ${PIDFILE}
fi
exit 0
;;
*)
echo "Usage: $0 start|stop" >&2
exit 3
;;
esac
Usually this is done using a 'pid-file' - a predetermined file that holds the process identifier of the currently running process. That way if it is called and told to stop, it looks up the pid-file and uses the kill command to send a signal to the currently running process.
There is another benefit of this as well - if you check for the existence of a pid-file (and the existence of that process) when the script is told to start, you can prevent accidentally starting the script twice, which would make stopping both instances problematic.
The stop action can create a file do.not.restart.server in an appropriate location.
The start action can be modified to check whether the do.not.restart.server file exists, and avoid restarting the server if it is. It can/should probably remove the file for future restarts - or maybe it should remove it before it goes to sleep.
Okay, here is a working script, it does what I need. The only improvement I could still wish for is how to return the prompt from the sleep command.
The functionality is there so I am posting it in case others needed as well.
Thanks Dan and Jonathan Leffler for your help and ideas.
#! /bin/sh
PATH=/sbin:/usr/sbin:/bin:/usr/bin
. /lib/lsb/init-functions
export PIDFILESTART=/tmp/network-safty-restart-start.pid
export PIDFILESTOP=/tmp/network-safty-restart-stop.pid
#export FILE=/etc/network/interfaces
export FILE=/tmp/interfaces
case "$1" in
start)
if [ -f ${PIDFILESTART} ]; then
rm ${PIDFILESTART}
fi
if [ -f ${PIDFILESTOP} ]; then
rm ${PIDFILESTOP}
fi
ps -fe | grep ${1} | head -n1 | cut -d" " -f 6 > ${PIDFILESTART}
sleep 3600
log_action_msg "WARNING: Will in 60 sec rename ${FILE} and then restart"
sleep 60
if ! [ -f ${PIDFILESTOP} ]; then
log_action_msg "Restarting NOW"
SUFFIX=$(date +%s)
cp ${FILE} ${FILE}.${SUFFIX}
sleep 1
cp ${FILE}.org ${FILE}
sleep 1
reboot -d -f -i
else
rm ${PIDFILESTOP}
log_action_msg "NOT Restaring as you wish"
fi
;;
stop)
if [ -f ${PIDFILESTART} ]; then
rm ${PIDFILESTART}
ps -fe | grep ${1} | head -n1 | cut -d" " -f 6 > ${PIDFILESTOP}
log_action_msg "Terminating restart script"
fi
log_action_msg "Terminated restart script"
exit 0
;;
*)
echo "Usage: $0 start|stop" >&2
exit 3
;;
esac

Resources