For some monitoring purposes, for a given list of files (that change path daily) I want to launch a tail -f in the background searching for specific strings
How do I properly get the exact pid of the tail commands generated by the for loop, so I could kill them later ?
Here's what I'm trying to do which I think gives me the pid of the script and not the tail process.
for f in $(find file...)
do
tail -f $f | while read line
do case $line in
*string_to_search*) echo " $line" | mutt -s "string detected in file : $1" mail#mail.com;
;;
esac
done &
echo $! >> pids.txt
done
Related
I'm basically trying to make debugging easier for other scripts of mine.
(Centos 7.6)
What I need is a script doing :
tail -f the last file entry in a directory
if a new file appears in this directory, it logs this new file, smoothly
if I send a SIGINT (Ctrl+C), it doesn't leave orphans
with the less possible add-ons for the maximum portability
This is my non working solution :
CURRENT_FILE=`ls -1t | head -n1`
tail -n 100 -f "$CURRENT_FILE" &
PID=$!
while [ true ];
do
#is there a new file in the directory ?
NEW_FILE=`ls -1t | head -n1`
if [[ "$CURRENT_FILE" != "$NEW_FILE" ]]; then
#yes, so kill last tail
kill -9 $PID
clear
#tail on the new file
CURRENT_FILE=$NEW_FILE
tail -n 100 -f "$CURRENT_FILE"
PID=$!
fi
sleep 1s
done
The problem with this solution is that when I'm sending SIGINT (Ctrl+C), what I normally do when exiting a "tail -f", it leaves an orphan child in the background.
I've searched solution with "trap" but I don't get it well, and it doesn't seem to work with an eternal process like "tail -f".
I'll be glad to here your thoughts about that and get into advanced bash programming.
You can trap whenever the script exits and kill the process then. You don't need -9 to kill your tail though, that's overkill.
You can also use inotify to tell you when something happens in the directory instead of sleeping and rechecking. Here's a basic building block. inotify has a lot of events you can wait for. You can add detection if the file was moved/renamed so you don't have to restart the tail in those cases etc.
#!/bin/bash
killpid() {
if [[ -n $PID ]]; then
kill $PID
PID=""
fi
}
trap killpid EXIT
DIR="."
CURRENT_FILE="$(ls -1t "$DIR" | head -n1)"
tailit() {
echo "::: $CURRENT_FILE :::"
tail -n 100 -f "$CURRENT_FILE" &
PID=$!
}
tailit
# wait for any file to be created, modified or deleted
while EVENT=$(inotifywait -q -e create,modify,delete "$DIR"); do
# extract event
ev=$(sed -E "s/^${DIR}\/ (\S+) .+$/\1/" <<< "$EVENT")
# extract the affected file
NEW_FILE=${EVENT#${DIR}/ $ev }
case $ev in
MODIFY)
# start tailing the file if we aren't tailing it already
if [[ $NEW_FILE != $CURRENT_FILE ]]; then
killpid
CURRENT_FILE="$NEW_FILE"
tailit
fi
;;
CREATE)
# a new file, tail it
killpid
CURRENT_FILE="$NEW_FILE"
tailit
;;
DELETE)
# stop tailing if the file we are tailing was deleted
if [[ $NEW_FILE == $CURRENT_FILE ]]; then
echo "::: $CURRENT_FILE removed :::"
CURRENT_FILE=""
killpid
fi
;;
esac
done
You can use trap solution at the beginning of your shell.
#! /bin/bash
trap ctrl_c INT
function ctrl_c() {
if [[ -n "$PID" ]]; then
kill -9 $PID
fi
exit 0
}
CURRENT_FILE=`ls -1t | head -n1`
tail -n 100 -f "$CURRENT_FILE" &
PID=$!
while [ true ];
do
#is there a new file in the directory ?
NEW_FILE=`ls -1t | head -n1`
if [[ "$CURRENT_FILE" != "$NEW_FILE" ]]; then
#yes, so kill last tail
kill -9 $PID
clear
#tail on the new file
CURRENT_FILE=$NEW_FILE
tail -n 100 -f "$CURRENT_FILE" &
PID=$!
fi
sleep 1s
done
I am trying to write a small shell script, make it go to sleep for some amount of time like 20 seconds and then run it. Now if i open another terminal and try to run the same script, it shouldn't run as the process is running else where. How do I do it?
I know i should write something, make it go to sleep captures its pid and write a condition that if this pis is running somewhere then don't let it run anywhere. but how do i do it? Please give a code.
echo "this is a process"
sleep 60
testfilepid = `ps ax | grep test1.sh | grep -v grep | tr -s " " | cut -f1 -d " "| tail -1`
echo $testfilepid
if [[ $tesfilepid = " " ]]
sh test1.sh
else
echo "this process is already running"
fi
This is what I tried. when i execute this in 2 windows, both the windows give me the output this is a process.
You could use pgrep to check that your script/process is running, and negate the output, this is a very basic example that could give you an idea:
if ! pgrep -f sleep >/dev/null; then echo "will sleep" && sleep 3; fi
Notice the !, pgrep -f sleep will search for a process matching against full argument lists. (you could customize this to your needs). so if nothing matches your pattern then your script will be called.
I can tail -f the last modified file using ls --sort=time | head -1 | xargs tail -f but I'm looking for something that would continuously run the first part as well, i.e. if a new files is created while the tail command is running it switches to tailing the new file. I thought about using watch command but that doesn't seem to work nicely with tail -f
It seems you need something like this:
#!/bin/bash
TAILPID=0
WATCHFILE=""
trap 'kill $(jobs -p)' EXIT # Makes sure we clean our mess (background processes) on exit
while true
do
NEWFILE=`ls --sort=time | head -n 1`
if [ "$NEWFILE" != "$WATCHFILE" ]; then
echo "New file has been modified"
echo "Now watching: $NEWFILE";
WATCHFILE=$NEWFILE
if [ $TAILPID -ne 0 ]; then
# Kill old tail
kill $TAILPID
wait $! &> /dev/null # supress "Terminated" message
fi
tail -f $NEWFILE &
TAILPID=$! # Storing tail PID so we could kill it later
fi
sleep 1
done
I have a filewatch program:
#!/bin/sh
# On Linux, uses inotifywait -mre close_write, and on OS X uses fswatch.
set -e
[[ "$#" -ne 1 ]] && echo "args count" && exit 2
if [[ `uname` = "Linux" ]]; then
inotifywait -mcre close_write "$1" | sed 's/,".*",//'
elif [[ `uname` = "Darwin" ]]; then
# sed on OSX/BSD wants -l for line-buffering
fswatch "$1" | sed -l 's/^[a-f0-9]\{1,\} //'
fi
echo "fswatch: $$ exiting"
And a construct i'm trying to use from a script (and I am testing with it on the command line on CentOS now):
filewatch . | while read line; do echo "file $line has changed\!\!"; done &
So what I am hoping this does is it will let me process, one line at a time, the output of inotify, which of course sends out one line for each file it has detected a change on.
Now for my script to clean stuff up properly I need to be able to kill this whole backgrounded pipeline when the script exits.
So i run it and then if I run kill on either the first part of the pipe or the second part, the other part does not terminate.
So I think if I kill the while read line part (which should be sh (zsh in the case of running on the cmd line)) then filewatch should be receiving a SIGPIPE. Okay so I am not handling that, I guess it can keep running.
If I kill filewatch, though, it looks like zsh continues with its while read line. Why?
The question applies to a script such as the following:
Script
#!/bin/sh
SRC="/tmp/my-server-logs"
echo "STARTING GREP JOBS..."
for f in `find ${SRC} -name '*log*2011*' | sort --reverse`
do
(
OUT=`nice grep -ci -E "${1}" "${f}"`
if [ "${OUT}" != "0" ]
then
printf '%7s : %s\n' "${OUT}" "${f}"
else
printf '%7s %s\n' "(none)" "${f}"
fi
) &
done
echo "WAITING..."
wait
echo "FINISHED!"
Current behavior
Pressing Ctrl+C in console terminates the script but not the already running grep processes.
Write a trap for Ctrl+c and in the trap kill all of the subprocesses. Put this before your wait command.
function handle_sigint()
{
for proc in `jobs -p`
do
kill $proc
done
}
trap handle_sigint SIGINT
A simple alternative is using a cat pipe. The following worked for me:
echo "-" > test.text;
for x in 1 2 3; do
( sleep $x; echo $x | tee --append test.text; ) &
done | cat
If I press Ctrl-C before the last number is printed to stdout. It also works if the text-generating command is something that takes a long time such as "find /", i.e. it is not only the connection to stdout through cat that is killed but actually the child process.
For large scripts that make extensive use of subprocesses the easiest way to ensure the indented Ctrl-C behaviour is wrapping the whole script into such a subshell, e.g.
#!/usr/bin/bash
(
...
) | cat
I am not sure though if this has the exactly same effect as Andrew's answer (i.e. I'm not sure what signal is sent to the subprocesses). Also I only tested this with cygwin, not with a native Linux shell.