I'm basically trying to make debugging easier for other scripts of mine.
(Centos 7.6)
What I need is a script doing :
tail -f the last file entry in a directory
if a new file appears in this directory, it logs this new file, smoothly
if I send a SIGINT (Ctrl+C), it doesn't leave orphans
with the less possible add-ons for the maximum portability
This is my non working solution :
CURRENT_FILE=`ls -1t | head -n1`
tail -n 100 -f "$CURRENT_FILE" &
PID=$!
while [ true ];
do
#is there a new file in the directory ?
NEW_FILE=`ls -1t | head -n1`
if [[ "$CURRENT_FILE" != "$NEW_FILE" ]]; then
#yes, so kill last tail
kill -9 $PID
clear
#tail on the new file
CURRENT_FILE=$NEW_FILE
tail -n 100 -f "$CURRENT_FILE"
PID=$!
fi
sleep 1s
done
The problem with this solution is that when I'm sending SIGINT (Ctrl+C), what I normally do when exiting a "tail -f", it leaves an orphan child in the background.
I've searched solution with "trap" but I don't get it well, and it doesn't seem to work with an eternal process like "tail -f".
I'll be glad to here your thoughts about that and get into advanced bash programming.
You can trap whenever the script exits and kill the process then. You don't need -9 to kill your tail though, that's overkill.
You can also use inotify to tell you when something happens in the directory instead of sleeping and rechecking. Here's a basic building block. inotify has a lot of events you can wait for. You can add detection if the file was moved/renamed so you don't have to restart the tail in those cases etc.
#!/bin/bash
killpid() {
if [[ -n $PID ]]; then
kill $PID
PID=""
fi
}
trap killpid EXIT
DIR="."
CURRENT_FILE="$(ls -1t "$DIR" | head -n1)"
tailit() {
echo "::: $CURRENT_FILE :::"
tail -n 100 -f "$CURRENT_FILE" &
PID=$!
}
tailit
# wait for any file to be created, modified or deleted
while EVENT=$(inotifywait -q -e create,modify,delete "$DIR"); do
# extract event
ev=$(sed -E "s/^${DIR}\/ (\S+) .+$/\1/" <<< "$EVENT")
# extract the affected file
NEW_FILE=${EVENT#${DIR}/ $ev }
case $ev in
MODIFY)
# start tailing the file if we aren't tailing it already
if [[ $NEW_FILE != $CURRENT_FILE ]]; then
killpid
CURRENT_FILE="$NEW_FILE"
tailit
fi
;;
CREATE)
# a new file, tail it
killpid
CURRENT_FILE="$NEW_FILE"
tailit
;;
DELETE)
# stop tailing if the file we are tailing was deleted
if [[ $NEW_FILE == $CURRENT_FILE ]]; then
echo "::: $CURRENT_FILE removed :::"
CURRENT_FILE=""
killpid
fi
;;
esac
done
You can use trap solution at the beginning of your shell.
#! /bin/bash
trap ctrl_c INT
function ctrl_c() {
if [[ -n "$PID" ]]; then
kill -9 $PID
fi
exit 0
}
CURRENT_FILE=`ls -1t | head -n1`
tail -n 100 -f "$CURRENT_FILE" &
PID=$!
while [ true ];
do
#is there a new file in the directory ?
NEW_FILE=`ls -1t | head -n1`
if [[ "$CURRENT_FILE" != "$NEW_FILE" ]]; then
#yes, so kill last tail
kill -9 $PID
clear
#tail on the new file
CURRENT_FILE=$NEW_FILE
tail -n 100 -f "$CURRENT_FILE" &
PID=$!
fi
sleep 1s
done
For some monitoring purposes, for a given list of files (that change path daily) I want to launch a tail -f in the background searching for specific strings
How do I properly get the exact pid of the tail commands generated by the for loop, so I could kill them later ?
Here's what I'm trying to do which I think gives me the pid of the script and not the tail process.
for f in $(find file...)
do
tail -f $f | while read line
do case $line in
*string_to_search*) echo " $line" | mutt -s "string detected in file : $1" mail#mail.com;
;;
esac
done &
echo $! >> pids.txt
done
I use the following script to check whether wget has finished downloading. To check for this, I'm looking for its PID, and when it is not found the computer shutdowns. This works fine for a single instance of wget, however, I'd like the script to look for all already running wget programs.
#!/bin/bash
while kill -0 $(pidof wget) 2> /dev/null; do
for i in '-' '/' '|' '\'
do
echo -ne "\b$i"
sleep 0.1
done
done
poweroff
EDIT: I'd would be great if the script would check if at least one instance of wget is running and only then check whether wget has finished and shutdown the computer.
In addition to the other answers, you can satisfy your check for at least one wget pid by initially reading the result of pidof wget into an array, for example:
pids=($(pidof wget))
if ((${#pids[#]} > 0)); then
# do your loop
fi
This also brings up a way to routinely monitor the remaining pids as each wget operation completes, for example,
edit
npids=${#pids[#]} ## save original number of pids
while (( ${#pids[#]} -gt 0 )); do ## while pids remain
for ((i = 0; i < npids; i++)); do ## loop, checking remaining pids
kill -0 ${pids[i]} || pids[$i]= ## if not unset in array
done
## do your sleep and spin
done
poweroff
There are probably many more ways to do it. This is just one that came to mind.
I don't think kill is a right Idea,
may be some thing on the lines like this
while [ 1 ]
do
live_wgets=0
for pid in `ps -ef | grep wget| awk '{print $2}'` ; # Adjust the grep
do
live_wgets=$((live_wgets+1))
done
if test $live_wgets -eq 0; then # shutdown
sudo poweroff; # or whatever that suits
fi
sleep 5; # wait for sometime
done
You can adapt your script in the following way:
#!/bin/bash
spin[0]="-"
spin[1]="\\"
spin[2]="|"
spin[3]="/"
DOWNLOAD=`ps -ef | grep wget | grep -v grep`
while [ -n "$DOWNLOAD" ]; do
for i in "${spin[#]}"
do
DOWNLOAD=`ps -ef | grep wget | grep -v grep`
echo -ne "\b$i"
sleep 0.1
done
done
sudo poweroff
However I would recommend using cron instead of an active waiting approach or even use wait
How to wait in bash for several subprocesses to finish and return exit code !=0 when any subprocess ends with code !=0?
I have a filewatch program:
#!/bin/sh
# On Linux, uses inotifywait -mre close_write, and on OS X uses fswatch.
set -e
[[ "$#" -ne 1 ]] && echo "args count" && exit 2
if [[ `uname` = "Linux" ]]; then
inotifywait -mcre close_write "$1" | sed 's/,".*",//'
elif [[ `uname` = "Darwin" ]]; then
# sed on OSX/BSD wants -l for line-buffering
fswatch "$1" | sed -l 's/^[a-f0-9]\{1,\} //'
fi
echo "fswatch: $$ exiting"
And a construct i'm trying to use from a script (and I am testing with it on the command line on CentOS now):
filewatch . | while read line; do echo "file $line has changed\!\!"; done &
So what I am hoping this does is it will let me process, one line at a time, the output of inotify, which of course sends out one line for each file it has detected a change on.
Now for my script to clean stuff up properly I need to be able to kill this whole backgrounded pipeline when the script exits.
So i run it and then if I run kill on either the first part of the pipe or the second part, the other part does not terminate.
So I think if I kill the while read line part (which should be sh (zsh in the case of running on the cmd line)) then filewatch should be receiving a SIGPIPE. Okay so I am not handling that, I guess it can keep running.
If I kill filewatch, though, it looks like zsh continues with its while read line. Why?
I need a platform independent (Linux/Unix|OSX) shell/bash command that will determine if a specific process is running. e.g. mysqld, httpd...
What is the simplest way/command to do this?
While pidof and pgrep are great tools for determining what's running, they are both, unfortunately, unavailable on some operating systems. A definite fail safe would be to use the following: ps cax | grep command
The output on Gentoo Linux:
14484 ? S 0:00 apache2
14667 ? S 0:00 apache2
19620 ? Sl 0:00 apache2
21132 ? Ss 0:04 apache2
The output on OS X:
42582 ?? Z 0:00.00 (smbclient)
46529 ?? Z 0:00.00 (smbclient)
46539 ?? Z 0:00.00 (smbclient)
46547 ?? Z 0:00.00 (smbclient)
46586 ?? Z 0:00.00 (smbclient)
46594 ?? Z 0:00.00 (smbclient)
On both Linux and OS X, grep returns an exit code so it's easy to check if the process was found or not:
#!/bin/bash
ps cax | grep httpd > /dev/null
if [ $? -eq 0 ]; then
echo "Process is running."
else
echo "Process is not running."
fi
Furthermore, if you would like the list of PIDs, you could easily grep for those as well:
ps cax | grep httpd | grep -o '^[ ]*[0-9]*'
Whose output is the same on Linux and OS X:
3519 3521 3523 3524
The output of the following is an empty string, making this approach safe for processes that are not running: echo ps cax | grep aasdfasdf | grep -o '^[ ]*[0-9]*'
This approach is suitable for writing a simple empty string test, then even iterating through the discovered PIDs.
#!/bin/bash
PROCESS=$1
PIDS=`ps cax | grep $PROCESS | grep -o '^[ ]*[0-9]*'`
if [ -z "$PIDS" ]; then
echo "Process not running." 1>&2
exit 1
else
for PID in $PIDS; do
echo $PID
done
fi
You can test it by saving it to a file (named "running") with execute permissions (chmod +x running) and executing it with a parameter: ./running "httpd"
#!/bin/bash
ps cax | grep httpd
if [ $? -eq 0 ]; then
echo "Process is running."
else
echo "Process is not running."
fi
WARNING!!!
Please keep in mind that you're simply parsing the output of ps ax which means that, as seen in the Linux output, it is not simply matching on processes, but also the arguments passed to that program. I highly recommend being as specific as possible when using this method (e.g. ./running "mysql" will also match 'mysqld' processes). I highly recommend using which to check against a full path where possible.
References:
http://linux.about.com/od/commands/l/blcmdl1_ps.htm
http://linux.about.com/od/commands/l/blcmdl1_grep.htm
You SHOULD know the PID !
Finding a process by trying to do some kind of pattern recognition on the process arguments (like pgrep "mysqld") is a strategy that is doomed to fail sooner or later. What if you have two mysqld running? Forget that approach. You MAY get it right temporarily and it MAY work for a year or two but then something happens that you haven't thought about.
Only the process id (pid) is truly unique.
Always store the pid when you launch something in the background. In Bash this can be done with the $! Bash variable. You will save yourself SO much trouble by doing so.
How to determine if process is running (by pid)
So now the question becomes how to know if a pid is running.
Simply do:
ps -o pid= -p <pid>
This is POSIX and hence portable. It will return the pid itself if the process is running or return nothing if the process is not running. Strictly speaking the command will return a single column, the pid, but since we've given that an empty title header (the stuff immediately preceding the equals sign) and this is the only column requested then the ps command will not use header at all. Which is what we want because it makes parsing easier.
This will work on Linux, BSD, Solaris, etc.
Another strategy would be to test on the exit value from the above ps command. It should be zero if the process is running and non-zero if it isn't. The POSIX spec says that ps must exit >0 if an error has occurred but it is unclear to me what constitutes 'an error'. Therefore I'm not personally using that strategy although I'm pretty sure it will work as well on all Unix/Linux platforms.
On most Linux distributions, you can use pidof(8).
It will print the process ids of all running instances of specified processes, or nothing if there are no instances running.
For instance, on my system (I have four instances of bashand one instance of remmina running):
$ pidof bash remmina
6148 6147 6144 5603 21598
On other Unices, pgrep or a combination of ps and grep will achieve the same thing, as others have rightfully pointed out.
This should work on most flavours of Unix, BSD and Linux:
PATH=/usr/ucb:${PATH} ps aux | grep httpd | grep -v grep
Tested on:
SunOS 5.10 [Hence the PATH=...]
Linux 2.6.32 (CentOS)
Linux 3.0.0 (Ubuntu)
Darwin 11.2.0
FreeBSD 9.0-STABLE
Red Hat Enterprise Linux ES release 4
Red Hat Enterprise Linux Server release 5
The simpliest way is to use ps and grep:
command="httpd"
running=`ps ax | grep -v grep | grep $command | wc -l`
if [ running -gt 0 ]; then
echo "Command is running"
else
echo "Command is not running"
fi
If your command has some command arguments, then you can also put more 'grep cmd_arg1' after 'grep $command' to filter out other possible processes that you are not interested in.
Example: show me if any java process with supplied argument:
-Djava.util.logging.config.file=logging.properties
is running
ps ax | grep -v grep | grep java | grep java.util.logging.config.file=logging.properties | wc -l
Putting the various suggestions together, the cleanest version I was able to come up with (without unreliable grep which triggers parts of words) is:
kill -0 $(pidof mysql) 2> /dev/null || echo "Mysql ain't runnin' message/actions"
kill -0 doesn't kill the process but checks if it exists and then returns true, if you don't have pidof on your system, store the pid when you launch the process:
$ mysql &
$ echo $! > pid_stored
then in the script:
kill -0 $(cat pid_stored) 2> /dev/null || echo "Mysql ain't runnin' message/actions"
Just a minor addition: if you add the -c flag to ps, you don't need to remove the line containing the grep process with grep -v afterwards. I.e.
ps acux | grep cron
is all the typing you'll need on a bsd-ish system (this includes MacOSX) You can leave the -u away if you need less information.
On a system where the genetics of the native ps command point back to SysV, you'd use
ps -e |grep cron
or
ps -el |grep cron
for a listing containing more than just pid and process name. Of course you could select the specific fields to print out using the -o <field,field,...> option.
I use pgrep -l httpd but not sure it is present on any platform...
Who can confirm on OSX?
This approach can be used in case commands 'ps', 'pidof' and rest are not available.
I personally use procfs very frequently in my tools/scripts/programs.
egrep -m1 "mysqld$|httpd$" /proc/[0-9]*/status | cut -d'/' -f3
Little explanation what is going on:
-m1 - stop process on first match
"mysqld$|httpd$" - grep will match lines which ended on mysqld OR httpd
/proc/[0-9]* - bash will match line which started with any number
cut - just split the output by delimiter '/' and extract field 3
You should know the PID of your process.
When you launch it, its PID will be recorded in the $! variable. Save this PID into a file.
Then you will need to check if this PID corresponds to a running process. Here's a complete skeleton script:
FILE="/tmp/myapp.pid"
if [ -f $FILE ];
then
PID=$(cat $FILE)
else
PID=1
fi
ps -o pid= -p $PID
if [ $? -eq 0 ]; then
echo "Process already running."
else
echo "Starting process."
run_my_app &
echo $! > $FILE
fi
Based on the answer of peterh. The trick for knowing if a given PID is running is in the ps -o pid= -p $PID instruction.
None of the answers worked for me, so heres mine:
process="$(pidof YOURPROCESSHERE|tr -d '\n')"
if [[ -z "${process// }" ]]; then
echo "Process is not running."
else
echo "Process is running."
fi
Explanation:
|tr -d '\n'
This removes the carriage return created by the terminal. The rest can be explained by this post.
This prints the number of processes whose basename is "chromium-browser":
ps -e -o args= | awk 'BEGIN{c=0}{
if(!match($1,/^\[.*\]$/)){sub(".*/","",$1)} # Do not strip process names enclosed by square brackets.
if($1==cmd){c++}
}END{print c}' cmd="chromium-browser"
If this prints "0", the process is not running. The command assumes process path does not contain breaking space. I have not tested this with suspended processes or zombie processes.
Tested using gwak as the awk alternative in Linux.
Here is a more versatile solution with some example usage:
#!/bin/sh
isProcessRunning() {
if [ "${1-}" = "-q" ]; then
local quiet=1;
shift
else
local quiet=0;
fi
ps -e -o pid,args= | awk 'BEGIN{status=1}{
name=$2
if(name !~ /^\[.*\]$/){sub(".*/","",name)} # strip dirname, if process name is not enclosed by square brackets.
if(name==cmd){status=0; if(q){exit}else{print $0}}
}END{exit status}' cmd="$1" q=$quiet
}
process='chromium-browser'
printf "Process \"${process}\" is "
if isProcessRunning -q "$process"
then printf "running.\n"
else printf "not running.\n"; fi
printf "Listing of matching processes (PID and process name with command line arguments):\n"
isProcessRunning "$process"
Here is my version. Features:
checks for exact program name (first argument of the function). search for "mysql" will not match running "mysqld"
searches program arguments (second argument of the function)
script:
#!/bin/bash
# $1 - cmd
# $2 - args
# return: 0 - no error, running; 1 - error, not running
function isRunning() {
for i in $(pidof $1); do
cat /proc/$i/cmdline | tr '\000' ' ' | grep -F -e "$2" 1>&2> /dev/null
if [ $? -eq 0 ]; then
return 0
fi
done
return 1
}
isRunning java "-Djava.util.logging.config.file=logging.properties"
if [ $? -ne 0 ]; then
echo "not running, starting..."
fi
[ $pid ] && [ -d /proc/$pid ] && command if you know the pid
The following shell function, being only based on POSIX standard commands and options should work on most (if not any) Unix and linux system. :
isPidRunning() {
cmd=`
PATH=\`getconf PATH\` export PATH
ps -e -o pid= -o comm= |
awk '$2 ~ "^.*/'"$1"'$" || $2 ~ "^'"$1"'$" {print $1,$2}'
`
[ -n "$cmd" ] &&
printf "%s is running\n%s\n\n" "$1" "$cmd" ||
printf "%s is not running\n\n" $1
[ -n "$cmd" ]
}
$ isPidRunning httpd
httpd is running
586 /usr/apache/bin/httpd
588 /usr/apache/bin/httpd
$ isPidRunning ksh
ksh is running
5230 ksh
$ isPidRunning bash
bash is not running
Note that it will choke when passed the dubious "0]" command name and will also fail to identify processes having an embedded space in their names.
Note too that the most upvoted and accepted solution demands non portable ps options and gratuitously uses a shell that is, despite its popularity, not guaranteed to be present on every Unix/Linux machine (bash)