My goal is to grep for a count of processes older than 'x' minutes in bash.
So far, I can grep for all the process execution times:
ps -eo etime,cmd | grep mysuperspecialprocess.sh | grep -Ev 'grep' | awk '{print $1}'
I pipe a wc -l to get a count at the end.
How can I grep or loop through the result to restrict it to processes older than a certain number of minutes?
Give this tested version a try.
It searches a specified running process, and count all occurences which has an associated elapsed time greater than a specified number of seconds.
Set parameters' values first and run it.
$ cat ./script.sh
#!/bin/bash --
process_name=mysuperspecialprocess.sh
elapsed_time_seconds=600
ps -eo etimes,cmd | fgrep "${process_name}" | fgrep -v fgrep | ( count=0 ; while read etimes cmd ; do \
if [ $etimes -gt $elapsed_time_seconds ] ;\
then \
count=$((count+1)) ;\
fi ;\
done ; echo $count )
etimes instructs ps to display seconds.
awk to the rescue!
starting with the etime field you can do something similar to this, ignores seconds.
$ awk 'NR>1{n=split($1,a,":"); # split the field
if(n==2) mins=a[1]+0; # if hours not present set mins
else {d=split(a[1],h,"-"); # check for days
hours=(d==2?h[1]*24+h[2]:h[1]); # incorporate days in hours
mins=hours*60+a[2]} # compute mins
print $1, mins}' etimes
00:04 0
02:30:44 150
01:03:11 63
1-01:01:01 1501
3-00:02:00 4322
Related
I have this script :
#!/bin/bash
ps -eo lstart,pid,cmd --sort lstart | while read line 2> /dev/null
do
if [ "$(date -d "${line::24}" "+%Y%m%d%H%M%S")" -gt "$(date -d "Thu Apr 7 00:55:38" "+%Y%m%d%H%M%S")" ] 2> /dev/null
then echo "Date : $(date -d "${line::24}" "+%d/%m/%Y %H:%M:%S") | PID & CMD : ${line:25:29}" >> process.log 2> /dev/null
fi 2> /dev/null
done
sort process.log | uniq > process.log
#sort process.log | uniq -u | tee process.log
My script runs automatically every 10 seconds, so I would like the identical lines to be deleted. As you can see, I tried with uniq but it doesn't work. I would like all lines in my file to be deleted if they are identical.
As I did, the second time the script is executed, there is nothing in the output file and I don't understand why.
I would also like nothing to be displayed in my terminal when the script runs. I used tee but when executing the uniq command, it returns an output in my terminal... How to remove it?
I thank you in advance for your help and wish you a good day
Thanks a lot
You should not parse ps output ever, especially not of lstart. Also, you are running date in the loop all the time, again and again.
I think something along this would be better to do:
some_date=$(date -d "Thu Apr 7 00:55:38" +%s)
now=$(date +%s)
how_much_time_ago=$(( now - some_date ))
ps -eo etimes,pid,cmd --sort etimes |
awk -v v="$how_much_time_ago" '$1 > v' |
while IFS=' ' read line etimes pid cmd; do
printf "Date : %s | PID & CMD : %s %s\n" \
"$(date -d "$((now - etimes))" "+%d/%m/%Y %H:%M:%S")" \
"$pid" "$cmd"
done |
sort |
uniq > process.log
Note that you can pipe the output of a while .... done | stuff loop to another thing normally. Instead of sprinkling 2>/dev/null everywhere, try to actually solve the issue, not hide the error.
I want to find out processes running more than 3 hrs, I have written a command for this but it's not returning expected output
ps -u <user> -o pid,stime,pcpu,pmem,etime,cmd --sort=start_time | \
grep <searchString> | grep -v grep| awk '{print $5}' | \
sed 's/:|-/ /g;'| awk '{print $4" "$3" "$2" "$1"}' | \
awk '$1+$2*60+$3*3600+$4*86400 > 10800'
but it's printing the values of etime in output. But expected output is, command should print the values of "pid,stime,pcpu,pmem,etime,cmd"
I am not able to find exact issue with this.
You are executing "awk '{print $5}'" which is taking in the input and printing out only column 5 which in your case is "etime" , everything from this point on is lost.
If your system supports etimes (notice the s on the end), you can easily do this with
ps -eo pid,etimes,etime,comm,user,tty | awk '{if ( $2>10800) print $0}'
on a system not supporting etimes which has a standard output of etime which hh:mm:ss or just mm:ss if no hours have passed
ps -eo pid,etime,comm,user,tty | awk '{seconds_old=10800 ; split($2,a,":",sep) ; if(length(a) < 3) b = (a[1] *60) + (a[2]) ; else b=((a[1]*3600) + (a[2] *60) + (a[3])) ; if(b > seconds_old ) print $0}'
Adjust "seconds_old" to change the age you want to test for:
There are various other methods of doing this using Find for example:
explained here:
https://serverfault.com/questions/181477/how-do-i-kill-processes-older-than-t
However, the solution should match your expected output
Try this:
ps -u <user> -o pid,stime,pcpu,pmem,etime=,cmd --sort=start_time|grep <searchString>|while read z;do tago=$(echo $z|awk '{print $5}'|sed -E 's/(:|-)/ /g'| awk '{print $4+$3*60+$2*3600+$1*86400}');if [ $tago -ge 10800 ];then echo $z;fi;done
It prints only processes >= 10800 secs old.
You can readjust the output further to fit your needs.
Able to find running process for more than 3 hrs with below command.
ps -u <user> -o pid,stime,pcpu,pmem,etime,cmd --sort=start_time |grep -v grep|awk 'substr($0,23,2) > 3'
We have a requirement to report on the number of lines written to a 7 day cycle of log files. The log files are called - [filename].log.1.gz for today, [filename].log.2.gz for yesterday up to [filename].log.7.gz for the 7th day
I was hoping to create a script that would output the numbers at once, instead of running the zcat [filename].log.1.gz | wc -l command against each line. I was also hoping to have a meaningful message against each outputted value
I can write a bash script that will do each line as the name of the files are the same, but I was hoping for something a bit more elegant
Instead of this
zcat position.log.3.gz | wc -l
zcat position.log.4.gz | wc -l
zcat position.log.5.gz | wc -l
zcat position.log.6.gz | wc -l
zcat position.log.7.gz | wc -l
I was hoping for something more like this
for i in {1..7}
c=$(zcat position.log.$i.gz | wc -l)
message=$"The count for "
date1=$(date --date='$i days ago')
result=$"$message$date1$c"
echo $result
done
However, I can't get this to run.
Any ideas?
You script with small fixes:
# for loop starts with a 'do'
for i in {1..7}; do
# indentation makes everything redable
# always quote your variables
c=$(zcat "position.log.$i.gz" | wc -l)
# The "" is just a string, no '$' sign before it
message="The count for "
# The '$i' is literally `$i`, if you want to _expand_ a variable, you have to use "$i"
date1=$(date --date="$i days ago")
# same as above, drop the $
result="$message$date1$c"
# always quote your variables
echo "$result"
done
Or maybe a tiny little bit shorter:
for i in {1..7}; do
echo "The count for $(date --date="$i days ago") is $(zcat "position.log.$i.gz" | wc -l)"
done
I have a file and I am processing it line by line and producing another file with the result. I want to monitor the percentage of completion. In my case, it is just the number of lines in the new file divide by the number of lines from the input file. A simple example would be:
$ cat infile
unix
is
awesome
$ cat infile | process.sh >> outfile &
Now, if I run my command, I should get 0.33 if process.sh completed the first line.
Any suggestions?
You can use pv for progress (in debian/ubuntu inside package pv):
pv -l -s `wc -l file.txt` file.txt | process.sh
This will use number of lines for progress.
Or you can use just the number of bytes:
pv file.txt | process.sh
The above commands will show you the percentage of completion and ETA.
You can use bc:
echo "scale=2; $(cat outfile | wc -l) / $(cat infile | wc -l) * 100" | bc
In addition, combine this with watch for updated progress:
watch -d "echo \"scale=2; \$(cat outfile | wc -l) / \$(cat infile | wc -l) * 100\" | bc"
TOTAL_LINES=`wc -l infile`
LINES=`wc -l outfile`
PERCENT=`echo "scale=2;${LINES}/${TOTAL_LINES}" | bc | sed -e 's_^\.__'`
echo "${PERCENT} % Complete"
scale=2 means you get two significant digits.
I'm trying to take the output of a ps axo uname,pid,etime,time,cmd and kill all related processes with an etime>=10
I need to kill pids only for on specific users which are stored in a separate file, users.txt
Basically, I want to see all my columns and I want to find all php{5,54} processes that belong to specific users, stored in a file, and kill those processes if the execution time is over ten minutes. (kill -9 not needed)
Example ps output:
username 574 01:37 00:00:18 /ramdisk/bin/php54 /home/username/public_html/index.php
usernum2 1367 10:28 00:00:16 /ramdisk/bin/php54 /home/usernum2/public_html/index.php
user3 3971 1-04:17:31 00:00:14 /ramdisk/bin/php54 /home/user3/public_html/index.php
usernum4 9130 14:05:32 00:00:29 /ramdisk/bin/php54 /home/usernum4/public_html/index.php
username 9189 1-01:31:12 00:00:25 /ramdisk/bin/php54 /home/username/public_html/index.php
My thought has been to put the ps output into a file (say procs.txt), and then grep through that.
EG:
ps axo uname,pid,etim,time,cmd | grep 'php5' | tee procs.txt
I could have two separate lines: one that says if column 3 is more than 5 characters kill the pid, which is easy, and then another with something like the following, but that doesn't leave me with the related pids, so I can't kill them:
for i in $( cat users.txt ); do grep $i procs.txt | awk '{print $3}' | awk -F: '{print $(NF-1)}' | awk '$1>=10{print $1}'; done
The following is a solution in bash.
To find all processes that match php5 or php54, for users in a file, I would use the U option to ps and egrep:
ps xo uname,pid,etim,time,cmd U"$(echo $(< users.txt))" | egrep '/php54? '
The U option will only display the processes that belong to the supplied list of users. The < is a shell builtin for cat, and the use of echo flattens the file entries into a single line that the ps command expects.
'/php54? ' indicates that the 4 is optional, and the space at the end makes sure it doesn't match (for example) php52. egrep uses regular expressions, so it allows the use of the ? meta character.
The output of this command can then be processed line by line with a while read:
ps xo uname,pid,etim,time,cmd U"$(echo $(< users.txt))" | egrep '/php54? ' \
| while read a
do cols=($a)
elapsed=${cols[2]}
if [ ${#elapsed} -gt 5 ]
then kill ${cols[1]}
elif [ ${elapsed%%:*} -gt 9 ]
then kill ${cols[1]}
fi
done
cols gets an array assignment, splitting the line a on whitespace boundaries. USER is in cols[0], PID in cols[1], and ELAPSED is in cols[2]. The PID is killed if the string length of ELAPSED is greater than 5 (indicating an elapsed time of at least an hour), or if the minutes field is greater than 9 (an elapsed time of at least 10 minutes). The %% expansion operator does a longest match suffix removal.
The solution can be compacted by read-ing directly into the relevant fields:
ps xo uname,pid,etim,time,cmd U"$(echo $(< users.txt))" | egrep '/php54? ' \
| while read user pid elapsed rest_of_line
do if [ ${#elapsed} -gt 5 ]
then kill $pid
elif [ ${elapsed%%:*} -gt 9 ]
then kill $pid
fi
done