sleep getting hang in shell script - linux

I had a cron script running on my Ubuntu PC which periodically checks for disk space. There was a "sleep 5" also used in this script. This system was up for a month, but an issue is encountered. There were multiple instances of cron script running (around thousands) and multiple instances of "sleep".
Crontab entry:
* * * * * /root/disk_check_script.sh
Content of disk_check_script.sh :
#!/bin/sh -x
/home/user/linux/process_health_daemons status
if [ $? -ne 0 ]; then
/home/user/linux/process_health_daemons start
fi
sleep 5
MINIMUM_FREE_SPACE_REQUIRED=50
AVAILABLE_FREE_SPACE=$(df -Ph $HOME | tail -1 | awk '{ print $4}' | awk -F "G" '{ print $1 }')
USED_SPACE=$(df -Ph $HOME | tail -1 | awk '{ print $3}' | awk -F "G" '{ print $1 }')
if [ "$AVAILABLE_FREE_SPACE" -lt "$MINIMUM_FREE_SPACE_REQUIRED" ]; then
touch /home/user/flag/no_space_left
fi
When I killed one of the sleep process, cron script associated with it also completed.
Does sleep hangs often? Is it required to restart the machine to avoid this? Any other method to avoid this.

Related

How to delete identical lines in my output files?

I have this script :
#!/bin/bash
ps -eo lstart,pid,cmd --sort lstart | while read line 2> /dev/null
do
if [ "$(date -d "${line::24}" "+%Y%m%d%H%M%S")" -gt "$(date -d "Thu Apr 7 00:55:38" "+%Y%m%d%H%M%S")" ] 2> /dev/null
then echo "Date : $(date -d "${line::24}" "+%d/%m/%Y %H:%M:%S") | PID & CMD : ${line:25:29}" >> process.log 2> /dev/null
fi 2> /dev/null
done
sort process.log | uniq > process.log
#sort process.log | uniq -u | tee process.log
My script runs automatically every 10 seconds, so I would like the identical lines to be deleted. As you can see, I tried with uniq but it doesn't work. I would like all lines in my file to be deleted if they are identical.
As I did, the second time the script is executed, there is nothing in the output file and I don't understand why.
I would also like nothing to be displayed in my terminal when the script runs. I used tee but when executing the uniq command, it returns an output in my terminal... How to remove it?
I thank you in advance for your help and wish you a good day
Thanks a lot

You should not parse ps output ever, especially not of lstart. Also, you are running date in the loop all the time, again and again.
I think something along this would be better to do:
some_date=$(date -d "Thu Apr 7 00:55:38" +%s)
now=$(date +%s)
how_much_time_ago=$(( now - some_date ))
ps -eo etimes,pid,cmd --sort etimes |
awk -v v="$how_much_time_ago" '$1 > v' |
while IFS=' ' read line etimes pid cmd; do
printf "Date : %s | PID & CMD : %s %s\n" \
"$(date -d "$((now - etimes))" "+%d/%m/%Y %H:%M:%S")" \
"$pid" "$cmd"
done |
sort |
uniq > process.log
Note that you can pipe the output of a while .... done | stuff loop to another thing normally. Instead of sprinkling 2>/dev/null everywhere, try to actually solve the issue, not hide the error.

LINUX- Checking the internal CPU, RAM usage of JVM's

So I am trying to check the internal CPU and RAM usage from JVM's.
The set up is that we have a single server which hosts 39 JVM's each running its own service and has it's own unique CPU & RAM allocation. So for example it could be set up like the following:
CPU(Cores) RAM(MB)
JVM1 2 300
JVM2 1 50
JVM3 5 1024
These are fictional as I don't have the actual values to hand.
I know the PID's for each of the JVM's but I am wondering how would I see the CPU and RAM usage of each JVM on it's own dissregarding the Host systems usage. Also is it possible to pass multiple PID's into a jstat -gc as I will be looking to script this?.
I know that if I use:
ps -p <PID> -o %CPU %RAM
That will give me the CPU and RAM of that process on the host machine.
Any Help is greatly appreciated.

After playing around for a while I came up with the following script which pulls the memory usage and total memory in use to 2DP:
Script
#!/bin/sh
service=$(mktemp)
jstatop=$(mktemp)
ServiceStatsPre=$(mktemp)
datet=$(date +%Y-%m-%d)
hourt=$(date +%H)
ps -ef | grep java | grep Service | awk '{key=substr($9,31,match($9,"#")-31); serv[key]++}END{for(name in serv){if($9 ~ /Service/){print name}}}'>$service
printf "%-8sService\t\t\t\t%7s PID\t%7s Used\t%7s Total\n" > $ServiceStatsPre
filename=$service
while read -r line; do
if [ ! -z $line ]; then
pid=$(ps -ef | grep java | grep Service |awk -v svc="$line" '{if (match($9,svc)){print $2}}')
rncnt=0
rnag=1
while [ $rnag -eq 1 ]; do
jstat -gc $pid > $jstatop
if [ $? -ne 0 ]; then
sleep 5;
rncnt++;
else
rnag=0
fi
if [ $rncnt -eq 5 ]; then
rnag=0
fi
done
cat $jstatop | awk '{if (NR !=1) print}'|awk -v pid="$pid" -v svc="$line" -v d="$datet" -v h="$hourt" '{printf("%-40s %7d %6.2fMB %6.2fMB %11s %3s\n",svc,pid,($1+$5+$7)/1024,($3+$4+$6+$8)/1024,d,h)}' >> $ServiceStatsPre
fi
done < $filename
#printf "Date,Hour,Service,Used,Total\n" > Service_Stats.csv #Uncomment this line on initial Run to create file with headders
cat $ServiceStatsPre | awk '{if (NR !=1) print}' |
awk '{
printf("%-1s,%1s,%1s,%4.2f,%4.2f\n",$5,$6,$1,$3,$4);
}' >> Service_Stats.csv
rm $service;
rm $jstatop;
rm $ServiceStatsPre;
Breakdown
ps -ef | grep java | grep Service | awk '{key=substr($9,31,match($9,"#")-31); serv[key]++}END{for(name in serv){if($9 ~ /Service/){print name}}}'>$service
This is returning the name of the Service and putting it to a tempary file $service
printf "%-8sService\t\t\t\t%7s PID\t%7s Used\t%7s Total\n" > $ServiceStatsPre
This is just creating Headings in a Temp file $ServiceStatsPre (I know this isn't needed but I was originally using it to debug to a file not a temp file)
Outer While loop
filename=$service
if [ ! -z $line ]; then
while read -r line; do
pid=$(ps -ef | grep java | grep Service |awk -v svc="$line" '{if (match($9,svc)){print $2}}')
rncnt=0
rnag=1
...
cat $jstatop | awk '{if (NR !=1) print}'|awk -v pid="$pid" -v svc="$line" -v d="$datet" -v h="$hourt" '{printf("%-40s %7d %6.2fMB %6.2fMB %11s %3s\n",svc,pid,($1+$5+$7)/1024,($3+$4+$6+$8)/1024,d,h)}' >> $ServiceStatsPre
fi
done < $filename
This is first checking to see if the line that it is passing in is not empty (if [ ! -z $line ]; then) then it is extracting the PID for each of the services in the temp file $service and is setting a check (rnag) + retry counter (rncnt). The final part of this outer while loop is calculating the stats, memory usage in MB (($1+$5+$7)/1024), Total Memory used (($3+$4+$6+$8)/1024) as well as retuning the PID,Service Name,Date and Hour.
Inner While Loop
...
while [ $rnag -eq 1 ]; do
jstat -gc $pid > $jstatop
if [ $? -ne 0 ]; then
sleep 5;
rncnt++;
else
rnag=0
fi
if [ $rncnt -eq 5 ]; then
rnag=0
fi
done
...
This is to basically ensure that the jstat is running correctly and isn't returning an error.
The final part of this script:
#printf "Date,Hour,Service,Used,Total\n" > Service_Stats.csv #Uncomment this line on initial Run to create file with headders
cat $ServiceStatsPre | awk '{if (NR !=1) print}' |
awk '{
printf("%-1s,%1s,%1s,%4.2f,%4.2f\n",$5,$6,$1,$3,$4);
}' >> Service_Stats.csv
is just to resolve the formatting and put it into a format so that it can be used in a CSV.
Output
This script out puts like the following:
Date,Hour,Service,Used,Total
2016-07-03,07,undMFSAdapterServiceVM,331.00,188.87
2016-07-03,07,entSCSServiceVM,332.50,278.19
2016-07-03,07,kServiceVM,457.50,132.91
2016-07-03,07,ServiceTuTrackVm,432.00,282.66
2016-07-03,07,ter1WMSAdapterServiceVm,233.00,77.02
2016-07-03,07,kingMFSAdapterServiceVM,451.50,126.69
2016-07-03,07,erBuilderServiceVM,261.50,211.27
2016-07-03,07,ter3MFSAdapterServiceVM,449.50,210.23
2016-07-03,07,rServiceVM1,1187.00,529.26
2016-07-03,07,rServiceVM2,597.50,398.43
2016-07-03,07,rServiceVM3,2786.00,819.30
2016-07-03,07,rServiceVM4,451.50,163.13
2016-07-03,07,MessagingServiceVm,457.50,357.11
2016-07-03,07,viceVM,444.50,263.59
2016-07-03,07,ServiceVM,1910.50,909.19
2016-07-03,07,undPackingMFSAdapterServiceVM,208.00,113.51
2016-07-03,07,gisticLockServiceVM,245.00,173.05
2016-07-03,07,kingMFSAdapterServiceVM,781.50,327.13
2016-07-03,07,ferWMSAdapterServiceVm,196.00,84.02
2016-07-03,07,geMFSAdapterServiceVM,499.50,256.91
2016-07-03,07,ferMFSAdapterServiceVM,456.50,246.89
2016-07-03,07,kingWMSAdapterServiceVm,195.00,73.70
2016-07-03,07,AdapterServiceVm,149.50,72.62
2016-07-03,07,ter2MFSAdapterServiceVM,455.00,136.02
2016-07-03,07,ionServiceVM,484.00,240.46
2016-07-03,07,ter3WMSAdapterServiceVm,266.00,138.70
2016-07-03,07,iceVm,135.50,106.07
2016-07-03,07,viceVm,3317.00,1882.15
2016-07-03,07,entBCSServiceVM,356.50,143.93
2016-07-03,07,ServiceVm,951.00,227.12
2016-07-03,07,lingServiceVM,145.50,76.61
2016-07-03,07,entDBHServiceVM,182.50,4.63
2016-07-03,07,kingWMSAdapterServiceVm,208.00,103.13
2016-07-03,07,gingServiceVM,1529.50,235.84
2016-07-03,07,ServiceVM,249.50,131.78
2016-07-03,07,ter1MFSAdapterServiceVM,453.00,394.11
2016-07-03,07,AdapterServiceVM,461.00,208.41
2016-07-03,07,ter2WMSAdapterServiceVm,178.50,79.93
2016-07-03,07,AdapterServiceVm,395.00,131.08
2016-07-03,07,ingServiceVM,184.50,126.28
The $3 has been trimmed just to hide service names etc..
Helpful Links
A nice explanation of jstat -gc output

Linux - Extract core utilization for a single process to a file

I am trying to figure out how to extract the cpu core utilization for a SINGLE process in linux and parse it. I know that I can get the overall core utilization via top and then press "1". I am already able to parse that. However, now I want to do the same thing for a single process. I tried it with ps and calculating the core utilization myself but I am not so sure if my script is accurate enough, something seems off. (Note that this version calculates overall core utilization since it is WIP) I get errors like this after a while in my terminal:
test.sh: line 31: +: syntax error: operand expected (error token is
"+")
I cannot figure out why this error just randomly occurs.
#!/bin/env bash
read -p "Enter PID to observe:" pid
run=false
ps -p $pid -L -o cputime,etime,psr,pcpu
for (( u = 1 ; u <= 100 ; u++ ))
do
lines=$(ps -p $pid -L -o psr,pcpu | awk 'END{print NR}')
for (( i=1; i<=$lines; i++))
do
core=$(ps -p $pid -L -o psr,pcpu | awk 'NR=='$i'{print $1}')
if [ $run == false ]
then cpuTimeS=+$(ps -p $pid -L -o cputime,etime | awk 'NR=='$i'{print $1}' | awk -F : '{ printf("%.2f\n", $1*60+$2*60+$3); }')
elapsedTimeS=+$(ps -p $pid -L -o cputime,etime | awk 'NR=='$i'{print $2}' | awk -F : '{ printf("%.2f\n", $1*60+$2*60+$3); }')
cpuTimeSi=${cpuTimeS%.*}
elapsedTimeSi=${elapsedTimeS%.*}
cpuTimeSiResult=$(( cpuTimeSi + cpuTimeSiResult ))
elapsedTimeSiResult=$(( elapsedTimeSi + elapsedTimeSiResult ))
else
cpuTimeE=+$(ps -p $pid -L -o cputime,etime | awk 'NR=='$i'{print $1}' | awk -F : '{ printf("%.2f\n", $1*60+$2*60+$3); }')
elapsedTimeE=+$(ps -p $pid -L -o cputime,etime | awk 'NR=='$i'{print $2}' | awk -F : '{ printf("%.2f\n", $1*60+$2*60+$3); }')
cpuTimeEi=${cpuTimeE%.*}
elapsedTimeEi=${elapsedTimeE%.*}
cpuTimeEiResult=$(( cpuTimeEi + cpuTimeEiResult ))
elapsedTimeEiResult=$(( elapsedTimeEi + elapsedTimeEiResult ))
fi
done
if [ "$run" = true ]
then result=$( echo "scale=2; ($cpuTimeEiResult - $cpuTimeSiResult) / ($elapsedTimeEiResult - $elapsedTimeSiResult) * 100.0" | bc)
echo "RESULT:" $result
echo "cpuTimeSTART:" $cpuTimeSiResult
echo "elapsedTimeSTART:" $elapsedTimeSiResult
echo "cpuTimeEND:" $cpuTimeEiResult
echo "elapsedTimeEND:" $elapsedTimeEiResult
fi
sleep 1
if [ "$run" = false ]
then run=true
else
run=false
fi
done
Are there any ideas on how I could solve this better?
I am glad for any advice

Looking at your actual request to get the CPU utilization from a particular process; this could be done in less lines; e.g. below is the CPU usage after giving a seq command.
#using the process name
$ ps -C seq -o %cpu
%CPU
10.4
#using the process id
$ ps -p 5710 -o %cpu
%CPU
10.4

bash script to kill php process older then an hour

I have the following:
kill -9 `ps aux | grep php | awk '$9 !~ /[0-9]:[0-9]/' | awk '{print $2}'`
What it does is kill process that have been left abandoned by fcgid and kills them to free RAM. I want to run this as a cron every hour but would like to kill processes older then an hour. I'm just not sure how to modified the script to do that.

Try the following bash code :
for i in $(pidof php); do
pidtime=$(stat -c '%Y' /proc/$i)
now=$(date +%s)
((now - pidtime >= 3600)) && { kill $i; sleep 1; kill &>/dev/null -9 $i; }
done
and the crontab :
crontab -e
0 * * * * /path/to/the/script.bash

Solved with:
/bin/ps -Ao"command,pid,ppid"|/bin/grep ' 1$'|/bin/grep /php|/bin/awk '{ print $2; }'|/usr/bin/xargs --no-run-if-empty kill -9

Idle time of a process in Linux

I need to calculate CPU usage (user mode, system mode, idle time) of a process in Linux.
I am able to calculate usage in user and system mode using utime and stime values from /proc/PID/stat, but I found nothing which is related to idle time.
I know I can get idle time from /proc/stat but this value is related to machine, not for particular process.
Is it possible to calculate idle time of a process knowing its PID (reading data from /proc directory)?

I don't know much about it but maybe the following works:
1) Get the process start up time. Im sure thats possible
2) Generate time difference (dTime = CurrentTime - TimeProcessStarted)
3) Substract the time the process is running ( dTime - (usageSystemMode + usageUserMode))
Hope this helps! :D

it's too late but, I guessed this command useful:
IFS=$'\n';for i in `ps -eo uname:20,pid,cmd | grep -v "USER\|grep\|root"`; \
do if [ $(id -g `echo $i | cut -d" " -f1`) -gt 1000 ] && \
[ $(echo $((($(date +%s) - $(date -d "$(ll -u \
--time-style=+"%y-%m-%d %H:%M:%S" /proc/$(echo $i | \
awk '{print $2}')/cwd | awk '{print $6" "$7}')" +%s))/3600))) >=1 ]; \
then echo $i; fi; done
to use it in bash file:
#!/bin/bash
IFS=$'\n'
for i in `ps -eo uname:20,pid,cmd | grep -v "USER\|grep\|root"`
do
Name="`echo $i | cut -d' ' -f1`"
Id="$(id -g $Name)"
Pid="`echo $i | awk '{print $2}'`"
Time1=$(date +%s)
Time2=$(date -d "$(/usr/bin/ls -lu --time-style=+"%y-%m-%d %H:%M:%S" \
/proc/$Pid/cwd | awk '{print $6" "$7}')" +%s)/3600
Time=$Time1-$Time2
if [ $Id -gt 1000 ] && [ $Time >=1 ]
then
echo $i
fi
done
you could change grep -v "grep\|root" as you wish.
this one line command list all processes which not root owner or system users.

Resources