How to write the names and number of processes to standard output on Linux terminal where only users with at least 10 running processes are shown?
ps -eo user,cmd | awk '{ usr[$1]+=1;prc[$1][$2]="" } END { for (i in usr) { if (usr[i]>=10) { print i" - "usr[i];for (j in prc[i]) { print i" - "j } } } }'
This can be achieved by using the -o option of ps to remove any "noise" and then piping the output through to awk.
Track the number of processes per user by creating an array usr and track the processes for each user by creating an 2 dimensional array prc. At the end, loop through the arrays, printing the count for each user where the process count is greater or equal to 10 followed by the actual processes.
Related
I am trying to write a bash script to create multiple .txt files.
With the below code I created the files, but when I run the script again I get the same output instead of having more files with increasing number.
#! /bin/bash
for z in $(seq -w 1 10);
do
[[ ! -f "${z}_name.txt" ]] && {touch "${z}_name.txt";}
done
Based in part on work by Raman Sailopal in a now-deleted answer (and on comments I made about that answer, as well as comments I made about the question), you could use:
shopt -s nullglob
touch $(seq -f '%.0f_name.txt' \
$(printf '%s\n' [0-9]*_name.txt |
awk 'BEGIN { max = 0 }
{ val = $0 + 0; if (val > max) max = val; }
END { print max + 1, max + 10 }'
)
)
The shopt -s nullglob command means that if there are no names that match the glob expression [0-9]*_name.txt, nothing will be generated in the arguments to the printf command.
The touch command is given a list of file names. The seq command formats a range of numbers using zero decimal places (so it formats them as integers) plus the rest of the name (_name.txt). The range is given by the output of printf … | awk …. The printf() command lists file names that start with a digit and end with _name.txt one per line. The awk command keeps a track of the current maximum number; it coerces the name into a number (awk ignores the material after the last digit) and checks whether the number is larger than before. At the end, it prints two values, the largest value plus 1 and the largest value plus 10 (defaulting to 1 and 10 if there were no files). Adding the -w option to seq is irrelevant when you specify -f and a format; the file names won't be generated with leading zeros. There are ways to deal with this if they're crucial — probably simplest is to drop the -f option to seq and add the -w option, and output the output through sed 's/$/_name.txt/'.
You can squish the awk script onto a single line; you can squish the whole command onto a single line. However, it is arguably easier to see the organization of the command when they are spread over multiple lines.
Note that (apart from a possible TOCTOU — Time of Check, Time of Use — issue), there is no need to check whether the files exist. They don't; they'd have been listed by the glob [0-9]*_name.txt if they did, and the number would have been accounted for. If you want to ensure no damage to existing files, you'd need to use set -C or set -o noclobber and then create the files one by one using shell I/O redirection.
[…time passes…]
Actually, you can have awk do the file name generation instead of using seq at all:
touch $(printf '%s\n' [0-9]*_name.txt |
awk 'BEGIN { max = 0 }
{ val = $0 + 0; if (val > max) max = val; }
END { for (i = max + 1; i <= max + 10; i++)
printf "%d_name.txt\n", i
}'
)
And, if you try a bit harder, you can get rid of the printf command too:
touch $(awk 'BEGIN { max = 0
for (i = 1; i <= ARGC; i++)
{
val = ARGV[i] + 0;
if (val > max)
max = val
}
for (i = max + 1; i <= max + 10; i++)
printf "%d_name.txt\n", i
}' [0-9]*_name.txt
)
Don't forget the shopt -s nullglob — that's still needed for maximum resiliency.
You might even choose to get rid of the separate touch command by having awk write to the files:
awk 'BEGIN { max = 0
for (i = 0; i < ARGC; i++)
{
val = ARGV[i] + 0;
if (val > max)
max = val
}
for (i = max + 1; i <= max + 10; i++)
{
name = sprintf("%d_name.txt", i)
printf "" > name
}
exit
}' [0-9]*_name.txt
Note the use of exit. Note that the POSIX specification for awk says that ARGC is the number of arguments in ARGV and that the elements in ARGV are indexed from 0 to ARGC - 1 — as in C programs.
There are few shell scripts that cannot be improved. The first version shown runs 4 commands; the last runs just one. That difference could be quite significant if there were many files to be processed.
Beware: eventually, the argument list generated by the glob will get too big; then you have to do more work. You might be obliged to filter the output from ls (with its attendant risks and dangers) and feed the output (the list of file names) into the awk script and process the lines of input once more. While your lists remain a few thousand files long, it probably won't be a problem.
$ lsisolcpus
core pid foreign comm cmdline
30 27213 2147 Test1 ./Test1
30 27214 2157 Test2 ./Test2
I use lsisolcpus to find cores used by different processes.
Question> Is there a way that I can automatically detect when more than one process is pinned to the same cpu core? In above example, I would like to see the script complains that there is a conflict because both Test1 and Test2 are pinned to the core 30.
How about:
lsisolcpus | awk 'NR==1 {next} { cpus[$1][$4]="" } END { for ( i in cpus) { cnt=0;for (n in cpus[i]) {cnt++} if (cnt>1) { print "WARNING - conflict on core - "i;for (o in cpus[i]) { print o } } } }'
Taking the output of the lsisolcpus command and feeding it into awk. We ignore the first line of the output (NR==1) and then build a 2 dimensional array of the output (cpus) indexed by core and command. We then loop through the array and then if there is more than one entry for each core (cnt>1), we print an error message and the commands tied to that core.
I need bash script to count processes of SPECIFIC users or all users. We can enter 0, 1 or more arguments. For example
./myScript.sh root deamon
should execute like this:
root 92
deamon 8
2 users has total processes: 100
If nothing is entered as parameter, then all users should be listed:
uuidd 1
awkd 2
daemon 1
root 210
kklmn 6
5 users has total processes: 220
What I have till now is script for all users, and it works fine (with some warnings). I just need part where arguments are entered (some kind of filter results). Here is script for all users:
cntp = 0 #process counter
cntu = 0 #user counter
ps aux |
awk 'NR>1{tot[$1]++; cntp++}
END{for(id in tot){printf "%s\t%4d\n",id,tot[id]; cntu++}
printf "%4d users has total processes:%4d\n", cntu, cntp}'
#!/bin/bash
users=$#
args=()
if [ $# -eq 0 ]; then
# all processes
args+=(ax)
else
# user processes, comma-separated list of users
args+=(-u${users// /,})
fi
# print the user field without header
args+=(-ouser=)
ps "${args[#]}" | awk '
{ tot[$1]++ }
END{ for(id in tot){ printf "%s\t%4d\n", id, tot[id]; cntu++ }
printf "%4d users has total processes:%4d\n", cntu, NR}'
The ps arguments are stored in array args and list either all processes with ax or user processes in the form -uuser1,user2
and -ouser= only lists the user field without header.
In the awk script I only removed the NR>1 test and variable cntp which can be replaced by NR.
Possible invocations:
./myScript.sh
./myScript.sh root daemon
./myScript.sh root,daemon
The following seems to work:
ps axo user |
awk -v args="$(IFS=,; echo "$*")" '
BEGIN {
# split args on comma
split(args, users, ",");
# associative array with user as indexes
for (i in users) {
enabled[users[i]] = 1
}
}
NR > 1 {
tot[$1]++;
cntp++;
}
END {
for(id in tot) {
# if we passed some arguments
# and its disabled
if (length(args) && enabled[id] == 0) {
continue
}
printf "%s\t%4d\n", id, tot[id];
cntu++;
}
printf "%4d users has total processes:%4d\n", cntu, cntp
}
'
Tested in repl.
In a bash script I need to wait until CPU usage gets below a threshold.
In other words, I'd need a command wait_until_cpu_low which I would use like this:
# Trigger some background CPU-heavy command
wait_until_cpu_low 40
# Some other commands executed when CPU usage is below 40%
How could I do that?
Edit:
target OS is: Red Hat Enterprise Linux Server release 6.5
I'm considering the average CPU usage (across all cores)
A much more efficient version just calls mpstat and awk once each, and keeps them both running until done; no need to explicitly sleep and restart both processes every second (which, on an embedded platform, could add up to measurable overhead):
wait_until_cpu_low() {
awk -v target="$1" '
$13 ~ /^[0-9.]+$/ {
current = 100 - $13
if(current <= target) { exit(0); }
}' < <(LC_ALL=C mpstat 1)
}
I'm using $13 here because that's where idle % is for my version of mpstat; substitute appropriately if yours differs.
This has the extra advantage of doing floating point math correctly, rather than needing to round to integers for shell-native math.
wait_for_cpu_usage()
{
current=$(mpstat 1 1 | awk '$12 ~ /[0-9.]+/ { print int(100 - $12 + 0.5) }')
while [[ "$current" -ge "$1" ]]; do
current=$(mpstat 1 1 | awk '$12 ~ /[0-9.]+/ { print int(100 - $12 + 0.5) }')
sleep 1
done
}
Notice it requires sysstat package installed.
You might use a function based on the top utility. But note, that doing so is not very reliable because the CPU utilization might - rapidly - change at any time. Meaning that just because the check succeeded, it is not guaranteed that the CPU utilization will stay low as long the following code runs. You have been warned.
The function:
function wait_for_cpu_usage {
threshold=$1
while true ; do
# Get the current CPU usage
usage=$(top -n1 | awk 'NR==3{print $2}' | tr ',' '.')
# Compared the current usage against the threshold
result=$(bc -l <<< "$usage <= $threshold")
[ $result == "1" ] && break
# Feel free to sleep less than a second. (with GNU sleep)
sleep 1
done
return 0
}
# Example call
wait_for_cpu_usage 25
Note that I'm using bc -l for the comparison since top prints the CPU utilization as a float value.
As noted by "Mr. Llama" in a comment above, I've used uptime to write my simple function:
function wait_cpu_low() {
threshold=$1
while true; do
current=$(uptime | awk '{ gsub(/,/, ""); print $10 * 100; }')
if [ $current -lt $threshold ]; then
break;
else
sleep 5
fi
done
}
In awk expression:
$10 is to get average CPU usage in last minute
$11 is to get average CPU usage in last 5 minutes
$12 is to get average CPU usage in last 15 minutes
And here is an usage example:
wait_cpu_low 20
It waits one minute average CPU usage is below 20% of one core of CPU.
If I look at /proc/6945/stat then I get a series of numbers, one of which is the number of CPU-centiseconds for which the process has been running.
But I'm running these processes on heavily-loaded boxes, and what I'm interested in is the clock-time when the job will finish, for which I want to know the clock-time that it started.
The timestamps on files in /proc/6945 look to be in the right sort of range but I can't find a particular file which consistently has the right clock-time on it.
As always I can't modify the process.
Timestamps of the directories in /proc are useless.
I was advised to look at 'man proc'; this says that /proc/$PID/stat field 21 records the start-time of the process in kernel jiffies since boot ... so:
open A,"< /proc/stat"; while (<A>) { if (/^btime ([0-9]*)/) { $btime = $1 } }
to obtain the boot time, then
my #statl = split " ",`cat /proc/$i/stat`;
$starttime_jiffies = $statl[21];
$starttime_ut = $btime + $starttime_jiffies / $jiffies_per_second;
$cputime = time-$starttime_ut
but I set $jiffies_per_second to 100 because I don't know how to ask the kernel for its value from perl.
I have a project on github that does this in perl. You can find it here:
https://github.com/cormander/psj
The code you're wanting is in lib/Proc/PID.pm, and here is the snippit (with comments removed):
use POSIX qw(ceil sysconf _SC_CLK_TCK);
sub _start_time {
my $pid = shift->pid;
my $tickspersec = sysconf(_SC_CLK_TCK);
my ($secs_since_boot) = split /\./, file_read("/proc/uptime");
$secs_since_boot *= $tickspersec;
my $start_time = (split / /, file_read("/proc/$pid/stat"))[21];
return ceil(time() - (($secs_since_boot - $start_time) / $tickspersec));
}
Beware the non-standard code function file_read in here, but that should be pretty straight forward.
Use the creation timestamp of the /proc/6945 directory (or whatever PID), rather than looking at the files it contains. For example:
ls -ld /proc/6945
Bash command to get the start date of some process:
date -d #$(cat /proc/PID/stat | awk "{printf \"%.0f\", $(grep btime /proc/stat | cut -d ' ' -f 2)+\$22/$(getconf CLK_TCK);}")