Here I'm accepting few mount points from the user and using each value to get space available on the host.
./user_input.ksh -string /m01,/m02,/m03
#!/bin/ksh
STR=$2
function showMounts {
echo "$STR"
arr=($(tr ',' ' ' <<< "$STR"))
printf "%s\n" "$(arr[#]}"
for x in "${arr[#]}"
do
free_space=`df -h "$x" | grep -v "Avail" | awk '{print $4}'`
echo "$x": free_space "$free_space"
done
#echo "$total_free_space"
}
Problems:
How can I exit for loop if any of the user input mount not avaialble?
currently it only add error in the log.
How to get total_free_space (i.e. sum of free_space)?
If you want to keep your code , test this (no ksh here). If you don't care, read Ed Morton's answer.
./user_input.ksh -string /m01,/m02,/m03
#!/bin/ksh
STR=$2
function showMounts {
echo "$STR"
arr=($(tr ',' ' ' <<< "$STR"))
printf "%s\n" "${arr[#]}"
for x in "${arr[#]}"; do
free_space=$(df -P "$x" | awk 'NR > 1 && !/Avail/{print $4}')
echo "$x: free_space $free_space"
((total_free_space+=$free_space))
done
echo "$((total_free_space/1024/1000))G"
}
showMounts
Caution:
"${arr[#]}"
not
"$(arr[#]}"
As I said in your last question, you do not need ANY of that, all you need is a one-liner like:
df -h "${STR//,/ }" | awk '/^ /{print $5, $3; sum+=$3} END{print sum}'
I have to say "like" because you haven't shown us the df -h /m01 /m02 /m03 output yet so I don't know exactly how to parse it.
Related
I've this data :
cat >data1.txt <<'EOF'
2020-01-27-06-00;/dev/hd1;100;/
2020-01-27-12-00;/dev/hd1;100;/
2020-01-27-18-00;/dev/hd1;100;/
2020-01-27-06-00;/dev/hd2;200;/usr
2020-01-27-12-00;/dev/hd2;200;/usr
2020-01-27-18-00;/dev/hd2;200;/usr
EOF
cat >data2.txt <<'EOF'
2020-02-27-06-00;/dev/hd1;120;/
2020-02-27-12-00;/dev/hd1;120;/
2020-02-27-18-00;/dev/hd1;120;/
2020-02-27-06-00;/dev/hd2;230;/usr
2020-02-27-12-00;/dev/hd2;230;/usr
2020-02-27-18-00;/dev/hd2;230;/usr
EOF
cat >data3.txt <<'EOF'
2020-03-27-06-00;/dev/hd1;130;/
2020-03-27-12-00;/dev/hd1;130;/
2020-03-27-18-00;/dev/hd1;130;/
2020-03-27-06-00;/dev/hd2;240;/usr
2020-03-27-12-00;/dev/hd2;240;/usr
2020-03-27-18-00;/dev/hd2;240;/usr
EOF
I would like to create a .txt file for each filesystem ( so hd1.txt, hd2.txt, hd3.txt and hd4.txt ) and put in each .txt file the sum of the value from each FS from each dataX.txt. I've some difficulties to explain in english what I want, so here an example of the result wanted
Expected content for the output file hd1.txt:
2020-01;/dev/hd1;300;/
2020-02;/dev/hd1;360;/
2020-03;/dev/hd1;390:/
Expected content for the file hd2.txt:
2020-01;/dev/hd2;600;/usr
2020-02;/dev/hd2;690;/usr
2020-03;/dev/hd2;720;/usr
The implementation I've currently tried:
for i in $(cat *.txt | awk -F';' '{print $2}' | cut -d '/' -f3| uniq)
do
cat *.txt | grep -w $i | awk -F';' -v date="$(cat *.txt | awk -F';' '{print $1}' | cut -d'-' -f-2 | uniq )" '{sum+=$3} END {print date";"$2";"sum}' >> $i
done
But it doesn't works...
Can you show me how to do that ?
Because the format seems to be so constant, you can delimit the input with multiple separators and parse it easily in awk:
awk -v FS='[;-/]' '
prev != $9 {
if (length(output)) {
print output >> fileoutput
}
prev = $9
sum = 0
}
{
sum += $9
output = sprintf("%s-%s;/%s/%s;%d;/%s", $1, $2, $7, $8, sum, $11)
fileoutput = $8 ".txt"
}
END {
print output >> fileoutput
}
' *.txt
Tested on repl generates:
+ cat hd1.txt
2020-01;/dev/hd1;300;/
2020-02;/dev/hd1;360;/
2020-03;/dev/hd1;390;/
+ cat hd2.txt
2020-01;/dev/hd2;600;/usr
2020-02;/dev/hd2;690;/usr
2020-03;/dev/hd2;720;/usr
Alternatively, you could -v FS=';' and use split to split first and second column to extract the year and month and the hdX number.
If you seek a bash solution, I suggest you invert the loops - first iterate over files, then over identifiers in second column.
for file in *.txt; do
prev=
output=
while IFS=';' read -r date dev num path; do
hd=$(basename "$dev")
if [[ "$hd" != "${prev:-}" ]]; then
if ((${#output})); then
printf "%s\n" "$output" >> "$fileoutput"
fi
sum=0
prev="$hd"
fi
sum=$((sum + num))
output=$(
printf "%s;%s;%d;%s" \
"$(cut -d'-' -f1-2 <<<"$date")" \
"$dev" "$sum" "$path"
)
fileoutput="${hd}.txt"
done < "$file"
printf "%s\n" "$output" >> "$fileoutput"
done
You could also almost translate awk to bash 1:1 by doing IFS='-;/' in while read loop.
I'm monitoring from an actively written to file:
My current solution is:
ws_trans=0
sc_trans=0
tail -F /var/log/file.log | \
while read LINE
echo $LINE | grep -q -e "enterpriseID:"
if [ $? = 0 ]
then
((ws_trans++))
fi
echo $LINE | grep -q -e "sc_ID:"
if [ $? = 0 ]
then
((sc_trans++))
fi
printf "\r WSTRANS: $ws_trans \t\t SCTRANS: $sc_trans"
done
However when attempting to do this with AWK I don't get the output - the $ws_trans and $sc_trans remains 0
ws_trans=0
sc_trans=0
tail -F /var/log/file.log | \
while read LINE
echo $LINE | awk '/enterpriseID:/ {++ws_trans} END {print | ws_trans}'
echo $LINE | awk '/sc_ID:/ {++sc_trans} END {print | sc_trans}'
printf "\r WSTRANS: $ws_trans \t\t SCTRANS: $sc_trans"
done
Attempting to do this to reduce load. I understand that AWK doesn't deal with bash variables, and it can get quite confusing, but the only reference I found is a non tail application of AWK.
How can I assign the AWK Variable to the bash ws_trans and sc_trans? Is there a better solution? (There are other search terms being monitored.)
You need to pass the variables using the option -v, for example:
$ var=0
$ printf %d\\n {1..10} | awk -v awk_var=${var} '{++awk_var} {print awk_var}'
To set the variable "back" you could use declare, for example:
$ declare $(printf %d\\n {1..10} | awk -v awk_var=${var} '{++awk_var} END {print "var=" awk_var}')
$ echo $var
$ 10
Your script could be rewritten like this:
ws_trans=0
sc_trans=0
tail -F /var/log/system.log |
while read LINE
do
declare $(echo $LINE | awk -v ws=${ws_trans} '/enterpriseID:/ {++ws} END {print "ws_trans="ws}')
declare $(echo $LINE | awk -v sc=${sc_trans} '/sc_ID:/ {++sc} END {print "sc_trans="sc}')
printf "\r WSTRANS: $ws_trans \t\t SCTRANS: $sc_trans"
done
I am facing some issue when I am reading the 3rd word(a hex string) of each line in a text file and compare it with a hex number. Can some one please help me on it.
#!/bin/bash
A=$1
cat $A | while read a; do
a1=$(echo \""$a"\" | awk '{ print $3 }')
#echo $a > cut -d " " -f 3
echo $a1
(("$a1" == 0x10F7))
echo $?
done
But when I use below, the comparison happens correctly,
a1= 0xADCAFE
(( "$a1" == 0x10F7 ))
echo $?
Then why it is showing issue when I read like below,
a1=$(echo \""$a"\" | awk '{ print $3 }')
or> a1=$(echo $a | awk '{ print $3 }')
echo $a prints intended hex value, but comparison does not happen.
Regards,
Running Awk inside a while read loop is an antipattern. Just do the loop in Awk; it's good at that.
awk '$3 == 4343' "$1"
If you want to compare against a string whose value is "0x10F7" then it's
awk '$3 == "0x10F7"' "$1"
If you want to match either, case insensitively etc, a regex is a good way to do that.
awk '$3 ~ /^(0x10[Ff]7|4343)$/' "$1"
Notice how the $1 in double quotes is handled by the shell, and gets replaced by a (properly quoted!) copy of the script's first command-line argument before Awk runs, while the Awk script in single quotes has its own namespace, so $3 is an Awk variable which refers to the third field in the current input line.
Either way, avoid the useless use of cat and always always always quote variables which contain file names with double quotes.
That's literal double quotes. You seem to have tried both a dangerous bare $a and a doubly double-quoted "\"$a\"" where the simple "$a" would be what you actually want.
Thank you all for your responses, Now my script is working fine. I was trying to match two files, below script does the purpose
#!/bin/bash
A=$1
B=$2
dos2unix -f "$A"
dos2unix -f "$B"
rm search_match.txt search_data_match.txt search_nomatch.txt search_data_nomatch.txt
while read line;do
search_word=$(echo $line | awk '{ print $1 }')
grep "$search_word" $B >> temp_file.txt
while read var;do
file1_hex=$(echo $line | awk '{ print $2 }')
file2_hex=$(echo $var | awk '{ print $3 }')
(("$file1_hex" == "$file2_hex"))
zero=$(echo $?)
if [ "$zero" -eq 0 ] ; then
echo $line >> search_match.txt
echo $var >> search_data_match.txt
else
echo $line >> search_nomatch.txt
echo $var >> search_data_nomatch.txt
fi
done < "temp_file.txt"
rm temp_file.txt
done < "$A"
My script is as below. When we run the script, it automatically saves the disk space usage in separate cells.
SIZES_1=`df -h | awk 'FNR == 1 {print $1","$2","$3","$4","$5","$6}'`
SIZES_2=`df -h | awk 'FNR == 2 {print $1","$2","$3","$4","$5","$6}'`
SIZES_3=`df -h | awk 'FNR == 3 {print $1","$2","$3","$4","$5","$6}'`
SIZES_4=`df -h | awk 'FNR == 4 {print $1","$2","$3","$4","$5","$6}'`
SIZES_5=`df -h | awk 'FNR == 5 {print $1","$2","$3","$4","$5","$6}'`
SIZES_6=`df -h | awk 'FNR == 6 {print $1","$2","$3","$4","$5","$6}'`
SIZES_7=`df -h | awk 'FNR == 7 {print $1","$2","$3","$4","$5","$6}'`
SIZES_8=`df -h | awk 'FNR == 8 {print $1","$2","$3","$4","$5","$6}'`
echo `date +%Z-%Y-%m-%d_%H-%M-%S` >>/home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_1" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_2" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_3" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_4" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_5" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_6" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_7" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_8" >> /home/jeevagan/test_scripts/sizes/excel.csv
This script is okay for my machine.
My doubt is, if somebody else's machine has many file systems, my script won't work to fetch all the file systems usage. How to make it to grab all those automatically?
Assuming you want all filesystems you can simplify that to:
printf '%s\n' "$(date +%Z-%Y-%m-%d_%H-%M-%S)" >> excel.csv
df -h | awk '{print $1","$2","$3","$4","$5","$6}' >> excel.csv
I would simplify this to
{ date +%Z-%F_%H-%M-%S; df -h | tr -s ' ' ','; } >> excel.csv
Group commands so only a single redirect is needed
Squeeze spaces and replace them with a single comma using tr
No need for echo `date` or similar: it's the same as just date
date +%Y-%m-%d is the same as date +%F
Notice that this has a little flaw in that the first line of the output of df -h, which looks something like this originally
Filesystem Size Used Avail Use% Mounted on
has a space in the heading of the last column, so it becomes
Filesystem,Size,Used,Avail,Use%,Mounted,on
with an extra comma. The original awk solution just cut off the last word of the line, though. Similarly, spaces in paths would trip up this solution.
To fix the comma problem, you could for example run
sed -i 's/Mounted,on$/Mounted on/' excel.csv
every now and so often.
As an aside, to replace all field separators in awk, instead of
awk '{print $1","$2","$3","$4","$5","$6}'
you can use
awk 'BEGIN { OFS = "," } { $1 = $1; print }'
or, shorter,
awk -v OFS=',' '{$1=$1}1'
how to retrive every portion separately from following file name? DSA4020_frontcover_20346501_2011-05.doc
I want to retrieve informations as below;
name = DSA4020
type = frontcover
id = 20346501
date = 2011-05
is it possible to do with sed??
Yes, you can:
pax$ echo 'DSA4020_frontcover_20346501_2011-05.doc' | sed
-e 's/^/name=/'
-e 's/_/\ntype=/'
-e 's/_/\nid=/'
-e 's/_/\ndate=/'
-e 's/\..*//'
name=DSA4020
type=frontcover
id=20346501
date=2011-05
That's all on one line, I've just split it for readability.
You could also do it with awk if you wish:
pax$ echo 'DSA4020_frontcover_20346501_2011-05.doc'
| awk -F_ '{print "name="$1"\ntype="$2"\nid="$3"\ndate="substr($4,1,7)}'
name=DSA4020
type=frontcover
id=20346501
date=2011-05
awk may be a better choice
# f=DSA4020_frontcover_20346501_2011-05.doc
# name=$(echo $f | awk -F_ '{print $1}')
# echo $name
DSA4020
# type=$(echo $f | awk -F_ '{print $2}')
# echo $type
frontcover
In pure bash
FILE="DSA4020_frontcover_20346501_2011-05.doc"
eval $(echo $FILE |(IFS="_";read a b c d; echo "name=$a;type=$b;id=$c;date=${d%.doc}"))
echo Name:$name Type:$type ID:$id DATE:$date