grep -A 26 "some text" somefile.txt |
awk '/other text/ { gsub(/M/, " "); print $4 }' |
sort -n -r | uniq | head -1
will return the largest in a list pulled from a large text file, but how do I store the output as a variable?
Use command substitution:
my_var=$(grep -A 26 "some text" somefile.txt |
awk '/other text/ { gsub(/M/, " "); print $4 }' |
sort -n -r | uniq | head -n1)
Also, for portability, I would suggest always using -n1 for the argument of head. I've come across a couple of incarnations of it where using -1 doesn't work.
For unnested cases back quotes will work too:
variable=`grep -A 26 "some text" somefile.txt |
awk '/other text/ { gsub(/M/, " "); print $4 }' |
sort -nru | head -1`
I'd suggest
variable_name=$(grep -A 26 "some text" somefile.txt |
awk '/other text/ { gsub(/M/, " "); print $4 }' |
sort -nru | head -1)
Related
Using bash, I want to print a number followed by sizes of 2 paths on one line. i.e. output of 3 commands on one line.
All the 3 items should be separated by ":"
echo -n "10001:"; du -sch /abc/def/* | grep 'total' | awk '{ print $1 }'; du -sch /ghi/jkl/* | grep 'total' | awk '{ print $1 }'
I am getting the output as -
10001:61M
:101M
But I want the output as -
10001:61M:101M
This should work for you. The two key elements added being the
tr - d '\n'
which effectively strips new line characters from the end of the output. As well as adding in the echo ":" to get the extra colon for formatting in there.
Hope this helps! Here's a link to the docs for tr command.
https://ss64.com/bash/tr.html
echo -n "10001:"; du -sch /abc/def/* | grep 'total' | awk '{ print $1 }' | tr -d '\n'; echo ":" | tr -d '\n'; du -sch /ghi/jkl/* | grep 'total' | awk '{ print $1 }'
Save your values to variables, and then use printf:
printf '%s:%s:%s\n' "$first" "$second" "$third"
I am using ssh in a shell script in order to go on multiple linux server and get disk information on a particular disk. I am running following but I am not able to figure out the quote sequencing...In this example I am just capturing the header for my report....
ssh dbadmin#myserver bash -c '"df -kh | grep File | awk '{ print \$1 " | " \$2 " | " \$3 " | " \$4 " | " \$5 }' | tail -n -1"'
and following error...
bash: -c: line 0: syntax error near unexpected token |'
bash: -c: line 0:df -kh | grep File | awk { print | | | | } | tail -n -1'
Any help or suggestions would be great...
Thanks
Better to use quoted here-doc and avoid escaping:
ssh -t -t dbadmin#myserver<<'EOF'
df -kh | awk -v OFS=" | " '/file/{ print $1, $2, $3, $4, $5 }' | tail -n -1
EOF
I have a line in a bash script that calculates the sum of unique IP requests to a certain page.
grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}' | sort | uniq -c | awk '{sum += 1; print } END { print " ", sum, "total"}'
I am trying to get the value of sum to a variable outside the awk statement so I can compare pages to each other. So far I have tried various combinations of something like this:
unique_sum=0
grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}' | sort | uniq -c | awk '{sum += 1; print ; $unique_sum=sum} END { print " ", sum, "total"}'
echo "${unique_sum}"
This results in an echo of "0". I've tried placing __$unique_sum=sum__ in the END, various combinations of initializing the variable (awk -v unique_sum=0 ...) and placing the variable assignment outside of the quoted sections.
So far, my Google-fu is failing horribly as most people just send the whole of the output to a variable. In this example, many lines are printed (one for each IP) in addition to the total. Failing a way to capture the 'sum' variable, is there a way to capture that last line of output?
This is probably one of the most sophisticated things I've tried in awk so my confidence that I've done anything useful is pretty low. Any help will be greatly appreciated!
You can't assign a shell variable inside an awk program. In general, no child process can alter the environment of its parent. You have to have the awk program print out the calculated value, and then shell can grab that value and assign it to a variable:
output=$( grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}' | sort | uniq -c | awk '{sum += 1; print } END {print sum}' )
unique_sum=$( sed -n '$p' <<< "$output" ) # grab the last line of the output
sed '$d' <<< "$output" # print the output except for the last line
echo " $unique_sum total"
That pipeline can be simplified quite a lot: awk can do what grep can do, so first
grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}'
is (longer, but only one process)
awk -F" - " -v date="$YESTERDAY" -v patt="$1" '$0 ~ date && $0 ~ patt {print $1}' "$ACCESSLOG"
And the last awk program just counts how many lines and can be replaced with wc -l
All together:
unique_output=$(
awk -F" - " -v date="$YESTERDAY" -v patt="$1" '
$0 ~ date && $0 ~ patt {print $1}
' "$ACCESSLOG" | sort | uniq -c
)
echo "$unique_output"
unique_sum=$( wc -l <<< "$unique_output" )
echo " $unique_sum total"
I'm trying to make part of the output of the first command as another command's argument.
Output of first command is like this, and 3000 is what I want:
XXXXXXXXXXXXX
abcdefg 1020 10:30
[1000] 3000
I extract the pattern by ./command1 | grep '^\[' | awk 'print $2', so it will print out 3000, the value I want.
I'd like to make 3000 as an argument of command2 ./command2 3000. How do I make this work?
command2 $( command1 | awk '/\[/{ print $2 }' )
You can use xargs to pass the input to a new command. In your example you need to include curly braces in your awk argument as well.
./command1 | grep '^\[' | awk '{ print $2 } ' | xargs ./command2
Or more concisely
.command1 | awk '/^\[/ { print $2 }' | xargs ./command2
Example:
echo "[1000] 3000" | awk '/^\[/ { print $2 }' | xargs echo
Output:
3000
There's also sed:
./command1 | sed -n 'n;n;p' | awk '{print $2}'
All together now:
./command2 $(./command1 | sed -n 'n;n;p' | awk '{print $2}') # ./command2 3000
sed will skip 2 lines and print the third.
I would personally try backticks first:
./command2 `./command1 | grep '^\[' | awk 'print $2'`
How do I sort a "python dictionary-style" array e.g. ( "A: 2" "B: 3" "C: 1" ) in bash by the value? I think, this code snippet will make it bit more clear about my question.
State="Total 4 0 1 1 2 0 0"
W=$(echo $State | awk '{print $3}')
C=$(echo $State | awk '{print $4}')
U=$(echo $State | awk '{print $5}')
M=$(echo $State | awk '{print $6}')
WCUM=( "Owner: $W;" "Claimed: $C;" "Unclaimed: $U;" "Matched: $M" )
echo ${WCUM[#]}
This will simply print the array: Owner: 0; Claimed: 1; Unclaimed: 1; Matched: 2
How do I sort the array (or the output), eliminating any pair with "0" value, so that the result like this:
Matched: 2; Claimed: 1; Unclaimed: 1
Thanks in advance for any help or suggestions. Cheers!!
Quick and dirty idea would be (this just sorts the output, not the array):
echo ${WCUM[#]} | sed -e 's/; /;\n/g' | awk -F: '!/ 0;?/ {print $0}' | sort -t: -k 2 -r | xargs
echo -e ${WCUM[#]} | tr ';' '\n' | sort -r -k2 | egrep -v ": 0$"
Sorting and filtering are independent steps, so if you only like to filter 0 values, it would be much more easy.
Append an
| tr '\n' ';'
to get it to a single line again in the end.
nonull=$(for n in ${!WCUM[#]}; do echo ${WCUM[n]} | egrep -v ": 0;"; done | tr -d "\n")
I don't see a good reason to end $W $C $U with a semicolon, but $M not, so instead of adapting my code to this distinction I would eliminate this special case. If not possible, I would append a semicolon temporary to $M and remove it in the end.
Another attempt, using some of the bash features, but still needs sort, that is crucial:
#! /bin/bash
State="Total 4 1 0 4 2 0 0"
string=$State
for i in 1 2 ; do # remove unnecessary fields
string=${string#* }
string=${string% *}
done
# Insert labels
string=Owner:${string/ /;Claimed:}
string=${string/ /;Unclaimed:}
string=${string/ /;Matched:}
# Remove zeros
string=(${string[#]//;/; })
string=(${string[#]/*:0;/})
string=${string[#]}
# Format
string=${string//;/$'\n'}
string=${string//:/: }
# Sort
string=$(sort -t: -nk2 <<< "$string")
string=${string//$'\n'/;}
echo "$string"