I am new in bash
I wrote a bash script and it gives me an output like this:
3387 /test/file1
23688 /test/file2
5813 /test/file3
10415 /test/file4
1304 /test/file5
46 /test/file6
8 /test/file7
138 /test/file8
I can delimit them by
wc -l /path/to/$dir/test | cut -d" " -f1
how can I add numbers to eachother and caculate them?
can I do:
output=`wc -l /path/to/$dir/test | cut -d" " -f1`
Is it possible to use "while" or "for" loop and add those numbers?
how?
thank you in advance
You want awk here to avoid explicit loops. If your output was in the file data.txt you could use:
$ awk '{sum += $1} END {print sum}' data.txt
44799
In your case, pipe the output of your script to awk:
$ your_script.sh | awk '{sum += $1} END {print sum}'
Since the output you gave in your question was the output of wc -l, try:
$ wc -l /path/to/$dir/test | awk '{sum += $1} END {print sum}'
(Aside for anyone else landing on this page: wc -l, when given wildcards, will also give you a total, but it's great to use awk in this case because you can deal directly with the total line count and pipe just that to another process.)
Related
i am trying to let this command work but it won't let me do anything
awk -F: ‘{if($3>'1000') print$1}’ passwd | sort > users.txt
I get an error which is saying:
bash: syntaxfout nabij onverwacht symbool '('
Can someone help me out?
You're using ‘ instead of '. And then, you should replace ' with " in the awk program (or just leave them out):
awk -F: '{if ($3 > 1000) print $1}' passw | ...
You're using backticks instead of single quotes. Try:
awk -F: '{if($3>1000) print $1} passwd | sort > users.txt
or just
awk -F: '$3>1000 {print $1}' passwd | sort > users.txt
I have data in file.txt as follows
BRAD CHICAGO|NORTH SAMSONCHESTER|
CORA|NEW ERICA|
CAMP LOGAN|KINGBERG|
NCHICAGOS|ESTING|
CHICAGO|MANKING|
OCREAN|CHICAGO|
CHICAGO PIT|BULL|
CHICAGO |NEWYORK|
Question 1:
I want to search for the exact match for word "CHICAGO" in first column and print second column.
Output should look like:
MANKING
NEWYORK
Question 2:
If multiple matches found then can we limit the out to only one ? so that the output will be only MANKING or NEWYORK
I tried below
grep -E -i "^CHICAGO" file.txt | awk -F '|' '{print $2}'
but i am getting below output
MANKING
BULL
NEWYORK
Expected output for Question 1:
MANKING
NEWYORK
Expected output for Question 2:
MANKING
Here are some more ways:
Using grep and cut:
grep "^CHICAGO|" file.txt | cut -d'|' -f2
Using awk
awk -F"|" '/^CHICAGO\|/{print $2}' file.txt
For question 2 simply pipe it to head, i.e:
grep "^CHICAGO|" file.txt | cut -d'|' -f2 | head -n1
Similarly for the awk command.
how about an awk solution?
awk -F'|' '$1 == "CHICAGO"{print $2}' file
to only print one output, exit once you have a match, i.e.
awk -F'|' '$1 == "CHICAGO"{print $2; exit}' file
Making that more generic, you can pass in a variable, i.e.
awk -v trgt="CHICAGO" -F'|' '{targ="^" trgt " *$"; if ( $1 ~ targ ) {print $2}}' file
The " *$" regex limits the match to zero or more trailing spaces without any extra chars at the end of the target string. So this will meet your criteria to match skip matching CHICAGO PIT|BULL.
AND this can be further reduced to
awk -v trgt="CHICAGO" -F'|' '{ if ( $1 ~ "^" trgt " *$" ) {print $2}}' file
constructing the regex "in-place" in with the comparison.
So you could use more verbose variable names to "describe" how the regex is being constructed from the input and the regex "wrappers" (as in the 3rd example) OR, you can just combine the input variable with the regex syntax in place. That is just a matter of taste or documentation conventions.
You might want to include a comment to explain you are constructing a regex test that would look like the $1 ~ /^CHICAGO *$/.
IHTH
I have the following script:
#!/bin/bash
TotalMem=$(top -n 1 | grep Mem | awk 'NR==1{print $4}') #integer
UsadoMem=$(top -n 1 | grep Mem | awk 'NR==1{print $8}') #integer
PorcUsado='scale=2;UsadoMem/TotalMem'|bc -l
echo $PorcUsado
The variable PorcUsado returns empty. I search for the use of bc, but something is wrong...
You're assigning PorcUsado to scale=2;UsadoMem/TotalMem and then piping the output of that assignment (nothing) into bc. You probably want the pipe inside a command substitution, e.g. (using a here string instead of a pipe):
PorcUsado=$(bc -l <<<'scale=2;UsadoMem/TotalMem')
But you'll also need to evaluate those shell variables - bc can't do it for you:
PorcUsado=$(bc -l <<<"scale=2;$UsadoMem/$TotalMem")
Notice the use of " instead of ' and the $ prefix to allow Bash to evaluate the variables.
Also, if this is the whole script, you can just skip the PorcUsado variable at all and let bc write directly to stdout.
#!/bin/bash
TotalMem=$(top -n 1 | grep Mem | awk 'NR==1{print $4}') #integer
UsadoMem=$(top -n 1 | grep Mem | awk 'NR==1{print $8}') #integer
bc -l <<<"scale=2;$UsadoMem/$TotalMem"
Why pipe top output at all? Seems too costly.
$ read used buffers < <(
awk -F':? +' '
{a[$1]=$2}
END {printf "%d %d", a["MemTotal"]-a["MemFree"], a["Buffers"]}
' /proc/meminfo
)
Of course, it can easily be a one-liner if you value brevity over readability.
I think the pipe is the problem try something like this:
PorcUsado=$(echo "scale=2;$UsadoMem/$TotalMem" | bc -l)
i haven't tested it yet but you have to echo the string and pipe the result from echo to bc.
EDIT: Correcting the variable names
You don't need grep or bc, since awk can search and do math all by itself:
top -n 1 -l 1 | awk '/Mem/ {printf "%0.2f\n",$8/$4;exit}'
I have a line in a bash script that calculates the sum of unique IP requests to a certain page.
grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}' | sort | uniq -c | awk '{sum += 1; print } END { print " ", sum, "total"}'
I am trying to get the value of sum to a variable outside the awk statement so I can compare pages to each other. So far I have tried various combinations of something like this:
unique_sum=0
grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}' | sort | uniq -c | awk '{sum += 1; print ; $unique_sum=sum} END { print " ", sum, "total"}'
echo "${unique_sum}"
This results in an echo of "0". I've tried placing __$unique_sum=sum__ in the END, various combinations of initializing the variable (awk -v unique_sum=0 ...) and placing the variable assignment outside of the quoted sections.
So far, my Google-fu is failing horribly as most people just send the whole of the output to a variable. In this example, many lines are printed (one for each IP) in addition to the total. Failing a way to capture the 'sum' variable, is there a way to capture that last line of output?
This is probably one of the most sophisticated things I've tried in awk so my confidence that I've done anything useful is pretty low. Any help will be greatly appreciated!
You can't assign a shell variable inside an awk program. In general, no child process can alter the environment of its parent. You have to have the awk program print out the calculated value, and then shell can grab that value and assign it to a variable:
output=$( grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}' | sort | uniq -c | awk '{sum += 1; print } END {print sum}' )
unique_sum=$( sed -n '$p' <<< "$output" ) # grab the last line of the output
sed '$d' <<< "$output" # print the output except for the last line
echo " $unique_sum total"
That pipeline can be simplified quite a lot: awk can do what grep can do, so first
grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}'
is (longer, but only one process)
awk -F" - " -v date="$YESTERDAY" -v patt="$1" '$0 ~ date && $0 ~ patt {print $1}' "$ACCESSLOG"
And the last awk program just counts how many lines and can be replaced with wc -l
All together:
unique_output=$(
awk -F" - " -v date="$YESTERDAY" -v patt="$1" '
$0 ~ date && $0 ~ patt {print $1}
' "$ACCESSLOG" | sort | uniq -c
)
echo "$unique_output"
unique_sum=$( wc -l <<< "$unique_output" )
echo " $unique_sum total"
I want to grep a file and extract the third part of this line
#define SIM_VERSION_COMPAT 1302
with awk. So I wrote:
grep "#define SIM_VERSION_COMPAT" global.h | awk '{ print $$3 }'
The result should be 1302 but I get nothing (blank).
No need to use grep and pipe you can use awk like this:
awk '/#define SIM_VERSION_COMPAT/{print $3}' global.h
[spatel#tc01 ~]$ echo "#define SIM_VERSION_COMPAT 1302" | awk '{ print $3 }'
1302
Just using grep:
$ grep -Po '(?<=#define SIM_VERSION_COMPAT )[0-9]+' global.h
1302
This uses positive lookbehind to match lines containing #define SIM_VERSION_COMPAT but only prints the digit string following.
You can also use cut command as well
grep "#define SIM_VERSION_COMPAT" temp.txt | cut -d" " -f 3