I have this file that contains number in a column:
[root#server1]# cat numbers.txt
30
25
15
I need to add them together so I do this:
[root#autonoc cattests]# cat numbers.txt | paste -s -d+ | bc
70
But after I add them together I need to divide them by 60,something like this:
[root#server1]# cat numbers.txt | paste -s -d+ | "new command" | bc
how can I do that?
Using awk:
$ awk '{s+=$1} END{print s/60}' numbers.txt
1.16667
How it works
s+=$1
The number on each lien of numbers.txt is added to variable s.
END{print s/60}
After we have finished reading the file, we print the value of s divided by 60.
bc -l <<< '('$(paste -s -d+ numbers.txt)')/60'
awk -v div=60 '{tot+=$0}END{print tot/div}' numbers.txt
Using the -v div=60 can be extended further to accept any user input, like
read -p "Enter the div value: " div
awk -v div="$div" ...
IHTH
You can use dc
dc -f numbers.txt -e '++3k60/[result = ]np'
Related
i have a shell script
#/bin/bash
var1=`cat log.json | grep "accountnumber" | awk -F ' ' '{print $1}'`
echo $var
output of shell script is :-
23466
283483
324932
87374
I want match the above number which is already store in another file (below is the file format ) and print its value .
23466=account-1
283483=account-2
324932=account-3
87374=account-4
127632=account-5
1324237=account-6
73642=account-7
324993284=account-8
.
.
4543454=account-200
exapected output
account-1
account-2
account-3
account-4
a Compact one line solution can be:
join -t "=" <(sort bf) <(sort fa) | cut -d '=' -f 2
here fa is a file containing out-put of your bash script and bf is the file that has 23466=account-1 format
the output is:
account-1
account-2
account-3
account-4
#!/bin/bash
for var1 in $(awk -F ' ' '/accountnumber/{print $1}' log.json)
do
awk -F= '$1=="'"$var1"'"{print $2}' anotherfile
done
For a moment there was another answer that almost worked that I think is much slicker than what I wrote. Probably faster / more efficient on large files too. Here it is fixed.
awk -F ' ' '/accountnumber/{print $1}' log.json \
| sort -n \
| join -t= - accountfile \
| cut -d= -f2
I have the following script:
#!/bin/bash
TotalMem=$(top -n 1 | grep Mem | awk 'NR==1{print $4}') #integer
UsadoMem=$(top -n 1 | grep Mem | awk 'NR==1{print $8}') #integer
PorcUsado='scale=2;UsadoMem/TotalMem'|bc -l
echo $PorcUsado
The variable PorcUsado returns empty. I search for the use of bc, but something is wrong...
You're assigning PorcUsado to scale=2;UsadoMem/TotalMem and then piping the output of that assignment (nothing) into bc. You probably want the pipe inside a command substitution, e.g. (using a here string instead of a pipe):
PorcUsado=$(bc -l <<<'scale=2;UsadoMem/TotalMem')
But you'll also need to evaluate those shell variables - bc can't do it for you:
PorcUsado=$(bc -l <<<"scale=2;$UsadoMem/$TotalMem")
Notice the use of " instead of ' and the $ prefix to allow Bash to evaluate the variables.
Also, if this is the whole script, you can just skip the PorcUsado variable at all and let bc write directly to stdout.
#!/bin/bash
TotalMem=$(top -n 1 | grep Mem | awk 'NR==1{print $4}') #integer
UsadoMem=$(top -n 1 | grep Mem | awk 'NR==1{print $8}') #integer
bc -l <<<"scale=2;$UsadoMem/$TotalMem"
Why pipe top output at all? Seems too costly.
$ read used buffers < <(
awk -F':? +' '
{a[$1]=$2}
END {printf "%d %d", a["MemTotal"]-a["MemFree"], a["Buffers"]}
' /proc/meminfo
)
Of course, it can easily be a one-liner if you value brevity over readability.
I think the pipe is the problem try something like this:
PorcUsado=$(echo "scale=2;$UsadoMem/$TotalMem" | bc -l)
i haven't tested it yet but you have to echo the string and pipe the result from echo to bc.
EDIT: Correcting the variable names
You don't need grep or bc, since awk can search and do math all by itself:
top -n 1 -l 1 | awk '/Mem/ {printf "%0.2f\n",$8/$4;exit}'
I have a string id=12345&data=23456
I want to print 12345 23456
Currently I just know how to split one of them separately by awk
echo id=12345&data=23456 | awk -F"id=" '{print substr($2,1,5)}'
and it's similar for data.
How can I combine those awk command to get the desired result?
regex groups can be one solution but awk can't handle regex groups but, gawk can.
Example
echo "id=12345&data=23456" | gawk 'match($0, /^id=([^&]*)&data=(.*)$/, groups) {print groups[1] " " groups[2]}'
Output
12345 23456
There's no need for external processes. You can use the builtin read to extract the two numbers:
$ IFS="=&" read _ num1 _ num2 <<< "id=12345&data=23456"
$ printf "%s\n" "$num1" "$num2"
12345
23456
With awk:
echo "id=12345&data=23456" | awk -F[\&=] '{ print $2,$4}'
With grep and tr:
echo "id=12345&data=23456" | grep -o '[0-9]\+' | tr '\n' ' '
Note: This above command will add one more space at the end.
Looks like a query string to me...I would suggest parsing it as such. For example, using PHP:
echo "id=12345&data=23456" | php -r 'parse_str(fgets(STDIN), $query); print_r($query);'
This gives the output:
Array
(
[id] => 12345
[data] => 23456
)
So to get the output you were looking for, you could go for:
$ echo "id=12345&data=23456" | php -r 'parse_str(fgets(STDIN), $query); echo $query["id"] . " " . $query["data"];'
12345 23456
For a quick and dirty alternative, you could use sed:
$ echo "id=12345&data=23456" | sed -r 's/id=([^&]+)&data=([^&]+)/\1 \2/'
12345 23456
This captures the part following id= up to the & and the part following &data= up to the next & (if there is one). The disadvantage of this approach is that it breaks if the two parts of the query string are in the opposite order but it might be good enough for your use case.
Alternative Code
echo "id=12345&data=23456" | tr -s '&' '\n' | cut -d '=' -f 2 | tr -s '\n' ''
Depending what you're going to do with the output, one of these may be all you need:
$ echo 'id=12345&data=23456' | tr -c -s '[0-9]' ' '
12345 23456 $
$ echo 'id=12345&data=23456' | tr -s '[a-z=&]' ' '
12345 23456
$
When I am trying to run the below Script it says invalid option 3 for cat..Whats the problem?
I am tried to use index file which specifies which file is ham and which is spam...to read the files and train spamfilter
#!bin/bash
DirBogoDict=$1
BogoFilter=/home/gunna/Downloads/bogofilter-1.2.4/src/bogofilter
x=0
for i in 'cat index | fgrep spam | head -300 | awk -F "/" '{print$2"/"$3}''
do
x=$((x+1)) ; echo $x
cat /home/gunna/Downloads/db-6.1.19.NC/build_unix/ceas08-1/$i| $BogoFilter -d $DirBogoDict -M -k 1024 -s
done
for i in 'cat index | fgrep ham | head -300 | awk -F "/" '{print$2"/"$3}''
do
x=$((x+1)) ; echo $x
cat /home/gunna/Downloads/db-6.1.19.NC/build_unix/ceas08-1/$i | $BogoFilter -d $DirBogoDict -M -k 1024 -n
done
This part
'cat index | fgrep spam | head -300 | awk -F "/" '{print$2"/"$3}''
needs to be in back-ticks, not single quotes
`cat index | fgrep spam | head -300 | awk -F "/" '{print$2"/"$3}'`
And you could probably simplify it a little with
for i in `fgrep spam index | head -300 | awk "/" '{print$2"/"$3}'`
Kdopen has explained the error you got , here is the improved code for similar for-loop function.
DirBogoDict=$1
BogoFilter=/home/gunna/Downloads/bogofilter-1.2.4/src/bogofilter
awk '/spam/&&++myctr<=300{print $2 FS $3}' FS="/" index |while read i
do
cat /home/gunna/Downloads/db-6.1.19.NC/build_unix/ceas08-1/"$i"| $BogoFilter -d ${DirBogoDict} -M -k 1024 -s
done
awk '/ham/&&++myctr<=300{print $2 FS $3}' FS="/" index |while read i
do
cat /home/gunna/Downloads/db-6.1.19.NC/build_unix/ceas08-1/"$i"| $BogoFilter -d ${DirBogoDict} -M -k 1024 -s
done
Also look at your file names , since cat is giving an error and an option is invalid. To demonstrate this, Let say you have a file a name -3error
executing the following command
cat -3error
will gave
cat: invalid option -- '3'
cat therefore is thinking the "-" is followed by one of its command line arguments. As a result you probably get an invalid option error.
I'm sorry for the very noob question, but I'm kind of new to bash programming (started a few days ago). Basically what I want to do is keep one file with all the word occurrences of another file
I know I can do this:
sort | uniq -c | sort
the thing is that after that I want to take a second file, calculate the occurrences again and update the first one. After I take a third file and so on.
What I'm doing at the moment works without any problem (I'm using grep, sed and awk), but it looks pretty slow.
I'm pretty sure there is a very efficient way just with a command or so, using uniq, but I can't figure out.
Could you please lead me to the right way?
I'm also pasting the code I wrote:
#!/bin/bash
# count the number of word occurrences from a file and writes to another file #
# the words are listed from the most frequent to the less one #
touch .check # used to check the occurrances. Temporary file
touch distribution.txt # final file with all the occurrences calculated
page=$1 # contains the file I'm calculating
occurrences=$2 # temporary file for the occurrences
# takes all the words from the file $page and orders them by occurrences
cat $page | tr -cs A-Za-z\' '\n'| tr A-Z a-z > .check
# loop to update the old file with the new information
# basically what I do is check word by word and add them to the old file as an update
cat .check | while read words
do
word=${words} # word I'm calculating
strlen=${#word} # word's length
# I use a black list to not calculate banned words (for example very small ones or inunfluent words, like articles and prepositions
if ! grep -Fxq $word .blacklist && [ $strlen -gt 2 ]
then
# if the word was never found before it writes it with 1 occurrence
if [ `egrep -c -i "^$word: " $occurrences` -eq 0 ]
then
echo "$word: 1" | cat >> $occurrences
# else it calculates the occurrences
else
old=`awk -v words=$word -F": " '$1==words { print $2 }' $occurrences`
let "new=old+1"
sed -i "s/^$word: $old$/$word: $new/g" $occurrences
fi
fi
done
rm .check
# finally it orders the words
awk -F": " '{print $2" "$1}' $occurrences | sort -rn | awk -F" " '{print $2": "$1}' > distribution.txt
Well, I'm not sure that I've got the point of the thing you are trying to do,
but I would do it this way:
while read file
do
cat $file | tr -cs A-Za-z\' '\n'| tr A-Z a-z | sort | uniq -c > stat.$file
done < file-list
Now you have statistics for all your file, and now you simple aggregate it:
while read file
do
cat stat.$file
done < file-list \
| sort -k2 \
| awk '{if ($2!=prev) {print s" "prev; s=0;}s+=$1;prev=$2;}END{print s" "prev;}'
Example of usage:
$ for i in ls bash cp; do man $i > $i.txt ; done
$ cat <<EOF > file-list
> ls.txt
> bash.txt
> cp.txt
> EOF
$ while read file; do
> cat $file | tr -cs A-Za-z\' '\n'| tr A-Z a-z | sort | uniq -c > stat.$file
> done < file-list
$ while read file
> do
> cat stat.$file
> done < file-list \
> | sort -k2 \
> | awk '{if ($2!=prev) {print s" "prev; s=0;}s+=$1;prev=$2;}END{print s" "prev;}' | sort -rn | head
3875 the
1671 is
1137 to
1118 a
1072 of
793 if
744 and
533 command
514 in
507 shell