i am trying to let this command work but it won't let me do anything
awk -F: ‘{if($3>'1000') print$1}’ passwd | sort > users.txt
I get an error which is saying:
bash: syntaxfout nabij onverwacht symbool '('
Can someone help me out?
You're using ‘ instead of '. And then, you should replace ' with " in the awk program (or just leave them out):
awk -F: '{if ($3 > 1000) print $1}' passw | ...
You're using backticks instead of single quotes. Try:
awk -F: '{if($3>1000) print $1} passwd | sort > users.txt
or just
awk -F: '$3>1000 {print $1}' passwd | sort > users.txt
My script is as below. When we run the script, it automatically saves the disk space usage in separate cells.
SIZES_1=`df -h | awk 'FNR == 1 {print $1","$2","$3","$4","$5","$6}'`
SIZES_2=`df -h | awk 'FNR == 2 {print $1","$2","$3","$4","$5","$6}'`
SIZES_3=`df -h | awk 'FNR == 3 {print $1","$2","$3","$4","$5","$6}'`
SIZES_4=`df -h | awk 'FNR == 4 {print $1","$2","$3","$4","$5","$6}'`
SIZES_5=`df -h | awk 'FNR == 5 {print $1","$2","$3","$4","$5","$6}'`
SIZES_6=`df -h | awk 'FNR == 6 {print $1","$2","$3","$4","$5","$6}'`
SIZES_7=`df -h | awk 'FNR == 7 {print $1","$2","$3","$4","$5","$6}'`
SIZES_8=`df -h | awk 'FNR == 8 {print $1","$2","$3","$4","$5","$6}'`
echo `date +%Z-%Y-%m-%d_%H-%M-%S` >>/home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_1" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_2" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_3" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_4" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_5" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_6" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_7" >> /home/jeevagan/test_scripts/sizes/excel.csv
echo "$SIZES_8" >> /home/jeevagan/test_scripts/sizes/excel.csv
This script is okay for my machine.
My doubt is, if somebody else's machine has many file systems, my script won't work to fetch all the file systems usage. How to make it to grab all those automatically?
Assuming you want all filesystems you can simplify that to:
printf '%s\n' "$(date +%Z-%Y-%m-%d_%H-%M-%S)" >> excel.csv
df -h | awk '{print $1","$2","$3","$4","$5","$6}' >> excel.csv
I would simplify this to
{ date +%Z-%F_%H-%M-%S; df -h | tr -s ' ' ','; } >> excel.csv
Group commands so only a single redirect is needed
Squeeze spaces and replace them with a single comma using tr
No need for echo `date` or similar: it's the same as just date
date +%Y-%m-%d is the same as date +%F
Notice that this has a little flaw in that the first line of the output of df -h, which looks something like this originally
Filesystem Size Used Avail Use% Mounted on
has a space in the heading of the last column, so it becomes
Filesystem,Size,Used,Avail,Use%,Mounted,on
with an extra comma. The original awk solution just cut off the last word of the line, though. Similarly, spaces in paths would trip up this solution.
To fix the comma problem, you could for example run
sed -i 's/Mounted,on$/Mounted on/' excel.csv
every now and so often.
As an aside, to replace all field separators in awk, instead of
awk '{print $1","$2","$3","$4","$5","$6}'
you can use
awk 'BEGIN { OFS = "," } { $1 = $1; print }'
or, shorter,
awk -v OFS=',' '{$1=$1}1'
This is just an example of what I run into a lot:
I would like to copy all .bash_histories to one directory.
grep "/bin/bash" /etc/passwd | awk -F: '{ print "cp " $6"/.bash_history /backup" $6 ".bash_history" }
Output:
cp /home/peter/.bash_history /backup/home/peter/.bash_history
cp /home/john/.bash_history /backup/home/john/.bash_history
What I would like is an output like this:
cp /home/peter/.bash_history /backup/_home_peter_.bash_history
cp /home/john/.bash_history /backup/_home_john_.bash_history
And that this output will be executed.
(It's not specifically about this issue, but just in general how to reformat with awk and sed and execute the new created command line, without really creating a script for it)
The awk script to obtain a similar output will be
grep "/bin/bash" /etc/passwd |head -2 | awk -F: '{ print "cp " $6 "/.bash_history backup/_home_"$1".bash_history" }'
giving an output like
cp /root/.bash_history backup/_home_root.bash_history
cp /home/xxx/.bash_history backup/_home_xxx.bash_history
Now inorder to excecute the commands, the system() function within the awk would be helpfull
system(command) would excecute any command, and return value being the exit status of the command.
The above script can be modified as
grep "/bin/bash" /etc/passwd |head -2 | awk -F: '{ system("cp " $6 "/.bash_history backup/_home_"$1".bash_history;") }'
Test run:
$ grep "/bin/bash" /etc/passwd |head -2 | awk -F: '{ system("cp " $6 "/.bash_history backup/_home_"$1".bash_history;") }'
$ ls backup/
_home_xxx.bash_history _home_root.bash_history
PS: It is not recommend to create directories in your root folder. So i intentionally replaced /backup in your script to backup.
Also inorder for the script to be successful, the backup folder must be created before hand.
getent passwd | grep \/bin\/bash | cut -d ":" -f 6 | while read a; do eval "cp $a/.bash_history /backup/$(echo $a | sed 's#/#_#g')_.bash_history"; done
This uses getent to fetch the passwd file and cut gets the 6th field like your awk statement did, then it reads each entry line by line and builds the string and executes it with eval.
getent passwd | grep \/bin\/bash | cut -d ":" -f 6 | while read a; do eval "cp $a/.bash_history /backup/$(echo $a | sed 's#/#_#g')_.bash_history"; done
Worked perfectly! Issue solved!
I have a line in a bash script that calculates the sum of unique IP requests to a certain page.
grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}' | sort | uniq -c | awk '{sum += 1; print } END { print " ", sum, "total"}'
I am trying to get the value of sum to a variable outside the awk statement so I can compare pages to each other. So far I have tried various combinations of something like this:
unique_sum=0
grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}' | sort | uniq -c | awk '{sum += 1; print ; $unique_sum=sum} END { print " ", sum, "total"}'
echo "${unique_sum}"
This results in an echo of "0". I've tried placing __$unique_sum=sum__ in the END, various combinations of initializing the variable (awk -v unique_sum=0 ...) and placing the variable assignment outside of the quoted sections.
So far, my Google-fu is failing horribly as most people just send the whole of the output to a variable. In this example, many lines are printed (one for each IP) in addition to the total. Failing a way to capture the 'sum' variable, is there a way to capture that last line of output?
This is probably one of the most sophisticated things I've tried in awk so my confidence that I've done anything useful is pretty low. Any help will be greatly appreciated!
You can't assign a shell variable inside an awk program. In general, no child process can alter the environment of its parent. You have to have the awk program print out the calculated value, and then shell can grab that value and assign it to a variable:
output=$( grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}' | sort | uniq -c | awk '{sum += 1; print } END {print sum}' )
unique_sum=$( sed -n '$p' <<< "$output" ) # grab the last line of the output
sed '$d' <<< "$output" # print the output except for the last line
echo " $unique_sum total"
That pipeline can be simplified quite a lot: awk can do what grep can do, so first
grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}'
is (longer, but only one process)
awk -F" - " -v date="$YESTERDAY" -v patt="$1" '$0 ~ date && $0 ~ patt {print $1}' "$ACCESSLOG"
And the last awk program just counts how many lines and can be replaced with wc -l
All together:
unique_output=$(
awk -F" - " -v date="$YESTERDAY" -v patt="$1" '
$0 ~ date && $0 ~ patt {print $1}
' "$ACCESSLOG" | sort | uniq -c
)
echo "$unique_output"
unique_sum=$( wc -l <<< "$unique_output" )
echo " $unique_sum total"
I have a text files with a line like this in them:
MC exp. sig-250-0 events & $0.98 \pm 0.15$ & $3.57 \pm 0.23$ \\
sig-250-0 is something that can change from file to file (but I always know what it is for each file). There are lines before and above this, but the string "MC exp. sig-250-0 events" is unique in the file.
For a particular file, is there a good way to extract the second number 3.57 in the above example using bash?
use awk for this:
awk '/MC exp. sig-250-0/ {print $10}' your.txt
Note that this will print: $3.57 - with the leading $, if you don't like this, pipe the output to tr:
awk '/MC exp. sig-250-0/ {print $10}' your.txt | tr -d '$'
In comments you wrote that you need to call it in a script like this:
while read p ; do
echo $p,awk '/MC exp. sig-$p/ {print $10}' filename | tr -d '$'
done < grid.txt
Note that you need a sub shell $() for the awk pipe. Like this:
echo "$p",$(awk '/MC exp. sig-$p/ {print $10}' filename | tr -d '$')
If you want to pass a shell variable to the awk pattern use the following syntax:
awk -v p="MC exp. sig-$p" '/p/ {print $10}' a.txt | tr -d '$'
More lines would've been nice but I guess you would like to have a simple use awk.
awk '{print $N}' $file
If you don't tell awk what kind of field-separator it has to use it will use just a space ' '. Now you just have to count how many fields you have got to get your field you want to get. In your case it would be 10.
awk '{print $10}' file.txt
$3.57
Don't want the $?
Pipe your awk result to cut:
awk '{print $10}' foo | cut -d $ -f2
-d will use the $ als field-separator and -f will select the second field.
If you know you always have the same number of fields, then
#!/bin/bash
file=$1
key=$2
while read -ra f; do
if [[ "${f[0]} ${f[1]} ${f[2]} ${f[3]}" == "MC exp. $key events" ]]; then
echo ${f[9]}
fi
done < "$file"