I have this bash statement for printing a specific cell from a .csv file.
set `cat $filename | awk -v FS=',' '{print $2}' | head -5 | tail -n 1`
The '{print $2}' part determines the column and the head -5 part determines the row.
Can I substitute a $counter variable in place of $2 (e.g., '{print $counter}')?
The answer is "yes" -- and there are a couple ways to do what you want. The proper way is to declare an awk variable using -v:
awk -F',' -v c=$counter 'NR==6 { print $c; exit }' "$filename"
(You will forgive me for moving some things around to do everything in awk, for passing "$filename" to awk safely, and for getting rid of set and back ticks -- that were doing nothing for the cause.)
Another way to do this is a bit of a "hackish" way -- leveraging shell quoting rules. This method requires some escaping to ensure that the first $ character (that references the intended field in awk) is not interpreted by the shell... The following works in bash (and POSIX sh):
awk -F',' "NR==6 { print \$$counter; exit }" "$filename"
Yes and all pipes could be removed. Variables are passed to awk with -v var=value.
Give a try to this tested version. Provide a value to the ̀€col and row variables:
set $(awk -F "," -v col=2 -v row=5 'NR==row {print $col; exit}' "${filename}")
$(command) is prefered to `command`, this later is deprecated.
NR is the current line number.
"${filename}" is expanded by the shell to its value: the double quotes will help if the filename contains some special chars.
Related
This question already has answers here:
How do I use shell variables in an awk script?
(7 answers)
Closed 1 year ago.
good morning, I am a newbie in the world of scripts and I have this problem and I don't know why it happens to me and why I have it wrong, thanks in advance.
For example, I have this command that searches for users with X less letters:
cut -d: -f1 /etc/passwd | awk 'length($1) <= 4'
It works correctly but when I substitute a 4 for a variable with the same value it doesn't do it well:
number=4
echo -e $(cut -d: -f1 /etc/passwd | awk 'length($1) <= $number')
The same error happens to me here too, when I search for users who have an old password
awk -F: '{if($3<=18388)print$1}' < /etc/shadow
Works, but when I use the variable it stops working
variable=18388
awk -F: '{if($3<=$variable)print$1}' < /etc/shadow
Consider using awk's ability to import variables via the -v option, eg:
number=4
cut -d: -f1 /etc/passwd | awk -v num="${number}" 'length($1) <= num'
Though if the first field from /etc/passwd contains white space, and you want to consider the length of the entire field, you should replace $1 with $0, eg:
number=4
cut -d: -f1 /etc/passwd | awk -v num="${number}" 'length($0) <= num'
Even better, eliminate cut and the subprocess (due to the pipe) and use awk for the entire operation by designating the input field separator as a colon:
number=4
awk -F':' -v num="${number}" 'length($1) <= num { print $1 }' /etc/passwd
You need to use double quotes:
echo -e $(cut -d: -f1 /etc/passwd | awk "length(\$1) <= $number")
In single quotes, variables are not interpreted.
Updated: Here is an illustrative example:
> X=1; echo '$X'; echo "$X"
$X
1
Update 2: as rightfully pointed out in the comment, when using double quotes one need to make sure that the variables that are meant for awk to interpret are escaped, so they are not interpreted during script evaluation. Command updated above.
Alternatively, this task could be done using only a single GNU sed one-liner:
num=4
sed -E "s/:.*//; /..{$num}/d" /etc/passwd
Another one-liner, using only grep would be
grep -Po "^[^:]{1,$num}(?=:)" /etc/passwd
but this requires a grep supporting perl regular expressions, for the lookahead construct (?=...).
I need to grep some pattern and further i need to print some output within that. Currently I am using the below command which is working fine. But I like to eliminate using multiple pipe and want to use single awk command to achieve the same output. Is there a way to do it using awk?
root#Server1 # cat file
Jenny:Mon,Tue,Wed:Morning
David:Thu,Fri,Sat:Evening
root#Server1 # awk '/Jenny/ {print $0}' file | awk -F ":" '{ print $2 }' | awk -F "," '{ print $1 }'
Mon
I want to get this output using single awk command. Any help?
You can try something like:
awk -F: '/Jenny/ {split($2,a,","); print a[1]}' file
Try this
awk -F'[:,]+' '/Jenny/{print $2}' file.txt
It is using muliple -F value inside the [ ]
The + means one or more since it is treated as a regex.
For this particular job, I find grep to be slightly more robust.
Unless your company has a policy not to hire people named Eve.
(Try it out if you don't understand.)
grep -oP '^[^:]*Jenny[^:]*:\K[^,:]+' file
Or to do a whole-word match:
grep -oP '^[^:]*\bJenny\b[^:]*:\K[^,:]+' file
Or when you are confident that "Jenny" is the full name:
grep -oP '^Jenny:\K[^,:]+' file
Output:
Mon
Explanation:
The stuff up until \K speaks for itself: it selects the line(s) with the desired name.
[^,:]+ captures the day of week (in this case Mon).
\K cuts off everything preceding Mon.
-o cuts off anything following Mon.
I'm trying to take a column and pipe it through an echo command. If possible, I would like to keep it in one line or do this as efficiently as possible. While researching, I found that I have to use single quotes to expand the variable and to escape the double quotes.
Here's what I was trying:
awk -F ',' '{print $2}' file1.txt | while read line; do echo "<href=\"'${i}'\">'${i}'</a>"; done
But, I keep getting the number of lines than the single line's output. If you know how to caputure each line in field 4, that would be so helpful.
File1.txt:
Hello,http://example1.com
Hello,http://example2.com
Hello,http://example3.com
Desired output:
<href="http://example1.com">http://example1.com</a>
<href="http://example2.com">http://example2.com</a>
<href="http://example3.com">http://example3.com</a>
$ awk -F, '{printf "<href=\"%s\">%s</a>\n", $2, $2}' file
<href="http://example1.com">http://example1.com</a>
<href="http://example2.com">http://example2.com</a>
<href="http://example3.com">http://example3.com</a>
Or slightly briefer but less robustly:
$ sed 's/.*,\(.*\)/<href="\1">\1<\/a>/' file
<href="http://example1.com">http://example1.com</a>
<href="http://example2.com">http://example2.com</a>
<href="http://example3.com">http://example3.com</a>
So I have this string:
$var=server#10.200.200.20:/home/some/directory/file
I just want to extract the directory address meaning I only want the bit after the ":" character and get:
/home/some/directory/file
thanks.
I need a generic command so the cut command wont work as the $var variable doesn't have a fixed length.
Using sed:
$ var=server#10.200.200.20:/home/some/directory/file
$ echo $var | sed 's/.*://'
/home/some/directory/file
This might work for you:
echo ${var#*:}
See Example 10-10. Pattern matching in parameter substitution
This will also do.
echo $var | cut -f2 -d":"
For completeness, using cut
cut -d : -f 2 <<< $var
And using only bash:
IFS=: read a b <<< $var ; echo $b
You don't say which shell you're using. If it's a POSIX-compatible one such as Bash, then parameter expansion can do what you want:
Parameter Expansion
...
${parameter#word}
Remove Smallest Prefix Pattern.
The word is expanded to produce a pattern. The parameter expansion then results in parameter, with the smallest portion of the prefix matched by the pattern deleted.
In other words, you can write
$var="${var#*:}"
which will remove anything matching *: from $var (i.e. everything up to and including the first :). If you want to match up to the last :, then you could use ## in place of #.
This is all assuming that the part to remove does not contain : (true for IPv4 addresses, but not for IPv6 addresses)
This should do the trick:
$ echo "$var" | awk -F':' '{print $NF}'
/home/some/directory/file
awk -F: '{print $2}' <<< $var
I have a file with the following layout:
123,01-08-2006
124,01-09-2007
125,01-10-2009
126,01-12-2010
How can I convert it into the following by using AWK?
123,2006-08-01
124,2007-09-01
125,2009-10-01
126,2009-12-01
Didn't read the question properly the first time. You need a field separator that can be either a dash or a comma. Once you have that you can use the dash as an output field separator (as it's the most common) and fake the comma using concatenation:
awk -F',|-' 'OFS="-" {print $1 "," $4,$3,$2}' file
Pure awk
awk -F"," '{ n=split($2,b,"-");$2=b[3]"-"b[2]"-"b[1];$i=$1","$2 } 1' file
sed
sed -r 's/(^.[^,]*,)([0-9]{2})-([0-9]{2})-([0-9]{4})/\1\4-\3-\2/' file
sed 's/\(^.[^,]*,\)\([0-9][0-9]\)-\([0-9][0-9]\)-\([0-9]\+\)/\1\4-\3-\2/' file
Bash
#!/bin/bash
while IFS="," read -r a b
do
IFS="-"
set -- $b
echo "$a,$3-$2-$1"
done <"file"
Unfortunately, I think standard awk only allows one field separator character so you'll have to pre-process the data. You can do this with tr but if you really want an awk-only solution, use:
pax> echo '123,01-08-2006
124,01-09-2007
125,01-10-2009
126,01-12-2010' | awk -F, '{print $1"-"$2}' | awk -F- '{print $1","$4"-"$3"-"$2}'
This outputs:
123,2006-08-01
124,2007-09-01
125,2009-10-01
126,2010-12-01
as desired.
The first awk changes the , characters to - so that you have four fields separated with the same character (this is the bit I'd usually use tr ',' '-' for).
The second awk prints them out in the order you specified, correcting the field separators at the same time.
If you're using an awk implementation that allows multiple FS characters, you can use something like:
gawk -F ',|-' '{print $1","$4"-"$3"-"$2}'
If it doesn't need to be awk, you could use Perl too:
$ perl -nle 'print "$1,$4-$3-$2" while (/(\d{3}),(\d{2})-(\d{2})-(\d{4})\s*/g)' < file.txt