awk -F '$' works well with single-dollar-sign-separated string (a$b$c for example), but when it comes to multiple dollar signs, awk does not work.
The expected output is: 1$23, I have tried the following combinations but in vain:
$ printf '1$23$$$456' | awk -F '$$$' '{print $1}'
1$23$$$456
$ printf '1$23$$$456' | awk -F '\$\$\$' '{print $1}'
1$23$$$456
$ printf '1$23$$$456' | awk -F '\\$\\$\\$' '{print $1}'
1$23$$$456
$ printf '1$23$$$456' | awk -F '$' '{print $1}'
1
I wonder if there is a way to split a string by a sequence of dollar signs using awk?
update
$ awk --version
awk version 20070501
$ echo $SHELL
/usr/local/bin/fish
The problem is due to fish quoting rules. The important difference is that fish, unlike Bash, allows escaping a single quote within a single quoted string, like so:
$ echo '\''
'
and, consequently, a literal backslash has to be escaped as well:
$ echo '\\'
\
So, to get what in Bash corresponds to \\, we have to use \\\\:
$ printf '1$23$$$456' | awk -F '\\\\$\\\\$\\\\$' '{print $1}'
1$23
awk and various shells have nasty behaviours with escaping characters with back-slashes. Various shells could have different behaviours and sometimes you really need to escape like crazy to make it work. The easiest is to use [$] for a single symbol. This always works for field separators as FS is a regular expression if it is more than one symbol.
$ awk -F '[$][$][$]' '{...}' file
More \
#> printf '1$23$$$456' | awk -F '\\$\\$\\$' '{print $1}'
1$23
Maybe not use awk if that's throwing you curves?
$: echo '1$23$$$456' | sed 's/$$$.*//'
1$23
Why farm it out to a subshell at all for somehting that's just string processing?
$: x='1$23$$$456'
$: echo "${x%%\$\$\$*}"
1$23
Related
This question already has answers here:
How do I use shell variables in an awk script?
(7 answers)
Closed 1 year ago.
good morning, I am a newbie in the world of scripts and I have this problem and I don't know why it happens to me and why I have it wrong, thanks in advance.
For example, I have this command that searches for users with X less letters:
cut -d: -f1 /etc/passwd | awk 'length($1) <= 4'
It works correctly but when I substitute a 4 for a variable with the same value it doesn't do it well:
number=4
echo -e $(cut -d: -f1 /etc/passwd | awk 'length($1) <= $number')
The same error happens to me here too, when I search for users who have an old password
awk -F: '{if($3<=18388)print$1}' < /etc/shadow
Works, but when I use the variable it stops working
variable=18388
awk -F: '{if($3<=$variable)print$1}' < /etc/shadow
Consider using awk's ability to import variables via the -v option, eg:
number=4
cut -d: -f1 /etc/passwd | awk -v num="${number}" 'length($1) <= num'
Though if the first field from /etc/passwd contains white space, and you want to consider the length of the entire field, you should replace $1 with $0, eg:
number=4
cut -d: -f1 /etc/passwd | awk -v num="${number}" 'length($0) <= num'
Even better, eliminate cut and the subprocess (due to the pipe) and use awk for the entire operation by designating the input field separator as a colon:
number=4
awk -F':' -v num="${number}" 'length($1) <= num { print $1 }' /etc/passwd
You need to use double quotes:
echo -e $(cut -d: -f1 /etc/passwd | awk "length(\$1) <= $number")
In single quotes, variables are not interpreted.
Updated: Here is an illustrative example:
> X=1; echo '$X'; echo "$X"
$X
1
Update 2: as rightfully pointed out in the comment, when using double quotes one need to make sure that the variables that are meant for awk to interpret are escaped, so they are not interpreted during script evaluation. Command updated above.
Alternatively, this task could be done using only a single GNU sed one-liner:
num=4
sed -E "s/:.*//; /..{$num}/d" /etc/passwd
Another one-liner, using only grep would be
grep -Po "^[^:]{1,$num}(?=:)" /etc/passwd
but this requires a grep supporting perl regular expressions, for the lookahead construct (?=...).
I have CSV columns that I'm working with:
info,example-string,super-example-string,otherinfo
I would like to get:
example-string super example string
Right now, I'm running the following command:
awk -F ',' '{print $3}' | sed "s/-//g"
But, then I have to paste the lines together to combine $2 and $3.
Is there anyway to do something like this?
awk -F ',' '{print $2" "$3}' | sed "s/-//g"
Except, where the sed command is only performed on $3 and $2 stays in place? I'm just concerned later on if the lines don't match up, the data could be misaligned.
Please note: I need to keep the pipe for the SED command. I just used a simple example but I end up running a lot of commands after that as well.
Try:
$ awk -F, '{gsub(/-/," ",$3); print $2,$3}' file
example-string super example string
How it works
-F,
This tells awk to use a comma as the field separator.
gsub(/-/," ",$3)
This replaces all - in field 3 with spaces.
print $2,$3
This prints fields 2 and 3.
Examples using pipelines
$ echo 'info,example-string,super-example-string,otherinfo' | awk -F, '{gsub(/-/," ",$3); print $2,$3}'
example-string super example string
In a pipeline with sed:
$ echo 'info,example-string,super-example-string,otherinfo' | awk -F, '{gsub(/-/," ",$3); print $2,$3}' | sed 's/string/String/g'
example-String super example String
Though best solution will be either use a single sed or use single awk. Since you have requested to use awk and sed solution so providing this. Also considering your actual data will be same as shown sample Input_file.
awk -F, '{print $2,$3}' Input_file | sed 's/\([^ ]*\)\([^-]*\)-\([^-]*\)-\([^-]*\)/\1 \2 \3 \4/'
Output will be as follows.
example-string super example string
This question already has answers here:
Using awk with variables
(3 answers)
Closed 8 years ago.
I have following variable set in my unix environment. If i try to use it in awk command its not working but the same command is working when i dont use $b variable
$b="NEW"
when i try following command it is not working
echo "$a" | tr [a-z] [A-Z] |awk -v RS=, '/TABLE/&&/CREATE/&&/`echo ${b}`/{print $NF}'
But, if i replace the $b value to NEW as below its working
echo "$a" | tr [a-z] [A-Z] |awk -v RS=, '/TABLE/&&/CREATE/&&/NEW/{print $NF}'
You cannot use a bash var inside awk like that. Instead, use:
echo "$a" | tr [a-z] [A-Z] | awk -v RS=, -v myvar=$b '/TABLE/&&/CREATE/&& $0~myvar {print $NF}'
See an example:
$ var="hello"
$ awk -v text=$var 'BEGIN{print text}'
hello
Also, to me it works with tr 'a-z' 'A-Z' instead of tr [a-z] [A-Z]. And based on Mark Setchell suggestion, you can skip it by using the IGNORECASE = 1:
echo "$a" | awk -v RS=, -v myvar=$b 'BEGIN{IGNORECASE=1} /TABLE/&&/CREATE/&& $0~myvar {print $NF}'
Regarding your question:
if i replace the $b value to NEW as below its working
It works because the value of your variable is NEW and what you end up doing is using that in the regex, which is exactly how it is supposed to be done.
about your second question:
can not use unix $variable in awk command
You cannot use shell variables in awk like that. You need to create an awk variable by using -v option and assigning your bash variable.
awk -v awkvar="$bashvar" '/ /{ ... }'
This makes your existing syntax as:
echo "$a" | tr [a-z] [A-Z] | awk -v RS=, -v var="$b" '/TABLE/&&/CREATE/&&/var/{print $NF}'
This again won't work because inside /../ variables are not interpolated, meaning they are considered literally. So, you need to do:
echo "$a" | tr [a-z] [A-Z] |awk -v RS=, -v var="$b" '/TABLE/&&/CREATE/&&$0~var{print $NF}'
I'm trying to understand why the command below doesn't work (output is empty):
echo 'aaa\tbbb' | awk -F '\\t' '{print $2}'
I would expect the output to be 'bbb'.
Interestingly this works (output is 'bbb'):
echo 'aaa\tbbb' | awk -F 't' '{print $2}'
And this works as well (ouptut is 'tbbb'):
echo 'aaa\tbbb' | awk -F '\\' '{print $2}'
It looks as if \\\t is read as backslash followed by tab instead of escaped backslash followed by t.
Is there a proper way to write this command?
You need to tell echo to interpret backslash escapes. Try:
$ echo -e 'aaa\tbbb' | awk -F '\t' '{print $2}'
bbb
man echo would tell:
-e enable interpretation of backslash escapes
I have a file with the following layout:
123,01-08-2006
124,01-09-2007
125,01-10-2009
126,01-12-2010
How can I convert it into the following by using AWK?
123,2006-08-01
124,2007-09-01
125,2009-10-01
126,2009-12-01
Didn't read the question properly the first time. You need a field separator that can be either a dash or a comma. Once you have that you can use the dash as an output field separator (as it's the most common) and fake the comma using concatenation:
awk -F',|-' 'OFS="-" {print $1 "," $4,$3,$2}' file
Pure awk
awk -F"," '{ n=split($2,b,"-");$2=b[3]"-"b[2]"-"b[1];$i=$1","$2 } 1' file
sed
sed -r 's/(^.[^,]*,)([0-9]{2})-([0-9]{2})-([0-9]{4})/\1\4-\3-\2/' file
sed 's/\(^.[^,]*,\)\([0-9][0-9]\)-\([0-9][0-9]\)-\([0-9]\+\)/\1\4-\3-\2/' file
Bash
#!/bin/bash
while IFS="," read -r a b
do
IFS="-"
set -- $b
echo "$a,$3-$2-$1"
done <"file"
Unfortunately, I think standard awk only allows one field separator character so you'll have to pre-process the data. You can do this with tr but if you really want an awk-only solution, use:
pax> echo '123,01-08-2006
124,01-09-2007
125,01-10-2009
126,01-12-2010' | awk -F, '{print $1"-"$2}' | awk -F- '{print $1","$4"-"$3"-"$2}'
This outputs:
123,2006-08-01
124,2007-09-01
125,2009-10-01
126,2010-12-01
as desired.
The first awk changes the , characters to - so that you have four fields separated with the same character (this is the bit I'd usually use tr ',' '-' for).
The second awk prints them out in the order you specified, correcting the field separators at the same time.
If you're using an awk implementation that allows multiple FS characters, you can use something like:
gawk -F ',|-' '{print $1","$4"-"$3"-"$2}'
If it doesn't need to be awk, you could use Perl too:
$ perl -nle 'print "$1,$4-$3-$2" while (/(\d{3}),(\d{2})-(\d{2})-(\d{4})\s*/g)' < file.txt