file1
ABY37499|ANK37528|DEL37508|SRILANKA|195203230000|445500759
ARJU7499|CHA38008|DEL37508|SRILANKA|195203230000|445500759
IB1704174|ANK37528|DEL37508|SRILANKA|195203230000|445500759
IB1704174|CHA38008|DEL37508|SRILANKA|195203230000|445500759
ABY37500|ANK37529|DEL37509|BRAZIL|195203240000|445500757
ARJU7500|CHA38009|DEL37509|BRAZIL|195203240000|445500757
IB1704175|ANK37529|DEL37509|BRAZIL|195203240000|445500757
i want to convert the fifth column date to another format script below
#!/bin/sh
dt="%Y-%m-%d %H:%M"
awk -F '|' '{print $5}' file1 | sed 's/.\{8\}/& /g'> f1.txt
aa=`(date -f f1.txt +"$dt")`
echo "$aa"
awk -F '|' '$5=$aa' file1
echo "$aa" got desired output but i cannot assign $aa to $5 please help me.
Thanks
I corrected my answer after the commento of Etan Reisner
from AWK man:
The input is read in units called records, and processed by the rules
of your program one record at a time. By default, each record is one
line. Each record is automatically split into chunks called fields.
This makes it more convenient for programs to work on the parts of a
record.
Fields are stored in variables $1, $2, ...
And
The contents of a field, as seen by awk, can be changed within an awk
program; this changes what awk perceives as the current input record.
see the man page
thus, this expression:
awk -F '|' '$5=$aa' file1
does not have the effect of substitute the fifth column of file1.
You have to write the modified output to a second file.
May be this could help you in sed
echo 195203240000 | sed -n -e "s_\(....\)\(..\)\(..\)\(..\)\(..\)_\1-\2-\3 \4:\5_p"
1952-03-24 00:00
This awk script should do what you want.
It isn't exactly pretty but it works assuming the input format is consistent.
awk '{$5=sprintf("%s-%s-%s %s:%s\n",
substr($5,1,4), substr($5,5,2), substr($5,7,2),
substr($5,9,2), substr($5,11,2))} 7' file1 > file1.new
It assigns the new value for the field to $5 and then uses 7 (as a truth-y value) to get the default awk {print} action to print the modified line.
Related
I need to grep some pattern and further i need to print some output within that. Currently I am using the below command which is working fine. But I like to eliminate using multiple pipe and want to use single awk command to achieve the same output. Is there a way to do it using awk?
root#Server1 # cat file
Jenny:Mon,Tue,Wed:Morning
David:Thu,Fri,Sat:Evening
root#Server1 # awk '/Jenny/ {print $0}' file | awk -F ":" '{ print $2 }' | awk -F "," '{ print $1 }'
Mon
I want to get this output using single awk command. Any help?
You can try something like:
awk -F: '/Jenny/ {split($2,a,","); print a[1]}' file
Try this
awk -F'[:,]+' '/Jenny/{print $2}' file.txt
It is using muliple -F value inside the [ ]
The + means one or more since it is treated as a regex.
For this particular job, I find grep to be slightly more robust.
Unless your company has a policy not to hire people named Eve.
(Try it out if you don't understand.)
grep -oP '^[^:]*Jenny[^:]*:\K[^,:]+' file
Or to do a whole-word match:
grep -oP '^[^:]*\bJenny\b[^:]*:\K[^,:]+' file
Or when you are confident that "Jenny" is the full name:
grep -oP '^Jenny:\K[^,:]+' file
Output:
Mon
Explanation:
The stuff up until \K speaks for itself: it selects the line(s) with the desired name.
[^,:]+ captures the day of week (in this case Mon).
\K cuts off everything preceding Mon.
-o cuts off anything following Mon.
I'm trying to take a column and pipe it through an echo command. If possible, I would like to keep it in one line or do this as efficiently as possible. While researching, I found that I have to use single quotes to expand the variable and to escape the double quotes.
Here's what I was trying:
awk -F ',' '{print $2}' file1.txt | while read line; do echo "<href=\"'${i}'\">'${i}'</a>"; done
But, I keep getting the number of lines than the single line's output. If you know how to caputure each line in field 4, that would be so helpful.
File1.txt:
Hello,http://example1.com
Hello,http://example2.com
Hello,http://example3.com
Desired output:
<href="http://example1.com">http://example1.com</a>
<href="http://example2.com">http://example2.com</a>
<href="http://example3.com">http://example3.com</a>
$ awk -F, '{printf "<href=\"%s\">%s</a>\n", $2, $2}' file
<href="http://example1.com">http://example1.com</a>
<href="http://example2.com">http://example2.com</a>
<href="http://example3.com">http://example3.com</a>
Or slightly briefer but less robustly:
$ sed 's/.*,\(.*\)/<href="\1">\1<\/a>/' file
<href="http://example1.com">http://example1.com</a>
<href="http://example2.com">http://example2.com</a>
<href="http://example3.com">http://example3.com</a>
I have the following string that I want to extract name and id from and store them in a variable. This is just an example, the list can be longer but they are separated the same way.
[["freepbx","NEWUPDATES","There are 6 modules available for online upgrades"],["cidlookup","noauth","OpenCNAM Requires Authentication"]]
The id's in the string is freepbx and cidlookup, the names are NEWUPDATES and noauth.
I'd like them to come out like:
freepbx NEWUPDATES
cidlookup noauth
I'm running a program from command line that needs it's input this way.
Any help is greatly appreciated!
This is one way to do it:
echo '[["freepbx","NEWUPDATES","There are 6 modules available for online upgrades"],["cidlookup","noauth","OpenCNAM Requires Authentication"]]' | sed -e 's/\],\[/\n/g' -e 's/\(\[\[\)*"//g' | awk -F ',' '{print $1, $2}'
freepbx NEWUPDATES
cidlookup noauth
Explanation:
The sed command s/\],\[/\n/g will replace all ], [ which separate each record with a new line(\n) character. This will allow you to treat each line as a separate record which makes all other tools much easier:)
The second sed command s/\(\[\[\)*"//g will remove the quotes and the initial [[ at the start of the first record. This cleans up things from your data leaving only the , between your fields.
Finally, awk command -F ',' '{print $1, $2}', the -F tells awk to use the , as field separator (instead of space) and $1 and $2 to print the first and second fields.
awk to the rescue!
$ awk -F'"' -v RS="\\\],\\\[" '{print $2,$4}' file
freepbx NEWUPDATES
cidlookup noauth
If jq is available:
jq -r '.[] | "\(.[0]) \(.[1])"'
Pipe .[] ( all elements in the array) output to print only 0th and 1st element "\(.[0]) \(.[1])"as in desired output.
How can I use cut with --output-delimiter=""? I want to join two columns using cut.
I tried the following command. However cat -v shows that there are non printable characters. Specifically "^#". Any suggestions to how can I overcome this?
cut -d, -f 3,6 --output-delimiter="" file1.csv | cat -v
This is the content of my file
011,IBM,Palmisano,t,t,t
012,INTC,Otellini,t,t,t
013,SAP,Snabe,t,t,t
014,VMW,Maritz,t,t,t
015,ORCL,Ellison,t,t,t
017,RHT,Whitehurst,t,t,t
When i run my command I'm seeing
Palmisano^#t
Otellini^#t
Snabe^#t
Maritz^#t
Ellison^#t
Whitehurst^#t
Expected output: Basically I want to exclude ^# in the output
Palmisanot
Otellinit
Snabet
Maritzt
Ellisont
Whitehurstt
Thank you.
The output delimiter is not an empty string, but probably the NULL character. You might want to try
cut -d, -f 3,6 --output-delimiter=$'\00' file1.csv
(Assuming your shell supports $'...'-quoting; bash and zsh are fine here, not sure about others).
edit:
cut apparently puts the NULL character if the output separator is set to the empty string. I do not see a way around it.
If awk is an acceptable solution, this will do the trick:
awk -F, '{print $3 $6}' file*
If you want to be more verbose and explicit:
awk 'BEGIN{FS=","; OFS=""}; {print $3,$6}' file*
FS="," sets the field separator to ,.
OFS="" sets the Output Field Separator to the empty string.
You probably don't want to cut by fields but instead by characters or perhaps bytes. See the description of -c and/or -b in the man page, instead of using -f.
I have a file with the following layout:
123,01-08-2006
124,01-09-2007
125,01-10-2009
126,01-12-2010
How can I convert it into the following by using AWK?
123,2006-08-01
124,2007-09-01
125,2009-10-01
126,2009-12-01
Didn't read the question properly the first time. You need a field separator that can be either a dash or a comma. Once you have that you can use the dash as an output field separator (as it's the most common) and fake the comma using concatenation:
awk -F',|-' 'OFS="-" {print $1 "," $4,$3,$2}' file
Pure awk
awk -F"," '{ n=split($2,b,"-");$2=b[3]"-"b[2]"-"b[1];$i=$1","$2 } 1' file
sed
sed -r 's/(^.[^,]*,)([0-9]{2})-([0-9]{2})-([0-9]{4})/\1\4-\3-\2/' file
sed 's/\(^.[^,]*,\)\([0-9][0-9]\)-\([0-9][0-9]\)-\([0-9]\+\)/\1\4-\3-\2/' file
Bash
#!/bin/bash
while IFS="," read -r a b
do
IFS="-"
set -- $b
echo "$a,$3-$2-$1"
done <"file"
Unfortunately, I think standard awk only allows one field separator character so you'll have to pre-process the data. You can do this with tr but if you really want an awk-only solution, use:
pax> echo '123,01-08-2006
124,01-09-2007
125,01-10-2009
126,01-12-2010' | awk -F, '{print $1"-"$2}' | awk -F- '{print $1","$4"-"$3"-"$2}'
This outputs:
123,2006-08-01
124,2007-09-01
125,2009-10-01
126,2010-12-01
as desired.
The first awk changes the , characters to - so that you have four fields separated with the same character (this is the bit I'd usually use tr ',' '-' for).
The second awk prints them out in the order you specified, correcting the field separators at the same time.
If you're using an awk implementation that allows multiple FS characters, you can use something like:
gawk -F ',|-' '{print $1","$4"-"$3"-"$2}'
If it doesn't need to be awk, you could use Perl too:
$ perl -nle 'print "$1,$4-$3-$2" while (/(\d{3}),(\d{2})-(\d{2})-(\d{4})\s*/g)' < file.txt