I have a text file where there is a date string of "2014-06-01T03:11:00Z " in every line. I would like to replace that with "2014-06-01 03:11Z " using sed.
I've been trying to use this code but, it's failing me:
sed -i 's/[0-9]-[0-9]-[0-9]T[0-9]:[0-9]:[0-9]Z/[0-9]-[0-9]-[0-9] [0-9]:[0-9]Z/g' \
/home/aaron/grads/data/metars/${YMD}/latest.metars
Your digit sub-expressions only match a single digit, but the date contains 2 or 4 digits. A simple version that would match dates is:
sed -i 's/\([0-9]*-[0-9]*-[0-9]*\)T\([0-9]*:[0-9]*\):[0-9]*Z/\1 \2Z/g' \
/home/aaron/grads/data/metars/${YMD}/latest.metars
However, this matches zero or more digits at each position where digits are expected. You really want to insist on the correct number of digits in each segment. A more refined version is:
sed -i 's/\([0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\}\)T\([0-9]\{2\}:[0-9]\{2\}\):[0-9]\{2\}Z/\1 \2Z/g' \
/home/aaron/grads/data/metars/${YMD}/latest.metars
And since your sed supports -i without specifying a back-up suffix (so it is probably GNU sed), you can probably abbreviate that to:
sed -r -i 's/([0-9]{4}-[0-9]{2}-[0-9]{2})T([0-9]{2}:[0-9]{2}):[0-9]{2}Z/\1 \2Z/g' \
/home/aaron/grads/data/metars/${YMD}/latest.metars
Try this GNU sed command to replace all the lines which contains the date string with the string you mentioned,
sed -ri 's/^.*([0-9]{4})-([0-9]{2})-([0-9]{2})\w*([0-9]{2}):([0-9]{2}):([0-9]{2})(.)(.*)$/\1-\2-\3 \4:\5\7/g' file
Example:
$ cat aa
jgklj 2014-06-01T03:11:00Z jhgkjhvk
blaf 2015-12-08T03:15:02Z bvcjghj
$ sed -r 's/^.*([0-9]{4})-([0-9]{2})-([0-9]{2})\w*([0-9]{2}):([0-9]{2}):([0-9]{2})(.)(.*)$/\1-\2-\3 \4:\5\7/g' aa
2014-06-01 03:11Z
2015-12-08 03:15Z
For to replace date only and print all the other text as it is then run the below command.
sed -ri 's/^(.*)([0-9]{4})-([0-9]{2})-([0-9]{2})\w*([0-9]{2}):([0-9]{2}):([0-9]{2})(.)(.*)$/\1\2-\3-\5 \5:\6\8\9/g' file
Example:
$ cat aa
jgklj 2014-06-01T03:11:00Z jhgkjhvk
blaf 2015-12-08T03:15:02Z bvcjghj
$ sed -r 's/^(.*)([0-9]{4})-([0-9]{2})-([0-9]{2})\w*([0-9]{2}):([0-9]{2}):([0-9]{2})(.)(.*)$/\1\2-\3-\5 \5:\6\8\9/g' aa
jgklj 2014-06-03 03:11Z jhgkjhvk
blaf 2015-12-03 03:15Z bvcjghj
You can use this method also
$-sed -r 's/^([^T]+).((.*):){1,2}.([^Z])/\1 \3/g'
Related
I have the following csv file:
id,"path",score,"file"
1,"/tmp/file 1.csv",5,"file 1.csv"
2,"/tmp/file2.csv",15,"file2.csv"
I want to convert it to:
"id","path","score","file"
"1","/tmp/file 1.csv","5","file 1.csv"
"2","/tmp/file2.csv","15","file2.csv"
How can I do it using sed/awk or any another linux tool?
Assuming that you want to quote all entries, coma is separator, and there are no white spaces between separator and entry (this one can be solved as well but for brevity I didn't include it).
$ cat csv1 | sed -e 's/^/\"/' -e 's/$/\"/' -e 's/,/\",\"/g' -e 's/\"\"/\"/g' > csv2
It replaces beginning ^, end $ of line, , with " and at the end removes duplicates.
Using Miller, if you run
mlr --csv --quote-all cat input.csv >output.csv
you will have
"id","path","score","file"
"1","/tmp/file 1.csv","5","file 1.csv"
"2","/tmp/file2.csv","15","file2.csv"
I have a file with records having timestamp fields that include GMT offset. I want to use the sed command to replace the value on the record to a regular timestamp (without GMT offset).
For example:
`$date -d '2012/11/01 00:50:22 -0800' '+%Y-%m-%d %H:%M:%S'`
returns this value which is what I am looking to do:
2012-11-01 01:50:22
Except I want to perform that operation on every line of this file and apply the date command to the timestamp value. Here is a sample record:
"SB","6GV96644X48128125","","","","T0006",2012/10/03 13:08:43 -0700,"NJ"
Here is my code:
head -1 myfile | sed 's/,[0-9: /\-]\{25\},/,'"`date -d \1 '+%Y-%m-%d %H:%M:%S'`"',/
which doesn't work: it just ignores \1 and replaces the matched pattern with today's date:
"SB","6GV96644X48128125","","","","T0006",2012-11-14 01:00:00,"NJ"
I hoped that \1 would result in the matched patterns be passed to the date function and return a regular timestamp value (as in the example I provided above showing how the date functions applies the GMT offset and returns a regular stimestamp string) and would replace the old value on the record.
I would use awk instead. For example:
awk '{cmd="date -d \""$7"\" \"+%Y-%m-%d %H:%M:%S\"";
cmd | getline k; $7=k; print}' FS=, OFS=, myFile
This will replace the 7th field with the results of running the date command on the original contents of the 7th field.
In sed:
head -1 datefile |
sed '
# handle % in input correctly
s/%/%%/g
# execute date(1) command
s/\(.*,\)\([0-9: /\-]\{25\}\)\(,.*\)/'"date -d '\2' '+\1%Y-%m-%d %H:%M:%S\3'"'/e'
'
This might work for you (GNU sed):
sed -r 's/^(([^,]*,){6})([^,]*)(.*)/printf "%s%s%s" '\''\1'\'' $(date -d '\''\3'\'' '\''+%Y-%m-%d %H:%M:%S'\'') '\''\4'\''/e;q' file
use
head -1 datefile | sed -e 's?\(..\)/\(..\)/\(.... ..:..:..\)?'"date -d '\2/\1/\3' '+%s'"'?e'
My Stress.k file is as follows
180.4430
*INCLUDE
$# filename
*STRESS_INITIALIZATION
*END
I want it to be like
180.4430
*INCLUDE
$# filename
*STRESS_INITIALIZATION
*/home/hassan/534.k
*END
for that I used sed as follows
a="$(cat flow.k)"
sed -i -e '/*END/i \*/home/hassan/$a.k ' Stress.k
where flow.k has only a single number like 534.k or something . Here sed put the line before END but it doesn't take the value of a , instead it puts the same alphabet and it doesn't understand $a.k.
Please also tell me how to delete the second last line or the line with a string hassan for example so that I can delete it first and the for the next step I use it to enter my required line.
if possible please also suggest the alternatives.
best regards
bash variables are only replaced when in double quotes, e.g.
sed -i -e "/*END/i \*/home/hassan/$a.k " Stress.k
Use double quotes to allow the variable to be expanded.
sed -i -e "/*END/i \*/home/hassan/$a.k " Stress.k
To replace the string, do it as you read in the file:
a=$(sed 's/534/100/' flow.k)
To delete a line:
sed '/hassan/d' inputfile
To read a file into the stream after the current line:
sed '/foo/r filename' inputfile
I have an input file like following.
R sfst 1000.0000
$ new time step for mass scaled calculation
R dt2ms -4.000E-7
$ friction value for blank
R mue 0.120000
$ blankholder force
R bhf 2.0000E+5
$ simulation time
R endtime 0.150000
i want to change the value on the line containing 'mue'
with following I can read it but cant change it.
awk ' /mue/ { print $3 } ' input.txt
The value is to be taken from another file fric.txt.
fric.txt contains only numbers, one on each line .
fric.txt has data like
0.1234
0.234
0.0234
.
.
Blockquote
It should be noted that ONLY the FIRST instance need to be replaced and the format i.e. white spacing be kept cosntant.
Blockquote
Can anybody guide me doing this using sed or awk?
Try this command:
$ awk '/mue/ && !seen {getline $3 <"fric.txt"; seen=1} 1' input.txt
This might work for you:
sed '/\<mue\>/!d;=;s/.* \([^ ]\+\).*/\1/;R fric.txt' input.txt |
sed 'N;N;s|\n|s/|;s|\n|/|;s|$|/|;q' >temp.sed
sed -i -f temp.sed input.txt
You can do it with a sed in the sed (assuming you like to take line 1 from fric.txt):
sed -ir 's/(.*mue[ \t]+)[0-9.]+(.*)/\1'$(sed -n '1{p;q}' fric.txt)'\2/' input.txt
I hope the below task will be very easy for sed lovers. I am not sed-guru, but I need to express the following task in sed, as sed is more popular on Linux systems.
The input text stream is something which is produced by "make depends" and looks like following:
pgm2asc.o: pgm2asc.c ../include/config.h amiga.h list.h pgm2asc.h pnm.h \
output.h gocr.h unicode.h ocr1.h ocr0.h otsu.h barcode.h progress.h
box.o: box.c gocr.h pnm.h ../include/config.h unicode.h list.h pgm2asc.h \
output.h
database.o: database.c gocr.h pnm.h ../include/config.h unicode.h list.h \
pgm2asc.h output.h
detect.o: detect.c pgm2asc.h pnm.h ../include/config.h output.h gocr.h \
unicode.h list.h
I need to catch only C++ header files (i.e. ending with .h), make the list unique and print as space-separated list prepending src/ as a path-prefix. This is achieved by the following perl script:
make libs-depends | perl -e 'while (<>) { while (/ ([\w\.\/]+?\.h)/g) { $a{$1} = 1; } } print join " ", map { "src/$_" } keys %a;'
The output is:
src/unicode.h src/pnm.h src/progress.h src/amiga.h src/ocr0.h src/ocr1.h src/otsu.h src/barcode.h src/gocr.h src/../include/config.h src/list.h src/pgm2asc.h src/output.h
Please, help to express this in sed.
Not sed but hope this helps you:
make libs-depends | grep -io --perl-regexp "[\w\.\/]+\.h " | sort -u | sed -e 's:^:src/:'
If you really want to do this in pure sed:
make libs-depends | sed 's/ /\n/g' | sed '/\.h$/!d;s/^/src\//' | sed 'G;/^\(.*\)\n.*\1/!h;$!d;${x;s/\n/ /g}'
The first sed command breaks the output up into separate lines, the second filters out everything but *.h and prepends 'src/', the third gloms the lines together without repetition.
Sed probably isn't the best tool here as it's stream-oriented. You could possibly use it to convert the spaces to newlines though, pipe that through sort and uniq, then use sed again to convert the newlines back to spaces.
Typing this on my phone, though, so can't give exact commands :(