Add a number to each line of a file in bash - linux
I have some files with some lines in Linux like:
2013/08/16,name1,,5000,8761,09:00,09:30
2013/08/16,name1,,5000,9763,10:00,10:30
2013/08/16,name1,,5000,8866,11:00,11:30
2013/08/16,name1,,5000,5768,12:00,12:30
2013/08/16,name1,,5000,11764,13:00,13:30
2013/08/16,name2,,5000,2765,14:00,14:30
2013/08/16,name2,,5000,4765,15:00,15:30
2013/08/16,name2,,5000,6765,16:00,16:30
2013/08/16,name2,,5000,12765,17:00,17:30
2013/08/16,name2,,5000,25665,18:00,18:30
2013/08/16,name2,,5000,45765,09:00,10:30
2013/08/17,name1,,5000,33765,10:00,11:30
2013/08/17,name1,,5000,1765,11:00,12:30
2013/08/17,name1,,5000,34765,12:00,13:30
2013/08/17,name1,,5000,12765,13:00,14:30
2013/08/17,name2,,5000,1765,14:00,15:30
2013/08/17,name2,,5000,3765,15:00,16:30
2013/08/17,name2,,5000,7765,16:00,17:30
My column separator is "," and in the third column (currently ,,), I need the entry number within the same day. For example, with date
2013/08/16 I have 11 lines and with date 2013/08/17 7 lines, so I need add the numbers for example:
2013/08/16,name1,1,5000,8761,09:00,09:30
2013/08/16,name1,2,5000,9763,10:00,10:30
2013/08/16,name1,3,5000,8866,11:00,11:30
2013/08/16,name1,4,5000,5768,12:00,12:30
2013/08/16,name1,5,5000,11764,13:00,13:30
2013/08/16,name2,6,5000,2765,14:00,14:30
2013/08/16,name2,7,5000,4765,15:00,15:30
2013/08/16,name2,8,5000,6765,16:00,16:30
2013/08/16,name2,9,5000,12765,17:00,17:30
2013/08/16,name2,10,5000,25665,18:00,18:30
2013/08/16,name2,11,5000,45765,09:00,10:30
2013/08/17,name1,1,5000,33765,10:00,11:30
2013/08/17,name1,2,5000,1765,11:00,12:30
2013/08/17,name1,3,5000,34765,12:00,13:30
2013/08/17,name1,4,5000,12765,13:00,14:30
2013/08/17,name2,5,5000,1765,14:00,15:30
2013/08/17,name2,6,5000,3765,15:00,16:30
2013/08/17,name2,7,5000,7765,16:00,17:30
I need do it in bash. How can I do it?
This one's good too:
awk -F, 'sub(/,,/, ","++a[$1]",")1' file
Output:
2013/08/16,name1,1,5000,8761,09:00,09:30
2013/08/16,name1,2,5000,9763,10:00,10:30
2013/08/16,name1,3,5000,8866,11:00,11:30
2013/08/16,name1,4,5000,5768,12:00,12:30
2013/08/16,name1,5,5000,11764,13:00,13:30
2013/08/16,name2,6,5000,2765,14:00,14:30
2013/08/16,name2,7,5000,4765,15:00,15:30
2013/08/16,name2,8,5000,6765,16:00,16:30
2013/08/16,name2,9,5000,12765,17:00,17:30
2013/08/16,name2,10,5000,25665,18:00,18:30
2013/08/16,name2,11,5000,45765,09:00,10:30
2013/08/17,name1,1,5000,33765,10:00,11:30
2013/08/17,name1,2,5000,1765,11:00,12:30
2013/08/17,name1,3,5000,34765,12:00,13:30
2013/08/17,name1,4,5000,12765,13:00,14:30
2013/08/17,name2,5,5000,1765,14:00,15:30
2013/08/17,name2,6,5000,3765,15:00,16:30
2013/08/17,name2,7,5000,7765,16:00,17:30
Related
force linux sort to use lexicographic order
I generated a text file with pseudo-random numbers like this: -853340442 1130519212 -2070936922 -707168664 -2076185735 -2135012102 166464098 1928545126 5768715 1060168276 -684694617 395859713 -680897578 -2095893176 1457930442 299309402 192205833 1878010157 -678911642 2062673581 -1801057195 795693402 -631504846 2117889796 448959250 547707556 -1115929024 168558507 7468411 1600190097 -746131117 1557335455 73377787 -1144524558 2143073647 -2044347857 1862106004 -193937480 1596949168 -1193502513 -920620244 -365340967 -677065994 500654963 1031304603 Now I try to put it in order using linux sort command: sort prng >prngsorted The result is not what I expected: 1060168276 -684694617 395859713 -1144524558 2143073647 -2044347857 -1193502513 -920620244 -365340967 166464098 1928545126 5768715 168558507 7468411 1600190097 1862106004 -193937480 1596949168 299309402 192205833 1878010157 448959250 547707556 -1115929024 -677065994 500654963 1031304603 -678911642 2062673581 -1801057195 -680897578 -2095893176 1457930442 -707168664 -2076185735 -2135012102 -746131117 1557335455 73377787 795693402 -631504846 2117889796 -853340442 1130519212 -2070936922 Obviously, sort tries to parse strings and extract numbers for sorting. And it seems to ignore minus signs. Is it possible to force sort to be a bit dumber and just compare lines lexicographically? The result should be like this: -1144524558 2143073647 -2044347857 -1193502513 -920620244 -365340967 -677065994 500654963 1031304603 -678911642 2062673581 -1801057195 -680897578 -2095893176 1457930442 -707168664 -2076185735 -2135012102 -746131117 1557335455 73377787 -853340442 1130519212 -2070936922 1060168276 -684694617 395859713 166464098 1928545126 5768715 168558507 7468411 1600190097 1862106004 -193937480 1596949168 299309402 192205833 1878010157 448959250 547707556 -1115929024 795693402 -631504846 2117889796 Note: I tried -d option but it did not help Note 2: Probably I should use another utility instead of sort?
The sort command takes account of your locale settings. Many of the locales ignore dashes for collation. You can get appropriate sorting with LC_COLLATE=C sort filename
custom sort with the help of awk
$ awk '{print ($1<0?"-":"+") "\t" $0}' file | sort -k1,1 -k2 | cut -f2-
-1144524558 2143073647 -2044347857
-1193502513 -920620244 -365340967
-677065994 500654963 1031304603
-678911642 2062673581 -1801057195
-680897578 -2095893176 1457930442
-707168664 -2076185735 -2135012102
-746131117 1557335455 73377787
-853340442 1130519212 -2070936922
1060168276 -684694617 395859713
166464098 1928545126 5768715
168558507 7468411 1600190097
1862106004 -193937480 1596949168
299309402 192205833 1878010157
448959250 547707556 -1115929024
795693402 -631504846 2117889796
sort by sign only first, then regular sort and remove sign afterwards...
Sort list python3
I would like to order this list. From: 01104D-BB'42 01104D-BB42 01104D-BB43 01104D-CC'42 01104D-CC'72 01104D-CC32 01104D-CC42 01104D-CC62 01104D-CC72 01104D-DD'74 01104D-DD'75 01104D-DD'76 01104D-DD'77 01104D-DD'78 01104D-DD75 01104D-DD76 01104D-DD77 01104D-DD78 01104D-EE'102 01104D-EE'12 01104D-EE'2 01104D-EE'32 01104D-EE'42 01104D-EE'52 01104D-EE'53 01104D-EE'72 01104D-EE'82 01104D-EE'92 01104D-EE102 01104D-EE12 01104D-EE2 01104D-EE3 01104D-EE32 01104D-EE42 01104D-EE52 01104D-EE62 01104D-EE72 01104D-EE82 01104D-EE83 01104D-EE92 01104D-EE93 To: 01104D-BB42 01104D-BB43 01104D-BB'42 01104D-CC32 01104D-CC42 01104D-CC62 01104D-CC72 01104D-CC'42 01104D-CC'72 01104D-DD75 01104D-DD76 01104D-DD77 01104D-DD78 01104D-DD'74 01104D-DD'75 01104D-DD'76 01104D-DD'77 01104D-DD'78 01104D-EE102 01104D-EE12 01104D-EE2 01104D-EE3 01104D-EE32 01104D-EE42 01104D-EE52 01104D-EE62 01104D-EE72 01104D-EE82 01104D-EE83 01104D-EE92 01104D-EE93 01104D-EE'102 01104D-EE'12 01104D-EE'2 01104D-EE'32 01104D-EE'42 01104D-EE'52 01104D-EE'53 01104D-EE'72 01104D-EE'82 01104D-EE'92 Can you help me? thanks
I'm guessing here, because you haven't explained how you want the sort to be done. But it looks like you want the character ' to sort after the digits 0-9, and the ascii sort order puts it before the digits. If that is correct, then you need to substitute a different character for '. A good choice might be ~ because it is the last printable ascii character.
If your data is in mylist, then
mylist.sort(key=lambda a: a.replace("'","~"))
will sort it in the order I'm guessing you want.
Unable to trim the last line in the Unix file
I am trying to create and unix file with some text in it with the below command:
ssh.sendLine("echo '"+"{1:F01ZYIBGB20AXXX0000000000}{2:O5481710NDEASES0XXXX12345678901511041511180930N}\n{4:\n:16R:GENL\n:20C::SEME//"+$TradeRef+"\n:23G:INST\n:16R:LINK\n:20C::RELA//"+$TradeRef+"\n:16S:LINK\n:16R:STAT\n:25D::MTCH//MACH\n:16S:STAT\n:16S:GENL\n:16R:SETTRAN\n:35B:ISIN DE0005933931\niShares Core DAX UCITS ETF DE\n:36B::SETT//UNIT/10,\n:97A::SAFE//8696\n:22F::SETR//TRAD\n:98A::SETT//20151118\n:98A::TRAD//20151118\n:16S:SETTRAN\n-}'"+">M548File.txt");
NOw this command is createing the file M548File.txt.
When id a cat this is what i get :
{1:F01ZYIBGB20AXXX0000000000}{2:O5481710NDEASES0XXXX12345678901511041511180930N}
{4:
:16R:GENL
:20C::SEME//11111111111111111111
:23G:INST
:16R:LINK
:20C::RELA//11111111111111111111
:16S:LINK
:16R:STAT
:25D::MTCH//MACH
:16S:STAT
:16S:GENL
:16R:SETTRAN
:35B:ISIN DE0005933931
iShares Core DAX UCITS ETF DE
:36B::SETT//UNIT/10,
:97A::SAFE//8696
:22F::SETR//TRAD
:98A::SETT//20151118
:98A::TRAD//20151118
:16S:SETTRAN
-}
However when i try to open the same file in notepad i get one extraline at the last which is basically an empty line making it a total of -- 23 lines as compared to 22 in cat.
I tried the Sed commands but it just not working
Any idea how to overcome this to get 22 lines in notepad(same as cat)?
entering text in a file at specific locations by identifying the number being integer or real in linux
I have an input like below 46742 1 48276 48343 48199 48198 46744 1 48343 48344 48200 48199 46746 1 48344 48332 48201 48200 48283 3.58077402e+01 -2.97697746e+00 1.50878647e+02 48282 3.67231688e+01 -2.97771595e+00 1.50419488e+02 48285 3.58558188e+01 -1.98122787e+00 1.50894850e+02 Each segment with the 2nd entry like 1 being integer is like thousands of lines and then starts the segment with the 2nd entry being real like 3.58077402e+01 Before anything beings I have to input a text like *Revolved *Gripped *Crippled 46742 1 48276 48343 48199 48198 46744 1 48343 48344 48200 48199 46746 1 48344 48332 48201 48200 *Cracked *Crippled 48283 3.58077402e+01 -2.97697746e+00 1.50878647e+02 48282 3.67231688e+01 -2.97771595e+00 1.50419488e+02 48285 3.58558188e+01 -1.98122787e+00 1.50894850e+02 so I need to enter specific texts at those locations. It is worth mentioning that the file is space delimited and not tabs delimited and that the text starting with * has to be at the very left of the line without spacing. The format of the rest of the file should be kept too. Any suggestions with sed or awk would be highly appreaciated! The text in the beginning could entered directly so that is not a prime problem since that is the start of the file, problematic is the second bunch of line so identify that the second entry has turned to real.
An awk with fixed strings:
awk 'BEGIN{print "*Revolved\n*Gripped\n*Crippled"}
match($2,"\+")&&!pr{print "*Cracked\n*Crippled";pr=1}1' yourfile
match($2,"\+")&&!pr : When + char is found at $2 field(real number) and pr flag is null.
vim search for epoch time strings, pipe to date -d and return the date into the file
I have a file with a lot of data in it, one being a last-modified="1231231231"
where 1231231231 is epoch time in milliseconds
<Translation
author_id="25"
id="02f18edd-ef7a-48e2-b614-b5888936017e"
language="de_DE"
last_modified="1325669156960"
phase="1"
target="[ phase="1" language="de_DE" ]"
translation_text="Funktionen"/>
Note the: last_modified="1325669156960"
I can run this:
:%s/\([0-9]\{10\}\)\([0-9]\{3\}\)/\1/g
to find all these occurrences and replace them with a "seconds" string:
last_modified="1325669156"
I can then pattern match on those 10 digits, and what I'd like to do is pipe them to the unix data -d command to return a formatted data stamp:
:%s/[0-9]\{10\}/&/g
In this example, instead of replacing with the same value as I found (I.e, the &),
I'd like to somehow pipe that value to what would be essentially:
date -d &
and return that as a formatted time stamp in the
last_modified="Wed Jan 4 07:13:32 MST 2012"
Any ideas on how to do this? I have to do this about every other week on various files.
You can use strftime() in vim. Find one proper format string to meet your needs.
I'm using %c here:
:%s/last_modified="\zs\(\d\{10}\)\d\{3}/\=strftime('%c', str2nr(submatch(1)))/g
result:
<Translation
author_id="25"
id="02f18edd-ef7a-48e2-b614-b5888936017e"
language="de_DE"
last_modified="2012-1-4 17:25:56"
phase="1"
target="[ phase="1" language="de_DE" ]"
translation_text="Funktionen"/>