grep + match exactly IP address with Regular Expression - linux
my target is to match exactly IP address with three octes , while the four IP octet must be valid octet - between <0 to 255>
For example I have the following IP's in file
$ more file
192.9.200.10
192.9.200.100
192.9.200.1555
192.9.200.1
192.9.200.aaa
192.9.200.#
192.9.200.:
192.9.200
192.9.200.
I need to match the first three octets - 192.9.200 while four octet must be valid ( 0-255)
so finally - expects result should be:
192.9.200.10
192.9.200.100
192.9.200.1
the basic syntax should be as the following:
IP_ADDRESS_THREE_OCTETS=192.9.200
cat file | grep -x $IP_ADDRESS_THREE_OCTETS.[ grep Regular Expression syntax ]
Please advice how to write the right "grep regular Expression" in the four octets in order to match the three octets , while the four octets must be valid?
You'd need to use some high-level tools to convert the text to a regex pattern, so you might as well use just that.
perl -ne'
BEGIN { $base = shift(#ARGV); }
print if /^\Q$base\E\.([0-9]+)$/ && 0 <= $1 && $1 <= 255;
' "$IP_ADDRESS_THREE_OCTETS" file
If hardcoding the base is acceptable, that reduces to:
perl -ne'print if /^192\.9\.200\.([0-9]+)$/ && 0 <= $1 && $1 <= 255' file
Both of these snippets also accept input from STDIN.
For a full IP address:
perl -ne'
BEGIN { $ip = shift(#ARGV); }
print if /^\Q$ip\E$/;
' 1.2.3.4 file
or
perl -nle'
BEGIN { $ip = shift(#ARGV); }
print if $_ eq $ip;
' 1.2.3.4 file
Regexp is not good for comparing numbers, I'd do this with awk:
$ awk -F. '$1==192 && $2==9 && $3==200 && $4>=0 && $4<=255 && NF==4' file
192.9.200.10
192.9.200.100
192.9.200.1
If you really want to use grep you need the -E flag for extended regexp or use egrep because you need alternation:
$ grep -Ex '192\.9\.200\.(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9][0-9]|[0-9])' file
192.9.200.10
192.9.200.100
192.9.200.1
$ IP=192\.9\.200\.
$ grep -Ex "$IP(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9][0-9]|[0-9])" file
Note: You must escaped . to mean a literal period.
grep -E '^((25[0-5]|2[0-4][0-9]|[1]?[1-9][0-9]?).){3}(25[0-5]|2[0-4][0-9]|[1]?[1-9]?[0-9])$'
This expression will not match IP addresses with leading 0s. e.g., it won't match 192.168.1.01
This expression will not match IP addresses with more than 4 octets. e.g., it won't match 192.168.1.2.3
If you really want to be certain that what you have is a valid IPv4 address, you can always check the return value of inet_aton() (part of the Socket core module).
Related
How can I truncate a line of text longer than a given length?
How would you go about removing everything after x number of characters? For example, cut everything after 15 characters and add ... to it. This is an example sentence should turn into This is an exam...
GnuTools head can use chars rather than lines: head -c 15 <<<'This is an example sentence' Although consider that head -c only deals with bytes, so this is incompatible with multi-bytes characters like UTF-8 umlaut ü. Bash built-in string indexing works: str='This is an example sentence' echo "${str:0:15}" Output: This is an exam And finally something that works with ksh, dash, zsh…: printf '%.15s\n' 'This is an example sentence' Even programmatically: n=15 printf '%.*s\n' $n 'This is an example sentence' If you are using Bash, you can directly assign the output of printf to a variable and save a sub-shell call with: trim_length=15 full_string='This is an example sentence' printf -v trimmed_string '%.*s' $trim_length "$full_string"
Use sed: echo 'some long string value' | sed 's/\(.\{15\}\).*/\1.../' Output: some long strin... This solution has the advantage that short strings do not get the ... tail added: echo 'short string' | sed 's/\(.\{15\}\).*/\1.../' Output: short string So it's one solution for all sized outputs.
Use cut: echo "This is an example sentence" | cut -c1-15 This is an exam This includes characters (to handle multi-byte chars) 1-15, c.f. cut(1) -b, --bytes=LIST select only these bytes -c, --characters=LIST select only these characters
Awk can also accomplish this: $ echo 'some long string value' | awk '{print substr($0, 1, 15) "..."}' some long strin... In awk, $0 is the current line. substr($0, 1, 15) extracts characters 1 through 15 from $0. The trailing "..." appends three dots.
Todd actually has a good answer however I chose to change it up a little to make the function better and remove unnecessary parts :p trim() { if (( "${#1}" > "$2" )); then echo "${1:0:$2}$3" else echo "$1" fi } In this version the appended text on longer string are chosen by the third argument, the max length is chosen by the second argument and the text itself is chosen by the first argument. No need for variables :)
Using Bash Shell Expansions (No External Commands) If you don't care about shell portability, you can do this entirely within Bash using a number of different shell expansions in the printf builtin. This avoids shelling out to external commands. For example: trim () { local str ellipsis_utf8 local -i maxlen # use explaining variables; avoid magic numbers str="$*" maxlen="15" ellipsis_utf8=$'\u2026' # only truncate $str when longer than $maxlen if (( "${#str}" > "$maxlen" )); then printf "%s%s\n" "${str:0:$maxlen}" "${ellipsis_utf8}" else printf "%s\n" "$str" fi } trim "This is an example sentence." # This is an exam… trim "Short sentence." # Short sentence. trim "-n Flag-like strings." # Flag-like strin… trim "With interstitial -E flag." # With interstiti… You can also loop through an entire file this way. Given a file containing the same sentences above (one per line), you can use the read builtin's default REPLY variable as follows: while read; do trim "$REPLY" done < example.txt Whether or not this approach is faster or easier to read is debatable, but it's 100% Bash and executes without forks or subshells.
Extract orders and match to trades from two files
I have two attached files (orders1.txt and trades1.txt) I need to write a Bash script (possibly awk?) to extract orders and match them to trades. The output should produce a report that prints comma separated values containing “ClientID, OrderID, Price, Volume”. In addition to this for each client, I need to print the total volume and turnover (turnover is the subtotal of price * volume on each trade). Can someone please help me with a bash script that will do the above using the attached files? Any help would be greatly appreciated orders1.txt Entry Time, Client ID, Security ID, Order ID 25455410,DOLR,XGXUa,DOLR1435804437 25455410,XFKD,BUP3d,XFKD4746464646 25455413,QOXA,AIDl,QOXA7176202067 25455415,QOXA,IRUXb,QOXA6580494597 25455417,YXKH,OBWQs,YXKH4575139017 25455420,JBDX,BKNs,JBDX6760353333 25455428,DOLR,AOAb,DOLR9093170513 25455429,JBDX,QMP1Sh,JBDX2756804453 25455431,QOXA,QIP1Sh,QOXA6563975285 25455434,QOXA,XMUp,QOXA5569701531 25455437,XFKD,QLOJc,XFKD8793976660 25455438,YXKH,MRPp,YXKH2329856527 25455442,JBDX,YBPu,JBDX0100506066 25455450,QOXA,BUPYd,QOXA5832015401 25455451,QOXA,SIOQz,QOXA3909507967 25455451,DOLR,KID1Sh,DOLR2262067037 25455454,DOLR,JJHi,DOLR9923665017 25455461,YXKH,KBAPBa,YXKH2637373848 25455466,DOLR,EPYp,DOLR8639062962 25455468,DOLR,UQXKz,DOLR4349482234 25455474,JBDX,EFNs,JBDX7268036859 25455481,QOXA,XCB1Sh,QOXA4105943392 25455486,YXKH,XBAFp,YXKH0242733672 25455493,JBDX,BIF1Sh,JBDX2840241688 25455500,DOLR,QSOYp,DOLR6265839896 25455503,YXKH,IIYz,YXKH8505951163 25455504,YXKH,ZOIXp,YXKH2185348861 25455513,YXKH,MBOOp,YXKH4095442568 25455515,JBDX,P35p,JBDX9945514579 25455524,QOXA,YXOKz,QOXA1900595629 25455528,JBDX,XEQl,JBDX0126452783 25455528,XFKD,FJJMp,XFKD4392227425 25455535,QOXA,EZIp,QOXA4277118682 25455543,QOXA,YBPFa,QOXA6510879584 25455551,JBDX,EAMp,JBDX8924251479 25455552,QOXA,JXIQp,QOXA4360008399 25455554,DOLR,LISXPh,DOLR1853653280 25455557,XFKD,LOX14p,XFKD1759342196 25455558,JBDX,YXYb,JBDX8177118129 25455567,YXKH,MZQKl,YXKH6485420018 25455569,JBDX,ZPIMz,JBDX2010952336 25455573,JBDX,COPe,JBDX1612537068 25455582,JBDX,HFKAp,JBDX2409813753 25455589,QOXA,XFKm,QOXA9692126523 25455593,XFKD,OFYp,XFKD8556940415 25455601,XFKD,FKQLb,XFKD4861992028 25455606,JBDX,RIASp,JBDX0262502677 25455608,DOLR,HRKKz,DOLR1739013513 25455615,DOLR,ZZXp,DOLR6727725911 25455623,JBDX,CKQPp,JBDX2587184235 25455630,YXKH,ZLQQp,YXKH6492126889 25455632,QOXA,ORPz,QOXA3594333316 25455640,XFKD,HPIXSh,XFKD6780729432 25455648,QOXA,ABOJe,QOXA6661411952 25455654,XFKD,YLIp,XFKD6374702721 25455654,DOLR,BCFp,DOLR8012564477 25455658,JBDX,ZMDKz,JBDX6885176695 25455665,JBDX,CBOe,JBDX8942732453 25455670,JBDX,FRHMl,JBDX5424320405 25455679,DOLR,YFJm,DOLR8212353717 25455680,XFKD,XAFp,XFKD4132890550 25455681,YXKH,PBIBOp,YXKH6106504736 25455684,DOLR,IFDu,DOLR8034515043 25455687,JBDX,JACe,JBDX8243949318 25455688,JBDX,ZFZKz,JBDX0866225752 25455693,QOXA,XOBm,QOXA5011416607 25455694,QOXA,IDQe,QOXA7608439570 25455698,JBDX,YBIDb,JBDX8727773702 25455705,YXKH,MXOp,YXKH7747780955 25455710,YXKH,PBZRYs,YXKH7353828884 25455719,QOXA,QFDb,QOXA2477859437 25455720,XFKD,PZARp,XFKD4995735686 25455722,JBDX,ZLKKb,JBDX3564523161 25455730,XFKD,QFH1Sh,XFKD6181225566 25455733,JBDX,KWVJYc,JBDX7013108210 25455733,YXKH,ZQI1Sh,YXKH7095815077 25455739,YXKH,XIJp,YXKH0497248757 25455739,YXKH,ZXJp,YXKH5848658513 25455747,JBDX,XASd,JBDX4986246117 25455751,XFKD,XQIKz,XFKD5919379575 25455760,JBDX,IBXPb,JBDX8168710376 25455763,XFKD,EVAOi,XFKD8175209012 25455765,XFKD,JXKp,XFKD2750952933 25455773,XFKD,PTBAXs,XFKD8139382011 25455778,QOXA,XJp,QOXA8227838196 25455783,QOXA,CYBIp,QOXA2072297264 25455792,JBDX,PZI1Sh,JBDX7022115629 25455792,XFKD,XIKQl,XFKD6434550362 25455792,DOLR,YKPm,DOLR6394606248 25455796,QOXA,JXOXPh,QOXA9672544909 25455797,YXKH,YIWm,YXKH5946342983 25455803,YXKH,JZEm,YXKH5317189370 25455810,QOXA,OBMFz,QOXA0985316706 25455810,QOXA,DAJPp,QOXA6105975858 25455810,JBDX,FBBJl,JBDX1316207043 25455819,XFKD,YXKm,XFKD6946276671 25455821,YXKH,UIAUs,YXKH6010226371 25455828,DOLR,PTJXs,DOLR1387517499 25455836,DOLR,DCEi,DOLR3854078054 25455845,YXKH,NYQe,YXKH3727923537 25455853,XFKD,TAEc,XFKD5377097556 25455858,XFKD,LMBOXo,XFKD4452678489 25455858,XFKD,AIQXp,XFKD5727938304 trades1.txt # The first 8 characters is execution time in microseconds since midnight # The next 14 characters is the order ID # The next 8 characters is the zero padded price # The next 8 characters is the zero padded volume 25455416QOXA6580494597 0000013800001856 25455428JBDX6760353333 0000007000002458 25455434DOLR9093170513 0000000400003832 25455435QOXA6563975285 0000034700009428 25455449QOXA5569701531 0000007500009023 25455447YXKH2329856527 0000038300009947 25455451QOXA5832015401 0000039900006432 25455454QOXA3909507967 0000026900001847 25455456DOLR2262067037 0000034700002732 25455471YXKH2637373848 0000010900006105 25455480DOLR8639062962 0000027500001975 25455488JBDX7268036859 0000005200004986 25455505JBDX2840241688 0000037900002029 25455521YXKH4095442568 0000046400002150 25455515JBDX9945514579 0000040800005904 25455535QOXA1900595629 0000015200006866 25455533JBDX0126452783 0000001700006615 25455542XFKD4392227425 0000035500009948 25455570XFKD1759342196 0000025700007816 25455574JBDX8177118129 0000022400000427 25455567YXKH6485420018 0000039000008327 25455573JBDX1612537068 0000013700001422 25455584JBDX2409813753 0000016600003588 25455603XFKD4861992028 0000017600004552 25455611JBDX0262502677 0000007900003235 25455625JBDX2587184235 0000024300006723 25455658XFKD6374702721 0000046400009451 25455673JBDX6885176695 0000010900009258 25455671JBDX5424320405 0000005400003618 25455679DOLR8212353717 0000041100003633 25455697QOXA5011416607 0000018800007376 25455696QOXA7608439570 0000013000007463 25455716YXKH7747780955 0000037000006357 25455719QOXA2477859437 0000039300009840 25455723XFKD4995735686 0000045500009858 25455727JBDX3564523161 0000021300000639 25455742YXKH7095815077 0000023000003945 25455739YXKH5848658513 0000042700002084 25455766XFKD5919379575 0000022200003603 25455777XFKD8175209012 0000033300006350 25455788XFKD8139382011 0000034500007461 25455793QOXA8227838196 0000011600007081 25455784QOXA2072297264 0000017000004429 25455800XFKD6434550362 0000030000002409 25455801QOXA9672544909 0000039600001033 25455815QOXA6105975858 0000034800008373 25455814JBDX1316207043 0000026500005237 25455831YXKH6010226371 0000011400004945 25455838DOLR1387517499 0000046200006129 25455847YXKH3727923537 0000037400008061 25455873XFKD5727938304 0000048700007298 I have the following script: ''' #!/bin/bash declare -A volumes declare -A turnovers declare -A orders # Read the first file, remembering for each order the client id while read -r line do # Jump over comments if [[ ${line:0:1} == "#" ]] ; then continue; fi; details=($(echo $line | tr ',' " ")) order_id=${details[3]} client_id=${details[1]} orders[$order_id]=$client_id done < $1 echo "ClientID,OrderID,Price,Volume" while read -r line do # Jump over comments if [[ ${line:0:1} == "#" ]] ; then continue; fi; order_id=$(echo ${line:8:20} | tr -d '[:space:]') client_id=${orders[$order_id]} price=${line:28:8} volume=${line: -8} echo "$client_id,$order_id,$price,$volume" price=$(echo $price | awk '{printf "%d", $0}') volume=$(echo $volume | awk '{printf "%d", $0}') order_turnover=$(($price*$volume)) old_turnover=${turnovers[$client_id]} [[ -z "$old_turnover" ]] && old_turnover=0 total_turnover=$(($old_turnover+$order_turnover)) turnovers[$client_id]=$total_turnover old_volumes=${volumes[$client_id]} [[ -z "$old_volumes" ]] && old_volumes=0 total_volume=$((old_volumes+volume)) volumes[$client_id]=$total_volume done < $2 echo "ClientID,Volume,Turnover" for client_id in ${!volumes[#]} do volume=${volumes[$client_id]} turnover=${turnovers[$client_id]} echo "$client_id,$volume,$turnover" done Can anyone think of anything more elegant? Thanks in advance C
Assumption 1: the two files are ordered, so line x represents an action that is older than x+1. If not, then further work is needed. The assumption makes our work easier. Let's first change the delimiter of traders into a comma: sed -i 's/ /,/g' traders.txt This will be done in place for sake of simplicity. So, you now have traders which is comma separated, as is orders. This is the Assumption 2. Keep working on traders: split all columns and add titles1. More on the reasons why in a moment. gawk -i inplace -v INPLACE_SUFFIX=.bak 'BEGINFILE{FS=",";OFS=",";print "execution time,order ID,price,volume";}{print substr($1,1,8),substr($1,9),substr($2,1,9),substr($2,9)}' traders.txt Ugly but works. Now let's process your data using the following awk script: BEGIN { FS="," OFS="," } { if (1 == NR) { getline line < TRADERS # consume title line print "Client ID,Order ID,Price,Volume,Turnover"; # consume title line. Remove print to forget it getline line < TRADERS # reads first data line split(line, transaction, ",") next } if (transaction[2] == $4) { print $2, $4, transaction[3], transaction[4], transaction[3]*transaction[4] getline line < TRADERS # reads new data line split(line, transaction, ",") } } called by: gawk -f script -v TRADERS=traders.txt orders.txt And there you have it. Some caveats: check the numbers, as implicit gawk number conversion might not be correct with zero-padded numbers. There is a fix for that in case; getline might explode if we run out of lines from traders. I haven't put any check, that's up to you no control over timestamps. Match is based on Order ID. Output file: Client ID,Order ID,Price,Volume,Turnover QOXA,QOXA6580494597,000001380,00001856,2561280 JBDX,JBDX6760353333,000000700,00002458,1720600 DOLR,DOLR9093170513,000000040,00003832,153280 QOXA,QOXA6563975285,000003470,00009428,32715160 QOXA,QOXA5569701531,000000750,00009023,6767250 YXKH,YXKH2329856527,000003830,00009947,38097010 QOXA,QOXA5832015401,000003990,00006432,25663680 QOXA,QOXA3909507967,000002690,00001847,4968430 DOLR,DOLR2262067037,000003470,00002732,9480040 YXKH,YXKH2637373848,000001090,00006105,6654450 DOLR,DOLR8639062962,000002750,00001975,5431250 JBDX,JBDX7268036859,000000520,00004986,2592720 JBDX,JBDX2840241688,000003790,00002029,7689910 YXKH,YXKH4095442568,000004640,00002150,9976000 JBDX,JBDX9945514579,000004080,00005904,24088320 QOXA,QOXA1900595629,000001520,00006866,10436320 JBDX,JBDX0126452783,000000170,00006615,1124550 XFKD,XFKD4392227425,000003550,00009948,35315400 XFKD,XFKD1759342196,000002570,00007816,20087120 JBDX,JBDX8177118129,000002240,00000427,956480 YXKH,YXKH6485420018,000003900,00008327,32475300 JBDX,JBDX1612537068,000001370,00001422,1948140 JBDX,JBDX2409813753,000001660,00003588,5956080 XFKD,XFKD4861992028,000001760,00004552,8011520 JBDX,JBDX0262502677,000000790,00003235,2555650 JBDX,JBDX2587184235,000002430,00006723,16336890 XFKD,XFKD6374702721,000004640,00009451,43852640 JBDX,JBDX6885176695,000001090,00009258,10091220 JBDX,JBDX5424320405,000000540,00003618,1953720 DOLR,DOLR8212353717,000004110,00003633,14931630 QOXA,QOXA5011416607,000001880,00007376,13866880 QOXA,QOXA7608439570,000001300,00007463,9701900 YXKH,YXKH7747780955,000003700,00006357,23520900 QOXA,QOXA2477859437,000003930,00009840,38671200 XFKD,XFKD4995735686,000004550,00009858,44853900 JBDX,JBDX3564523161,000002130,00000639,1361070 YXKH,YXKH7095815077,000002300,00003945,9073500 YXKH,YXKH5848658513,000004270,00002084,8898680 XFKD,XFKD5919379575,000002220,00003603,7998660 XFKD,XFKD8175209012,000003330,00006350,21145500 XFKD,XFKD8139382011,000003450,00007461,25740450 QOXA,QOXA8227838196,000001160,00007081,8213960 QOXA,QOXA2072297264,000001700,00004429,7529300 XFKD,XFKD6434550362,000003000,00002409,7227000 QOXA,QOXA9672544909,000003960,00001033,4090680 QOXA,QOXA6105975858,000003480,00008373,29138040 JBDX,JBDX1316207043,000002650,00005237,13878050 YXKH,YXKH6010226371,000001140,00004945,5637300 DOLR,DOLR1387517499,000004620,00006129,28315980 YXKH,YXKH3727923537,000003740,00008061,30148140 XFKD,XFKD5727938304,000004870,00007298,35541260 1: requires gawk 4.1.0 or higher
How to pass quoted arguments but with blank spaces in linux
I have a file with these arguments and their values this way # parameters.txt VAR1 001 VAR2 aaa VAR3 'Hello World' and another file to configure like this # example.conf VAR1 = 020 VAR2 = kab VAR3 = '' when I want to get the values in a function I use this command while read p; do VALUE=$(echo $p | awk '{print $2}') done < parameters.txt the firsts arguments throw the right values, but the last one just gets the 'Hello for the blank space, my question is how do I get the entire 'Hello World' value?
If you can use bash, there is no need to use awk: read and shell parameter expansion can be combined to solve your problem: while read -r name rest; do # Drop the '= ' part, if present. [[ $rest == '= '* ]] && value=${rest:2} || value=$rest # $value now contains the line's value, # but *including* any enclosing ' chars, if any. # Assuming that there are no *embedded* ' chars., you can remove them # as follows: value=${value//\'/} done < parameters.txt read by default also breaks a line into fields by whitespace, like awk, but unlike awk it has the ability to assign the remainder of the line to a varaible, namely the last one, if fewer variables than fields found are specified; read's -r option is generally worth specifying to avoid unexpected interpretation of \ chars. in the input. As for your solution attempt: awk doesn't know about quoting in input - by default it breaks input into fields by whitespace, irrespective of quotation marks. Thus, a string such as 'Hello World' is simply broken into fields 'Hello and World'. However, in your case you can split each input line into its key and value using a carefully crafted FS value (FS is the input field separator, which can be also be set via option -F; the command again assumes bash, this time for use of <(...), a so-called process substitution, and $'...', an ANSI C-quoted string): while IFS= read -r value; do # Work with $value... done < <(awk -F$'^[[:alnum:]]+ (= )?\'?|\'' '{ print $2 }' parameters.txt) Again the assumption is that values contain no embedded ' instances. Field separator regex $'^[[:alnum:]]+ (= )?\'?|\'' splits each line so that $2, the 2nd field, contains the value, stripped of enclosing ' chars., if any. xargs is the rare exception among the standard utilities in that it does understand single- and double-quoted strings (yet also without support for embedded quotes). Thus, you could take advantage of xargs' ability to implicitly strip enclosing quotes when it passes arguments to the specified command, which defaults to echo (again assumes bash): while read -r name rest; do # Drop the '= ' part, if present. [[ $rest == '= '* ]] && value=${rest:2} || value=$rest # $value now contains the line's value, strippe of any enclosing # single quotes by `xargs`. done < <(xargs -L1 < parameters.txt) xargs -L1 process one (1) line (-L) at a time and implicitly invokes echo with all tokens found on each line, with any enclosing quotes removed from the individual tokens.
The default field separator in awk is the space. So you are only printing the first word in the string passed to awk. You can specify the field separator on the command line with -F[field separator] Example, setting the field separator to a comma: $ echo "Hello World" | awk -F, '{print $1}' Hello World
Count total number of pattern between two pattern (using sed if possible) in Linux
I have to count all '=' between two pattern i.e '{' and '}' Sample: { 100="1"; 101="2"; 102="3"; }; { 104="1,2,3"; }; { 105="1,2,3"; }; Expected Output: 3 1 1
A very cryptic perl answer: perl -nE 's/\{(.*?)\}/ say ($1 =~ tr{=}{=}) /ge' The tr function returns the number of characters transliterated. With the new requirements, we can make a couple of small changes: perl -0777 -nE 's/\{(.*?)\}/ say ($1 =~ tr{=}{=}) /ges' -0777 reads the entire file/stream into a single string the s flag to the s/// function allows . to handle newlines like a plain character.
Perl to the rescue: perl -lne '$c = 0; $c += ("$1" =~ tr/=//) while /\{(.*?)\}/g; print $c' < input -n reads the input line by line -l adds a newline to each print /\{(.*?)\}/g is a regular expression. The ? makes the asterisk frugal, i.e. matching the shortest possible string. The (...) parentheses create a capture group, refered to as $1. tr is normally used to transliterate (i.e. replace one character by another), but here it just counts the number of equal signs. += adds the number to $c.
Awk is here too grep -o '{[^}]\+}'|awk -v FS='=' '{print NF-1}' example echo '{100="1";101="2";102="3";}; {104="1,2,3";}; {105="1,2,3";};'|grep -o '{[^}]\+}'|awk -v FS='=' '{print NF-1}' output 3 1 1
First some test input (a line with a = outside the curly brackets and inside the content, one without brackets and one with only 2 brackets) echo '== {100="1";101="2";102="3=3=3=3";} =; a=b {c=d} {}' Handle line without brackets (put a dummy char so you will not end up with an empty string) sed -e 's/^[^{]*$/x/' Handle line without equal sign (put a dummy char so you will not end up with an empty string) sed -e 's/{[^=]*}/x/' Remove stuff outside the brackets sed -e 's/.*{\(.*\)}/\1/' Remove stuff inside the double quotes (do not count fields there) sed -e 's/"[^"]*"//g' Use #repzero method to count equal signs awk -F "=" '{print NF-1}' Combine stuff echo -e '{100="1";101="2";102="3";};\na=b\n{c=d}\n{}' | sed -e 's/^[^{]*$/x/' -e 's/{[^=]*}/x/' -e 's/.*{\(.*\)}/\1/' -e 's/"[^"]*"//g' | awk -F "=" '{print NF-1}' The ugly temp fields x and replacing {} can be solved inside awk: echo -e '= {100="1";101="2=2=2=2";102="3";};\na=b\n{c=d}\n{}' | sed -e 's/^[^{]*$//' -e 's/.*{\(.*\)}/\1/' -e 's/"[^"]*"//g' | awk -F "=" '{if (NF>0) c=NF-1; else c=0; print c}' or shorter echo -e '= {100="1";101="2=2=2=2";102="3";};\na=b\n{c=d}\n{}' | sed -e 's/^[^{]*$//' -e 's/.*{\(.*\)}/\1/' -e 's/"[^"]*"//g' | awk -F "=" '{print (NF>0) ? NF-1 : 0; }'
No harder sed than done ... in. Restricting this answer to the environment as tagged, namely: linux shell unix sed wc will actually not require the use of wc (or awk, perl, or any other app.). Though echo is used, a file source can easily exclude its use. As for bash, it is the shell. The actual environment used is documented at the end. NB. Exploitation of GNU specific extensions has been used for brevity but appropriately annotated to make a more generic implementation. Also brace bracketed { text } will not include braces in the text. It is implicit that such braces should be present as {} pairs but the text src. dangling brace does not directly violate this tenet. This is a foray into the world of `sed`'ng to gain some fluency in it's use for other purposes. The ideas expounded upon here are used to cross pollinate another SO problem solution in order to aquire more familiarity with vetting vagaries of vernacular version variances. Consequently this pedantic exercice hopefully helps with the pedagogy of others beyond personal edification. To test easily, at least in the environment noted below, judiciously highlight the appropriate code section, carefully excluding a dangling pipe |, and then, to a CLI command line interface drag & drop, copy & paste or use middle click to enter the code. The other SO problem. linux - Is it possible to do simple arithmetic in sed addresses? # _______________________________ always needed ________________________________ echo -e '\n \n = = = {\n } = = = each = is outside the braces \na\nb\n { } so therefore are not counted \nc\n { = = = = = = = } while the ones here do count {\n100="1";\n101="2";\n102="3";\n}; \n {\n104="1,2,3";\n}; a\nb\nc\n {\n105="1,2,3";\n}; { dangling brace ignored junk = = = \n' | # _____________ prepatory conditioning needed for final solutions _____________ sed ' s/{/\n{\n/g; s/}/\n}\n/g; ' | # guarantee but one brace to a line sed -n '/{/ h; # so sed addressing can "work" here /{/,/}/ H; # use hHold buffer for only { ... } /}/ { x; s/[^=]*//g; p } ' | # then make each {} set a line of = # ____ stop code hi-lite selection in ^--^ here include quote not pipe ____ # ____ outputs the following exclusive of the shell " # " comment quotes _____ # # # ======= # === # = # = # _________________________________________________________________________ # ____________________________ "simple" GNU solution ____________________________ sed -e '/^$/ { s//0/;b }; # handle null data as 0 case: next! s/=/\n/g; # to easily count an = make it a nl s/\n$//g; # echo adds an extra nl - delete it s/.*/echo "&" | sed -n $=/; # sed = command w/ $ counts last nl e ' # who knew only GNU say you ah phoo # 0 # 0 # 7 # 3 # 1 # 1 # _________________________________________________________________________ # ________________________ generic incomplete "solution" ________________________ sed -e '/^$/ { s//echo 0/;b }; # handle null data as 0 case: next! s/=$//g; # echo adds an extra nl - delete it s/=/\\\\n/g; # to easily count an = make it a nl s/.*/echo -e & | sed -n $=/; ' # _______________________________________________________________________________ The paradigm used for the algorithm is instigated by the prolegomena study below. The idea is to isolate groups of = signs between { } braces for counting. These are found and each group is put on a separate line with ALL other adorning characters removed. It is noted that sed can easily "count", actually enumerate, nl or \n line ends via =. The first "solution" uses these sed commands: print branch w/o label starts a new cycle h/Hold for filling this sed buffer exchanage to swap the hold and pattern buffers = to enumerate the current sed input line substitute s/.../.../; with global flag s/.../.../g; and most particularly the GNU specific evaluate (execute can not remember the actual mnemonic but irrelevantly synonymous) The GNU specific execute command is avoided in the generic code. It does not print the answer but instead produces code that will print the answer. Run it to observe. To fully automate this, many mechanisms can be used not the least of which is the sed write command to put these lines in a shell file to be excuted or even embed the output in bash evaluation parentheses $( ) etc. Note also that various sed example scripts can "count" and these too can be used efficaciously. The interested reader can entertain these other pursuits. prolegomena: concept from counting # of lines between braces sed -n '/{/=;/}/=;' to sed -n '/}/=;/{/=;' | sed -n 'h;n;G;s/\n/ - /; 2s/^/ Between sets of {} \n the nl # count is\n /; 2!s/^/ /; p' testing "done in": linuxuser#ubuntu:~$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.2 LTS Release: 18.04 Codename: bionic linuxuser#ubuntu:~$ sed --version -----> sed (GNU sed) 4.4
And for giggles an awk-only alternative: echo '{ > 100="1"; > 101="2"; > 102="3"; > }; > { > 104="1,2,3"; > }; > { > 105="1,2,3"; > };' | awk 'BEGIN{RS="\n};";FS="\n"}{c=gsub(/=/,""); if(NF>2){print c}}' 3 1 1
perl + Replace IP address only if octet is VALID IP [duplicate]
This question already has answers here: grep + match exactly IP address with Regular Expression (4 answers) Closed 9 years ago. The target of the following perl one liner code is to replace the first three octets ( in case the four octet is digit/number - xxx.xxx.xxx.digit ) remark - I use linux and solaris machines The problem is that the perl one liner will also replace the first three octets while the four octet IP is not valid IP octet ( for example 5.5.5.555 ) The following perl one liner code example show how the perl syntax replaced the first three octets in spite the four octet isn’t VALID IP # export OLD_IP=1.1.1 # export NEW_IP=5.5.5 # echo 1.1.1.555 | perl -i -pe 'next if /^ *#/; s/(?<![\d.])\Q$ENV{OLD_IP}\E(?=\.\d)/$ENV{NEW_IP}/g' 5.5.5.555 Please advice what need to add in my perl one liner code , in order to replace the first three octets only if four octet is VALID IP ( between 0 – 255 )
If the final octet is always just a single digit then you can do the same as at the start of the pattern and ensure that there is no digit after the first. s/(?<![\d.])\Q$ENV{OLD_IP}\E(?=\.\d(?!\d))/$ENV{NEW_IP}/g This way is much easier than checking for a valid final octet from 0 to 255, which would look like s/(?<![\d.])\Q$ENV{OLD_IP}\E(?=\.(?:1?\d?\d|2[0-4][0-9]|25[0-5])(?!\d))/$ENV{NEW_IP}/g
My first thought is to use a regular expression: Next expresion should get you started, it checks for different groups, single digits 0 to 9, double digits 0 to 9, three digits starting with 1, three digits starting with 2 but not followed by 5, three digits followed by 5 restricted to 5 for the third digit ^(?:[23456789]|[0123456789][0123456789]|1[0123456789][0123456789]|2[01234][0123456789]|25[012345]?)$ did test it in an online regex tester and it seems to filter it nicely my knowledge of PERL has been dorment for over twelve years, but shouldn't it be possible to get the output and check that for value: /\d\.\d\.\d\.(\d)/ if $1 < 255
Here is a simple solution using the eval flag for substitution: perl -i -pe 's/\b(1\.1\.1\.(\d+))\b/ $2 >= 0 && $2 < 255 ? "5.5.5.$2" : $1/ge' Test: echo 1.1.1.12 | perl -i -pe 's/\b(1\.1\.1\.(\d+))\b/ $2 >= 0 && $2 < 255 ? "5.5.5.$2" : $1/ge' 5.5.5.12 echo 1.2.1.256 | perl -i -pe 's/\b(1\.1\.1\.(\d+))\b/ $2 >= 0 && $2 <= 255 ? "5.5.5.$2" : $1/ge' 1.2.1.256