Extract email string from string in bash - string

I have a variable: $change.
I have tried to extract email from it (find the string between "by" and "#"):
change="Change 1234 on 2016/08/31 by name#company.com 'cdex abcd'"
email=$(echo $change|sed -e 's/\by\(.*\)#/\1/')
It did not work.

You have an escape character before b, which makes it \b. And this is a word boundary, so something you don't want here.
See the difference:
$ echo "$change" | sed -e 's/\by\(.*\)#/\1/'
# ^
Change 1234 on 2016/08/31 by name#company.com 'cdex abcd'
$ echo "$change" | sed -e 's/by\(.*\)#/\1/'
# ^
Change 1234 on 2016/08/31 namecompany.com 'cdex abcd'
# ^
# by is not here any more
But if you want to get the name, just use .* to match everything up to by:
$ echo "$change" | sed -e 's/.*by\(.*\)#/\1/'
namecompany.com 'cdex abcd'
Finally, if what you want is just the data between by (note the trailing space) and #, use either of these (with -r you don't have to escape the captured groups):
sed -e 's/.*by \(.*\)#.*/\1/'
sed -r 's/.*by (.*)#.*/\1/'
With your input:
$ sed -e 's/.*by \(.*\)#.*/\1/' <<< "Change 1234 on 2016/08/31 by name#company.com 'cdex abcd'"
name

Using grep -oP you can use match reset \K:
grep -oP ' by \K[^#]*' <<< "$change"
name
or using lookbehind:
grep -oP '(?<= by )[^#]*' <<< "$change"
name

There is no need to resort sed, awk, grep, etc. use regular expression matching:
[[ $change =~ by\ ([^#]*)# ]] && email=${BASH_REMATCH[1]}
From the man page
An additional binary operator, =~, is available, with the same
precedence as == and !=. When it is used, the string to the
right of the operator is considered an extended regular expres-
sion and matched accordingly (as in regex(3)). The return value
is 0 if the string matches the pattern, and 1 otherwise. If the
regular expression is syntactically incorrect, the conditional
expression's return value is 2. If the shell option nocasematch
is enabled, the match is performed without regard to the case of
alphabetic characters. Any part of the pattern may be quoted to
force the quoted portion to be matched as a string. Bracket
expressions in regular expressions must be treated carefully,
since normal quoting characters lose their meanings between
brackets. If the pattern is stored in a shell variable, quoting
the variable expansion forces the entire pattern to be matched
as a string. Substrings matched by parenthesized subexpressions
within the regular expression are saved in the array variable
BASH_REMATCH. The element of BASH_REMATCH with index 0 is the
portion of the string matching the entire regular expression.
The element of BASH_REMATCH with index n is the portion of the
string matching the nth parenthesized subexpression.
It might be surprising, that the pattern is written without surrounding quotes, which is why it is probably a good idea to use a variable for the pattern instead:
regex='by ([^#]*)#'
[[ $change =~ $regex ]] && email=${BASH_REMATCH[1]}

With sed:
sed -E 's/.* by ([^#]+).*/\1/' <<<"$change"
With awk:
awk -F# '{sub(".* ", "", $1); print $1}' <<<"$change"
Example:
$ sed -E 's/.* by ([^#]+).*/\1/' <<<"Change 1234 on 2016/08/31 by name#company.com 'cdex abcd'"
name
$ awk -F# '{sub(".* ", "", $1); print $1}' <<<"Change 1234 on 2016/08/31 by name#company.com 'cdex abcd'"
name

awk version, this will use awk's inbuilt split function to split 6th field using "#" as delimiter and store it in an array named a. Print it for printing first value of array a.
echo $change |awk '{ split($6,a,"#"); print a[1]}'
name
In case you need complete email address then :
echo $change |awk '{print $6}'
name#company.com

Solution with Parameter Expansion
First, a temporary variable that deletes string upto by and a space
$ change="Change 1234 on 2016/08/31 by name#company.com 'cdex abcd'"
$ tmp="${change#*by }"
$ echo "$tmp"
name#company.com 'cdex abcd'
Then, extract either the string before #
$ email="${tmp%#*}"
$ echo "$email"
name
Or, extract complete email address
$ email="${tmp%% *}"
$ echo "$email"
name#company.com
Edit:
To extract multiple strings separated by comma:
$ change="Change 1234 on 2016/08/31 by name#company.com 'cdex abcd'"
$ email=$(echo "$change" | perl -ne 'print join(",",/(\S+)#/g)')
$ echo "$email"
name
$ change="by name#company.com asd abcd#xyz.net 123 tom#xyz asdf"
$ email=$(echo "$change" | perl -ne 'print join(",",/(\S+)#/g)')
$ echo "$email"
name,abcd,tom

Related

How to search and delete a pattern from a line?

I need to write a simple bash script that takes a text line
some-pattern something-else
and erases some-pattern and returns only something-else. I wrote a script to do the opposite with grep -o, but I don't know how I could do with this case. Any help is very much appreciated.
sample input:
"SNMPv2::sysLocation.0 = STRING: someLocation"
Desired Output:
"someLocation"
Considering " are NOT in your sample Input_file and expected output, could you please try following with GNU grep.
grep -oP '.*STRING: \K(.*)' Input_file
someLocation
For \K explanation:
\K is a PCRE extension to regex syntax discarding content prior to
that point from being included in match output
You can use sed to delete the part in front of what you want to keep.
Given:
$ echo "$s"
"SNMPv2::sysLocation.0 = STRING: someLocation"
You can do:
$ echo "$s" | sed -nE 's/^.*(someLocation)/\1/p'
someLocation
And if you want to add quotes:
$ echo "$s" | sed -nE 's/^.*(someLocation)/"\1"/p'
"someLocation"
If the portion after STRING: is variable, not fixed, you can use STRING: and the capture anchor:
$ echo "$s" | sed -nE 's/^.*STRING:[[:space:]]*(.*)/"\1"/p'
"someLocation"
Or, sed to capture and print the last word after the last space:
$ echo "$s" | sed -nE 's/([^[:space:]]*$)/\1/p'
You can also use awk if the last word is space separated from the other fields:
$ echo "$s" | awk '{print $NF}'
Or a pipeline with cut and rev works too:
$ echo "$s" | rev | cut -d' ' -f 1 | rev
You can use: echo ${STRING} | awk -F" " '/someLocation/ { print $NF }'
-F will use space (represented by double-quotes with space between them) as separator; /someLocation/ will search for your location; { print $NF } will show the last part of your string (which, I believe, is the place where location is.

Extract field after colon for lines where field before colon matches pattern

I have a file file1 which looks as below:
tool1v1:1.4.4
tool1v2:1.5.3
tool2v1:1.5.2.c8.5.2.r1981122221118
tool2v2:32.5.0.abc.r20123433554
I want to extract value of tool2v1 and tool2v2
My output should be 1.5.2.c8.5.2.r1981122221118 and 32.5.0.abc.r20123433554.
I have written the following awk but it is not giving correct result:
awk -F: '/^tool2v1/ {print $2}' file1
awk -F: '/^tool2v2/ {print $2}' file1
grep -E can also do the job:
grep -E "tool2v[12]" file1 |sed 's/^.*://'
If you have a grep that supports Perl compatible regular expressions such as GNU grep, you can use a variable-sized look-behind:
$ grep -Po '^tool2v[12]:\K.*' infile
1.5.2.c8.5.2.r1981122221118
32.5.0.abc.r20123433554
The -o option is to retain just the match instead of the whole matching line; \K is the same as "the line must match the things to the left, but don't include them in the match".
You could also use a normal look-behind:
$ grep -Po '(?<=^tool2v[12]:).*' infile
1.5.2.c8.5.2.r1981122221118
32.5.0.abc.r20123433554
And finally, to fix your awk which was almost correct (and as pointed out in a comment):
$ awk -F: '/^tool2v[12]/ { print $2 }' infile
1.5.2.c8.5.2.r1981122221118
32.5.0.abc.r20123433554
You can filter with grep:
grep '\(tool2v1\|tool2v2\)'
And then remove the part before the : with sed:
sed 's/^.*://'
This sed operation means:
^ - match from beginning of string
.* - all characters
up to and including the :
... and replace this matched content with nothing.
The format is sed 's/<MATCH>/<REPLACE>/'
Whole command:
grep '\(tool2v1\|tool2v2\)' file1|sed 's/^.*://'
Result:
1.5.2.c8.5.2.r1981122221118
32.5.0.abc.r20123433554
the question has already been answered though, but you can also use pure bash to achieve the desired result
#!/usr/bin/env bash
while read line;do
if [[ "$line" =~ ^tool2v* ]];then
echo "${line#*:}"
fi
done < ./file1.txt
the while loop reads every line of the file.txt, =~ does a regexp match to check if the value of $line variable if it starts with toolv2, then it trims : backward

Extract substring after a character

I'm trying to extract substring after the last period (dot).
examples below.
echo "filename..txt" should return "txt"
echo "filename.txt." should return ""
echo "filename" should return ""
echo "filename.xml" should return "xml"
I tried below. but works only if the character(dot) exists once. But my filename may have (dot) for 0 or more times.
echo "filename.txt" | cut -d "." -f2
Let's use awk!
awk -F"." '{print (NF>1)? $NF : ""}' file
This sets field separator to . and prints the last one. But if there is none, it prints an empty string.
Test
$ cat file
filename..txt
filename.txt.
filename
filename.xml
$ awk -F"." '{print (NF>1)? $NF : ""}' file
txt
xml
One can make this portable (so it's not Linux-only), avoiding an ERE dependency, with the following:
$ sed -ne 's/.*\.//p' <<< "file..txt"
txt
$ sed -ne 's/.*\.//p' <<< "file.txt."
$ sed -ne 's/.*\.//p' <<< "file"
$ sed -ne 's/.*\.//p' <<< "file.xml"
xml
Note that for testing purposes, I'm using a "here-string" in bash. If your shell is not bash, use whatever your shell uses to feed data to sed.
The important bit here is the use of sed's -n option, which tells it not to print anything by default, combined with the substitute command's explicit p flag, which tells sed to print only upon a successful substitution, which obviously requires a dot to be included in the pattern.
With this solution, the difference between "file.txt." and "file" is that the former returns the input line replaced with null (so you may still get a newline depending on your usage), whereas the latter returns nothing, as sed is not instructed to print, as no . is included in the input. The end result may well be the same, of course:
$ printf "#%s#\n" $(sed -ne 's/.*\.//p' <<< "file.txt.")
##
$ printf "#%s#\n" $(sed -ne 's/.*\.//p' <<< "file")
##
Simple to do with awk:
awk -F"." '{ print $NF }'
What this does: With dot as a delimiter, extract the last field from the input.
Use sed in 2 steps: first remove string without a dot and than remove up to the last dot:
sed -e 's/^[^.]*$//' -e 's/.*\.//'
Test:
for s in file.txt.. file.txt. file.txt filename file.xml; do
echo "$s -> $(echo "$s" | sed -e 's/^[^.]*$//' -e 's/.*\.//')"
done
Testresult:
file.txt.. ->
file.txt. ->
file.txt -> txt
filename ->
file.xml -> xml
Actually the answer of #ghoti is roughly the same, just a bit shorter (better).
This solution can be used by other readers who wants to do something like this in another language.

How to removing leading section in bash

How can I remove parts of a string up to a certain character?
Ex.) If I have the string testFile.txt.1 and testFile.txt.12345 how can I remove the 1 and 12345?
EDIT: I meant to remove and throw away the first part of a string up to a certain character and keep the end of it.
using just bash facilities
$ s=testFile.txt.1
$ echo ${s%.*}
testFile.txt
$ s=testFile.txt.12345
$ echo ${s%.*}
testFile.txt
to remove before leading zero
$ echo ${s#*.}
txt.12345
Other method, you can split your string up using IFS
$ s=testFile.txt.12345
$ IFS="."
$ set -- $s
$ echo $1
testFile
$ echo $2
txt
$ echo $3
12345
If you just want to remove a known suffix from a filename, you can use basename:
basename testFile.txt.1 .1
basename testFile.txt.12345 .12345
You could use sed to do a search and replace. The <<< operator will pass in a string on stdin.
$ sed 's/\.[0-9]*$//' <<< "testFile.txt.1"
testFile.txt
$ sed 's/\.[0-9]*$//' <<< "testFile.txt.12345"
testFile.txt
The question is a bit unclear - the example provided may mean you want to remove all #s, or remove the part after the last ".", or remove the part after the first "1", or even remove all charcters after character 13. Please clarify.
If you mean that you want to remove first N characters in a string (e.g. "up to a character # 13"), do echo testFile.txt.1 | cut -c14-. To retain the chars 1-13, on the other hand, do echo testFile.txt.1 | cut -c1-13
If you mean that you want to remove the beginning characters until the first occurence of a specific character (in your example that seems to be "1"), do echo testFile.txt.1 | perl -e 's/^[^1]*//;'. To remove everything AFTER the first "1", do echo testFile.txt.1 | perl -e 's/1.*$//;'
If you want to remove all the #s, do echo testFile.txt.1 | perl -e 's/\d//g;' or without Perl, echo testFile.txt.1 | tr -d "[0-9]"
If you want to remove everything after the last ".", do echo testFile.txt.1 | perl -e 's/\.[^.]+/./;'
# remove string prefix up to the first digit
var='testFile.txt.12345'
var='test1File.txt.12345'
var='testFile.txt.1'
var='testFile.txt'
echo "${var#"${var%%[[:digit:]]*}"}"

Escape a string for a sed replace pattern

In my bash script I have an external (received from user) string, which I should use in sed pattern.
REPLACE="<funny characters here>"
sed "s/KEYWORD/$REPLACE/g"
How can I escape the $REPLACE string so it would be safely accepted by sed as a literal replacement?
NOTE: The KEYWORD is a dumb substring with no matches etc. It is not supplied by user.
Warning: This does not consider newlines. For a more in-depth answer, see this SO-question instead. (Thanks, Ed Morton & Niklas Peter)
Note that escaping everything is a bad idea. Sed needs many characters to be escaped to get their special meaning. For example, if you escape a digit in the replacement string, it will turn in to a backreference.
As Ben Blank said, there are only three characters that need to be escaped in the replacement string (escapes themselves, forward slash for end of statement and & for replace all):
ESCAPED_REPLACE=$(printf '%s\n' "$REPLACE" | sed -e 's/[\/&]/\\&/g')
# Now you can use ESCAPED_REPLACE in the original sed statement
sed "s/KEYWORD/$ESCAPED_REPLACE/g"
If you ever need to escape the KEYWORD string, the following is the one you need:
sed -e 's/[]\/$*.^[]/\\&/g'
And can be used by:
KEYWORD="The Keyword You Need";
ESCAPED_KEYWORD=$(printf '%s\n' "$KEYWORD" | sed -e 's/[]\/$*.^[]/\\&/g');
# Now you can use it inside the original sed statement to replace text
sed "s/$ESCAPED_KEYWORD/$ESCAPED_REPLACE/g"
Remember, if you use a character other than / as delimiter, you need replace the slash in the expressions above wih the character you are using. See PeterJCLaw's comment for explanation.
Edited: Due to some corner cases previously not accounted for, the commands above have changed several times. Check the edit history for details.
The sed command allows you to use other characters instead of / as separator:
sed 's#"http://www\.fubar\.com"#URL_FUBAR#g'
The double quotes are not a problem.
The only three literal characters which are treated specially in the replace clause are / (to close the clause), \ (to escape characters, backreference, &c.), and & (to include the match in the replacement). Therefore, all you need to do is escape those three characters:
sed "s/KEYWORD/$(echo $REPLACE | sed -e 's/\\/\\\\/g; s/\//\\\//g; s/&/\\\&/g')/g"
Example:
$ export REPLACE="'\"|\\/><&!"
$ echo fooKEYWORDbar | sed "s/KEYWORD/$(echo $REPLACE | sed -e 's/\\/\\\\/g; s/\//\\\//g; s/&/\\\&/g')/g"
foo'"|\/><&!bar
Based on Pianosaurus's regular expressions, I made a bash function that escapes both keyword and replacement.
function sedeasy {
sed -i "s/$(echo $1 | sed -e 's/\([[\/.*]\|\]\)/\\&/g')/$(echo $2 | sed -e 's/[\/&]/\\&/g')/g" $3
}
Here's how you use it:
sedeasy "include /etc/nginx/conf.d/*" "include /apps/*/conf/nginx.conf" /etc/nginx/nginx.conf
It's a bit late to respond... but there IS a much simpler way to do this. Just change the delimiter (i.e., the character that separates fields). So, instead of s/foo/bar/ you write s|bar|foo.
And, here's the easy way to do this:
sed 's|/\*!50017 DEFINER=`snafu`#`localhost`\*/||g'
The resulting output is devoid of that nasty DEFINER clause.
It turns out you're asking the wrong question. I also asked the wrong question. The reason it's wrong is the beginning of the first sentence: "In my bash script...".
I had the same question & made the same mistake. If you're using bash, you don't need to use sed to do string replacements (and it's much cleaner to use the replace feature built into bash).
Instead of something like, for example:
function escape-all-funny-characters() { UNKNOWN_CODE_THAT_ANSWERS_THE_QUESTION_YOU_ASKED; }
INPUT='some long string with KEYWORD that need replacing KEYWORD.'
A="$(escape-all-funny-characters 'KEYWORD')"
B="$(escape-all-funny-characters '<funny characters here>')"
OUTPUT="$(sed "s/$A/$B/g" <<<"$INPUT")"
you can use bash features exclusively:
INPUT='some long string with KEYWORD that need replacing KEYWORD.'
A='KEYWORD'
B='<funny characters here>'
OUTPUT="${INPUT//"$A"/"$B"}"
Use awk - it is cleaner:
$ awk -v R='//addr:\\file' '{ sub("THIS", R, $0); print $0 }' <<< "http://file:\_THIS_/path/to/a/file\\is\\\a\\ nightmare"
http://file:\_//addr:\file_/path/to/a/file\\is\\\a\\ nightmare
Here is an example of an AWK I used a while ago. It is an AWK that prints new AWKS. AWK and SED being similar it may be a good template.
ls | awk '{ print "awk " "'"'"'" " {print $1,$2,$3} " "'"'"'" " " $1 ".old_ext > " $1 ".new_ext" }' > for_the_birds
It looks excessive, but somehow that combination of quotes works to keep the ' printed as literals. Then if I remember correctly the vaiables are just surrounded with quotes like this: "$1". Try it, let me know how it works with SED.
These are the escape codes that I've found:
* = \x2a
( = \x28
) = \x29
" = \x22
/ = \x2f
\ = \x5c
' = \x27
? = \x3f
% = \x25
^ = \x5e
sed is typically a mess, especially the difference between gnu-sed and bsd-sed
might just be easier to place some sort of sentinel at the sed side, then a quick pipe over to awk, which is far more flexible in accepting any ERE regex, escaped hex, or escaped octals.
e.g. OFS in awk is the true replacement ::
date | sed -E 's/[0-9]+/\xC1\xC0/g' |
mawk NF=NF FS='\xC1\xC0' OFS='\360\237\244\241'
1 Tue Aug 🤡 🤡:🤡:🤡 EDT 🤡
(tested and confirmed working on both BSD-sed and GNU-sed - the emoji isn't a typo that's what those 4 bytes map to in UTF-8 )
There are dozens of answers out there... If you don't mind using a bash function schema, below is a good answer. The objective below was to allow using sed with practically any parameter as a KEYWORD (F_PS_TARGET) or as a REPLACE (F_PS_REPLACE). We tested it in many scenarios and it seems to be pretty safe. The implementation below supports tabs, line breaks and sigle quotes for both KEYWORD and replace REPLACE.
NOTES: The idea here is to use sed to escape entries for another sed command.
CODE
F_REVERSE_STRING_R=""
f_reverse_string() {
: 'Do a string reverse.
To undo just use a reversed string as STRING_INPUT.
Args:
STRING_INPUT (str): String input.
Returns:
F_REVERSE_STRING_R (str): The modified string.
'
local STRING_INPUT=$1
F_REVERSE_STRING_R=$(echo "x${STRING_INPUT}x" | tac | rev)
F_REVERSE_STRING_R=${F_REVERSE_STRING_R%?}
F_REVERSE_STRING_R=${F_REVERSE_STRING_R#?}
}
# [Ref(s).: https://stackoverflow.com/a/2705678/3223785 ]
F_POWER_SED_ECP_R=""
f_power_sed_ecp() {
: 'Escape strings for the "sed" command.
Escaped characters will be processed as is (e.g. /n, /t ...).
Args:
F_PSE_VAL_TO_ECP (str): Value to be escaped.
F_PSE_ECP_TYPE (int): 0 - For the TARGET value; 1 - For the REPLACE value.
Returns:
F_POWER_SED_ECP_R (str): Escaped value.
'
local F_PSE_VAL_TO_ECP=$1
local F_PSE_ECP_TYPE=$2
# NOTE: Operational characters of "sed" will be escaped, as well as single quotes.
# By Questor
if [ ${F_PSE_ECP_TYPE} -eq 0 ] ; then
# NOTE: For the TARGET value. By Questor
F_POWER_SED_ECP_R=$(echo "x${F_PSE_VAL_TO_ECP}x" | sed 's/[]\/$*.^[]/\\&/g' | sed "s/'/\\\x27/g" | sed ':a;N;$!ba;s/\n/\\n/g')
else
# NOTE: For the REPLACE value. By Questor
F_POWER_SED_ECP_R=$(echo "x${F_PSE_VAL_TO_ECP}x" | sed 's/[\/&]/\\&/g' | sed "s/'/\\\x27/g" | sed ':a;N;$!ba;s/\n/\\n/g')
fi
F_POWER_SED_ECP_R=${F_POWER_SED_ECP_R%?}
F_POWER_SED_ECP_R=${F_POWER_SED_ECP_R#?}
}
# [Ref(s).: https://stackoverflow.com/a/24134488/3223785 ,
# https://stackoverflow.com/a/21740695/3223785 ,
# https://unix.stackexchange.com/a/655558/61742 ,
# https://stackoverflow.com/a/11461628/3223785 ,
# https://stackoverflow.com/a/45151986/3223785 ,
# https://linuxaria.com/pills/tac-and-rev-to-see-files-in-reverse-order ,
# https://unix.stackexchange.com/a/631355/61742 ]
F_POWER_SED_R=""
f_power_sed() {
: 'Facilitate the use of the "sed" command. Replaces in files and strings.
Args:
F_PS_TARGET (str): Value to be replaced by the value of F_PS_REPLACE.
F_PS_REPLACE (str): Value that will replace F_PS_TARGET.
F_PS_FILE (Optional[str]): File in which the replacement will be made.
F_PS_SOURCE (Optional[str]): String to be manipulated in case "F_PS_FILE" was
not informed.
F_PS_NTH_OCCUR (Optional[int]): [1~n] - Replace the nth match; [n~-1] - Replace
the last nth match; 0 - Replace every match; Default 1.
Returns:
F_POWER_SED_R (str): Return the result if "F_PS_FILE" is not informed.
'
local F_PS_TARGET=$1
local F_PS_REPLACE=$2
local F_PS_FILE=$3
local F_PS_SOURCE=$4
local F_PS_NTH_OCCUR=$5
if [ -z "$F_PS_NTH_OCCUR" ] ; then
F_PS_NTH_OCCUR=1
fi
local F_PS_REVERSE_MODE=0
if [ ${F_PS_NTH_OCCUR} -lt -1 ] ; then
F_PS_REVERSE_MODE=1
f_reverse_string "$F_PS_TARGET"
F_PS_TARGET="$F_REVERSE_STRING_R"
f_reverse_string "$F_PS_REPLACE"
F_PS_REPLACE="$F_REVERSE_STRING_R"
f_reverse_string "$F_PS_SOURCE"
F_PS_SOURCE="$F_REVERSE_STRING_R"
F_PS_NTH_OCCUR=$((-F_PS_NTH_OCCUR))
fi
f_power_sed_ecp "$F_PS_TARGET" 0
F_PS_TARGET=$F_POWER_SED_ECP_R
f_power_sed_ecp "$F_PS_REPLACE" 1
F_PS_REPLACE=$F_POWER_SED_ECP_R
local F_PS_SED_RPL=""
if [ ${F_PS_NTH_OCCUR} -eq -1 ] ; then
# NOTE: We kept this option because it performs better when we only need to replace
# the last occurrence. By Questor
# [Ref(s).: https://linuxhint.com/use-sed-replace-last-occurrence/ ,
# https://unix.stackexchange.com/a/713866/61742 ]
F_PS_SED_RPL="'s/\(.*\)$F_PS_TARGET/\1$F_PS_REPLACE/'"
elif [ ${F_PS_NTH_OCCUR} -gt 0 ] ; then
# [Ref(s).: https://unix.stackexchange.com/a/587924/61742 ]
F_PS_SED_RPL="'s/$F_PS_TARGET/$F_PS_REPLACE/$F_PS_NTH_OCCUR'"
elif [ ${F_PS_NTH_OCCUR} -eq 0 ] ; then
F_PS_SED_RPL="'s/$F_PS_TARGET/$F_PS_REPLACE/g'"
fi
# NOTE: As the "sed" commands below always process literal values for the "F_PS_TARGET"
# so we use the "-z" flag in case it has multiple lines. By Quaestor
# [Ref(s).: https://unix.stackexchange.com/a/525524/61742 ]
if [ -z "$F_PS_FILE" ] ; then
F_POWER_SED_R=$(echo "x${F_PS_SOURCE}x" | eval "sed -z $F_PS_SED_RPL")
F_POWER_SED_R=${F_POWER_SED_R%?}
F_POWER_SED_R=${F_POWER_SED_R#?}
if [ ${F_PS_REVERSE_MODE} -eq 1 ] ; then
f_reverse_string "$F_POWER_SED_R"
F_POWER_SED_R="$F_REVERSE_STRING_R"
fi
else
if [ ${F_PS_REVERSE_MODE} -eq 0 ] ; then
eval "sed -i -z $F_PS_SED_RPL \"$F_PS_FILE\""
else
tac "$F_PS_FILE" | rev | eval "sed -z $F_PS_SED_RPL" | tac | rev > "$F_PS_FILE"
fi
fi
}
MODEL
f_power_sed "F_PS_TARGET" "F_PS_REPLACE" "" "F_PS_SOURCE"
echo "$F_POWER_SED_R"
EXAMPLE
f_power_sed "{ gsub(/,[ ]+|$/,\"\0\"); print }' ./ and eliminate" "[ ]+|$/,\"\0\"" "" "Great answer (+1). If you change your awk to awk '{ gsub(/,[ ]+|$/,\"\0\"); print }' ./ and eliminate that concatenation of the final \", \" then you don't have to go through the gymnastics on eliminating the final record. So: readarray -td '' a < <(awk '{ gsub(/,[ ]+/,\"\0\"); print; }' <<<\"$string\") on Bash that supports readarray. Note your method is Bash 4.4+ I think because of the -d in readar"
echo "$F_POWER_SED_R"
IF YOU JUST WANT TO ESCAPE THE PARAMETERS TO THE SED COMMAND
MODEL
# "TARGET" value.
f_power_sed_ecp "F_PSE_VAL_TO_ECP" 0
echo "$F_POWER_SED_ECP_R"
# "REPLACE" value.
f_power_sed_ecp "F_PSE_VAL_TO_ECP" 1
echo "$F_POWER_SED_ECP_R"
IMPORTANT: If the strings for KEYWORD and/or replace REPLACE contain tabs or line breaks you will need to use the "-z" flag in your "sed" command. More details here.
EXAMPLE
f_power_sed_ecp "{ gsub(/,[ ]+|$/,\"\0\"); print }' ./ and eliminate" 0
echo "$F_POWER_SED_ECP_R"
f_power_sed_ecp "[ ]+|$/,\"\0\"" 1
echo "$F_POWER_SED_ECP_R"
NOTE: The f_power_sed_ecp and f_power_sed functions above was made available completely free as part of this project ez_i - Create shell script installers easily!.
Standard recommendation here: use perl :)
echo KEYWORD > /tmp/test
REPLACE="<funny characters here>"
perl -pi.bck -e "s/KEYWORD/${REPLACE}/g" /tmp/test
cat /tmp/test
don't forget all the pleasure that occur with the shell limitation around " and '
so (in ksh)
Var=">New version of \"content' here <"
printf "%s" "${Var}" | sed "s/[&\/\\\\*\\"']/\\&/g' | read -r EscVar
echo "Here is your \"text\" to change" | sed "s/text/${EscVar}/g"
If the case happens to be that you are generating a random password to pass to sed replace pattern, then you choose to be careful about which set of characters in the random string. If you choose a password made by encoding a value as base64, then there is is only character that is both possible in base64 and is also a special character in sed replace pattern. That character is "/", and is easily removed from the password you are generating:
# password 32 characters log, minus any copies of the "/" character.
pass=`openssl rand -base64 32 | sed -e 's/\///g'`;
If you are just looking to replace Variable value in sed command then just remove
Example:
sed -i 's/dev-/dev-$ENV/g' test to sed -i s/dev-/dev-$ENV/g test
I have an improvement over the sedeasy function, which WILL break with special characters like tab.
function sedeasy_improved {
sed -i "s/$(
echo "$1" | sed -e 's/\([[\/.*]\|\]\)/\\&/g'
| sed -e 's:\t:\\t:g'
)/$(
echo "$2" | sed -e 's/[\/&]/\\&/g'
| sed -e 's:\t:\\t:g'
)/g" "$3"
}
So, whats different? $1 and $2 wrapped in quotes to avoid shell expansions and preserve tabs or double spaces.
Additional piping | sed -e 's:\t:\\t:g' (I like : as token) which transforms a tab in \t.
An easier way to do this is simply building the string before hand and using it as a parameter for sed
rpstring="s/KEYWORD/$REPLACE/g"
sed -i $rpstring test.txt

Resources