This question already has answers here:
What special characters must be escaped in regular expressions?
(13 answers)
Closed 2 years ago.
I want to remove a line from a file and understand this can be done with sed.
The command below fails because of unterminated address regex
sed -i '/$settings['file_temp_path'] = '../tmp';/d' file.php
Having read the the answer at unterminated address regex while using sed . I now understand this is because characters [ and / must be escaped.
Having tried this, the code below is still unsuccessful
sed -i '/$settings\['file_temp_path'] = '..\/tmp';/d' file.php
What is wrong with this? What am I missing?
This might work for you (GNU sed):
sed -i '\#\$settings\['\''file_temp_path'\''\] = '\''\.\./tmp'\'';#d' file
When the regexp/replacement contains the match delimiter (usually /), it can either be escaped/quoted or the delimiters can be altered i.e. /\/tmp/ or \#/tmp#. Note that in the case of the substitution command s/\/tmp/replacement/ can also be s#/tmp#replacement# and leading delimiter does not need to escaped/quoted.
Meta characters i.e. ^,$,[,],*,.,\ and & must be escaped/quoted by \or placed in a character class e.g. . should be \. or [.].
As a rule of thumb, sed commands should be enclosed in single quotes ' and for single quotes to be included in the regexp they should be replaced by '\'' which closes off the existing commands, shell escapes/quotes a ' and reopens the next sed command.
Using double quotes " may also be used but may have unexpected side effects as they are open to shell interpolation.
N.B. If the regexp/substitution delimiter is put inside a character class it does not need to be escaped/quoted i.e. if / is the delimiter then [/] is the same as \/. Also note, that {,},|,? and + should not be escaped/quoted if they are to represent their literal value unless the -E or -r sed command line option is invoked, in which case they should be i.e + represents the plus sign as does \+ when the -E is invoked, whereas \+ and + when the -E or -r is invoked represent one or more of the preceding character/group.
You need to escape several special characters in your pattern, including $.
Example content of file.php:
foo
$settings['file_temp_path'] = '../tmp';
bar
Example code:
$ sed -i "s/\$settings\['file_temp_path'\] = '..\/tmp';//" file.php
$ cat file.php
foo
bar
Related
I'm having a directory with many files having special characters and spaces. I want to perform an operation with all these files so I'm trying to store all filenames in a list.txt and then run the command with this list.
The special characters in my list are & []'.
So basically I want to use sed to replace each occurence with \ + the character in question.
E.g. : filename .txt => filename\ .txt etc...
The thing is I have trouble handling apostrophes.
Here is my command as of now :
ls | sed 's/\ /\\ /g' | sed 's/\&/\\&/g' | sed "s/\'/\\'/g" | sed 's/\[/\\[/g' | sed 's/\]/\\]/g'
At first I had issues with, I believe, the apostrophes in the string command in conflict with the apostrophes surrounding the string. So I used double quotes instead, but it still doesn't work.
I've tried all these and nothing worked :
sed "s/\'/\\'/g" (escaping the apostrophe)
sed "s/'/\'/g" (escaping nothing)
sed "s/'/\\'/g" (escaping the backslash)
sed 's/"'"/\"'"/g' (double quoting single quote)
As a disclaimer, I must say, I'm completely new to sed. I just run my first sed command today, so maybe I'm doing something wrong I didn't realize.
PS : I've seen those thread, but no answer worked for me :
https://unix.stackexchange.com/questions/157076/how-to-remove-the-apostrophe-and-delete-the-space
How to replace to apostrophe ' inside a file using SED
This may do:
cat file
avbadf
test&rr
more [ yes
this ]
and'df
sed -r 's/(\x27|&|\[|\])/\\\1/g' file
avbadf
test\&rr
more \[ yes
this \]
and\'df
\x27 is equal to singe quote '
\x22 is equal to double quote "
Whoops, I found the answer to my question. Here is the working input :
sed "s/'/\\\'/g"
This will effectively replace any ' with \'.
However I'm having trouble understanding exactly what's happening here.
So if I understand correctly, we are escaping the backslash and the apostrophe in the replacement string. Now, if somebody could answer some those, I would be grateful :
Why don't we need to escape the first quote (the one in the pattern to find) ?
Why do we have to escape the backslash whereas for the other characters, there's no need ?
Why do we need to escape the second quote (the one in the replacement string) ?
I think all of your sed matches actually need that replacement pattern. This one seems to work for all examples:
ls | sed "s/\ /\\\ /g" | sed "s/\&/\\\&/g" | sed "s/\[/\\\[/g" | sed "s/\]/\\\]/g" | sed "s/'/\\\'/g"
So it is s/regex/replacement/command and 'regex' and 'replacement' have different sets of special characters.
The only one that's different is s/'/\\\'/g and there only because I don't believe there is any special ' character on the regex expression. There is some obscure \' special character in the replacement expression, for matching buffer ends in multi-line mode, accord to the docs. That might be why it needs an escape in the replacement side, but not in the regex side.
For example, \5 is a special character in the replacement expression, so to replace:
filename5.txt -> filename\5.txt
You would also need, as with apostrophe:
sed "s/5/\\\5/g"
It probably has to do with the mysterious inner works of sed parsing, it might read from right to left or something.
Please try the following:
sed 's/[][ &'\'']/\\&/g' file
By using the same example by #Jotne, the result will be:
gavbadf
gtest\&rr
gmore\ \[\ yes
gthis\ \]
gand\'df
[How it works]
The regex part in the sed s command above just defines a character
class of & []', which should be escaped with a backslash.
The right square bracket ] does not need escaping when put
immediately after the left square bracket [.
The obfuscating part will be the handling of a single quote.
We cannot put a single quote within single quotes even if we escape it.
The workaround is as follows: Say we have an assignment str='aaabbb'.
To put a single quote between "aaa" and "bbb", we can say as
str='aaa'\''bbb'.
It may look puzzling but it just concatenates the three sequences;
1) to close the single-quoted string as 'aaa'.
2) to put a single quote with an escaping backslash as \'.
3) to restart the single-quoted string as 'bbb'.
Hope this helps.
This question already has answers here:
How to insert strings containing slashes with sed? [duplicate]
(11 answers)
Closed 6 years ago.
I am trying to replace a script call with a script call from a sub directory.
For example, originally I had ./output.sh in my script to call the output.sh script in the current directory.
I want to replace ./output.sh with ../output.sh so that it calls output.sh in the parent directory.
I tried
sed -i -e 's/../\output.sh/./\output.sh/g' scriptName.sh
This returns with
char 17: unknown option to 's'
Any help with the sed escape character syntax would be great.
Sed is bad at this; you'll risk turning an already existing ../output.sh into .../output.sh if you're not careful.
This is the best sed can do:
sed -i 's#output\.sh#../$#g' scriptName.sh
(I'm using # in place of / so that there are no forward slashes to escape. Sed accepts any punctuation character in place of forward slash.)
Note that this will convert ../output.sh to ../../output.sh but at least it doesn't create that triple-dot error.
Instead, try perl:
perl -pie 's#(?<!\.\./)(output\.sh\b)#../$1#g' scriptName.sh
This uses a negative look-behind to ensure it doesn't traverse to the parent's parent. It also allows using \b to denote a word break just in case you have something like output.shelf somewhere.
This would do the trick. Substitutes every appearance of ./output.sh for ../output.sh :
sed 's/\.\/output\.sh/\.\.\/output\.sh/g' scriptName.sh
The escape character is \. You should use it to escape:
Every dot .. The dot is used as any character in regex.
Every slash /. The slash character is used as delimiter between regex on the scommand.
The slash character is special in sed's s command if you formulate it as s/something/replacement/flags, so the slashes in your file paths cause the error. Fortunately, you can use any other character right after s, for example s#something#replacement#flags. So, replacing the command with s#\./output.sh#../output.sh#g should do the trick.
Note two additional changes: you have to escape the dot in first expression since it's also a special character in regex and you also got reversed order of search expression and replacement (thanks charli for noticing this). You don't need the backslashes before o characters, either.
Alternatively, you can use / after s but escape the literal slashes you want to replace by preceding them with backslashes: s/\.\/output.sh/..\/\output.sh/g. It seems in your code you tried to use this solution but put the backslashes after instead of before the slashes.
This question already has answers here:
How to escape single quote in sed?
(9 answers)
Closed 6 years ago.
I have a config.js file which contents below strings
.constant('Digin_Engine_API', 'http://local.net:1929/')
I want to read this file and replace what ever the things which are there after the .constant('Digin_Engine_API'I tried using sedbut ddnt worked. This is what I used for sed
sed -i 's/^.constant('Digin_Engine_API', .*/http://cloud.lk:8080/' /var/config.js
As a summary my final out put (config.js) file needs to consists below.
Before
.constant('Digin_Engine_API', 'http://local.net:1929/')
After
.constant('Digin_Engine_API', 'http://cloud.lk:8080/')
You need to use double quotes around sed command since single quote is part of pattern
You should use an alternate delimiter since / is used in replacement
You need to capture the first part and use it in replacement
You need to quote the replacement and also add closing )
Sed command:
sed -i.bak "s~\(\.constant('Digin_Engine_API', \).*~\1'http://cloud.lk:8080')~" /var/config.js
cat /var/config.js
.constant('Digin_Engine_API', 'http://cloud.lk:8080')
Here you are:
sed -i -r "s_(\.constant\('Digin_Engine_API').*_\1, <new content>)_" file
Remarks
you cannot use ' in sed command if command is surrounded by '' also,
you must escape all ( and ) that are part or string, and not the sed grouping command,
you must escape . character, because it is also sed replacement for every char,
you must use another sed s separator instead of / if you need to use / in that command, but you can also escape / by \/.
This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 6 years ago.
I'm trying to replace text in a file with the output of another command. Unfortunately, the outputted text contains characters bash expands. For example, I'm running the following script to change the file (somestring references output that would break the sed command):
#!/bin/bash
somestring='$6$sPnfj/lnXwZVrec7$fCnL9uy1oWIMZduInKTHBAxhsQxGCsBpm2XfVFFqDPHKidrd93yfjbYvKgYexXHVcvkKdu9lbfy16Ek5GvKy/1'
sed '0,/^title/s/^title*/'"$somestring"'\n&/' $HOME/example.txt
sed fails with this error:
sed: -e expression #1, char 30: unknown option to `s'
I think bash is substuting the contents of $somestring when building the sed command, but is then trying to expand the resulting text. I can't put the entire sed script in single quotes, I need bash to expand it the first time, just not the second. Any suggestions? Thanks
here the forward slash / is the problem. If it's the only issue you can set sed to use a different delimiter.
for example
$ somestring="abc/def"; echo xxx | sed 's/xxx/'"$somestring"'/'
sed: -e expression #1, char 11: unknown option to `s'
$ somestring="abc/def"; echo xxx | sed 's_xxx_'"$somestring"'_'
abc/def
you also need to worry about & and \ chars and escape them if can appear in the replacement text.
If you can't control the the replacement string, either you have to sanitize with another sed script or, alternatively use r command to read it from a file. For example,
$ seq 5 | sed -e '/3/{r replace' -e 'd}'
1
2
3slashes///1ampersand&and2backslashes\\end
4
5
where
$ cat replace
3slashes///1ampersand&and2backslashes\\end
You have several errors here:
the string somestring has characters that are significative for sed command (the most important being '/' that you are using as a delimiter) You can escape it, by substituting it with a previous
somestring=$(echo "$somestring" | sed -e 's/\//\\\//g')
that will convert your / chars to \/ sequences.
you are using sed '0,/^title/s/^title*/'"$somestring"'\n&/' $HOME/example.txt which is looking to substitute the string titl followed by any number of e characters by that $somestring value, followed by a new line and the original one. Unfortunately, sed(1) doesn't allow you to use newline characters in the pattern substitution side of the s command, but you can afford the result by using the i command with a text consisting of you pattern (preceding any new line by a \ to interpret it as literal):
Finally the script leads to:
#!/bin/bash
somestring='$6$sPnfj/lnXwZVrec7$fCnL9uy1oWIMZduInKTHBAxhsQxGCsBpm2XfVFFqDPHKidrd93yfjbYvKgYexXHVcvkKdu9lbfy16Ek5GvKy/1'
somestring=$(echo "$somestring" | sed -e 's/\//\\\//g')
sed '/^title/i\
'"$somestring\\
" $HOME/example.txt
If your shell is Bash, you can use parameter substitution to replace the problematic /:
somestring="{somestring//\//\\/}"
That looks scary, but is easier to understand if you look at the version that replaces x with __:
somestring="${somestring//x/__}"
It might be easier to use (say) underscore as the delimiter for your sed s command, and then the substitution above would be
somestring="${somestring//_/\\_}"
If you already have backslashes, you'll need to first replace those:
somestring="${somestring//\\/\\\\}"
somestring="{somestring//\//\\/}"
If there were other characters that needed escaping (e.g. on the search side of s///), then you could extend the above appropriately.
This URL provides the cleanest answer:
Command to escape a string in bash
printf "%q" "$someVariable"
will escape any characters you need escaped for you.
How can I fix this:
abc="a/b/c"; echo porc | sed -r "s/^/$abc/"
sed: -e expression #1, char 7: unknown option to `s'
The substitution of variable $abc is done correctly, but the problem is that $abc contains slashes, which confuse sed. Can I somehow escape these slashes?
Note that sed(1) allows you to use different characters for your s/// delimiters:
$ abc="a/b/c"
$ echo porc | sed -r "s|^|$abc|"
a/b/cporc
$
Of course, if you go this route, you need to make sure that the delimiters you choose aren't used elsewhere in your input.
The GNU manual for sed states that "The / characters may be uniformly replaced by any other single character within any given s command."
Therefore, just use another character instead of /, for example ::
abc="a/b/c"; echo porc | sed -r "s:^:$abc:"
Do not use a character that can be found in your input. We can use : above, since we know that the input (a/b/c/) doesn't contain :.
Be careful of character-escaping.
If using "", Bash will interpret some characters specially, e.g. ` (used for inline execution), ! (used for accessing Bash history), $ (used for accessing variables).
If using '', Bash will take all characters literally, even $.
The two approaches can be combined, depending on whether you need escaping or not, e.g.:
abc="a/b/c"; echo porc | sed 's!^!'"$abc"'!'
You don't have to use / as pattern and replace separator, as others already told you. I'd go with : as it is rather rarely used in paths (it's a separator in PATH environment variable). Stick to one and use shell built-in string replace features to make it bullet-proof, e.g. ${abc//:/\\:} (which means replace all : occurrences with \: in ${abc}) in case of : being the separator.
$ abc="a/b/c"; echo porc | sed -r "s:^:${abc//:/\\:}:"
a/b/cporc
backslash:
abc='a\/b\/c'
space filling....
As for the escaping part of the question I had the same issue and resolved with a double sed that can possibly be optimized.
escaped_abc=$(echo $abc | sed "s/\//\\\AAA\//g" | sed "s/AAA//g")
The triple A is used because otherwise the forward slash following its escaping backslash is never placed in the output, no matter how many backslashes you put in front of it.