I would like to escape a file path that is stored in a variable in a bash script.
I read several threads about escaping back ticks or but it seems not working as it should:
I have this variable:
The variables value is entered during the bash script execution as user parameter
CONFIG="/home/teams/blabla/blabla.yaml"
I would need to change this to: \/home\/teams\/blabla\/blabla.yaml
How can I do that with in the script via sed or so (not manually)?
With GNU bash and its Parameter Expansion:
echo "${CONFIG//\//\\/}"
Output:
\/home\/teams\/blabla\/blabla.yaml
Using the solution from this question, in your case it will look like this:
CONFIG=$(echo "/home/teams/blabla/blabla.yaml" | sed -e 's/[]\/$*.^[]/\\&/g')
echo "/home/teams/blabla/blabla.yaml" | sed 's/\//\\\//g'
\/home\/teams\/blabla\/blabla.yaml
explanation:
backslash is used to set the following letter/symbol as an regular expression or vice versa. double backslash is used when you need a backslash as letter.
Why does that need escaping? Is this an XY Problem?
If the issue is that you are trying to use that variable in a substitution regex, then the examples given should work, but you might benefit by removing some of the "leaning toothpick syndrom", which many tools can do just by using a different match delimiter. sed, for example:
$: sed "s,SOME_PLACEHOLDER_VALUE,$CONFIG," <<< SOME_PLACEHOLDER_VALUE
/home/teams/blabla/blabla.yaml
Be very careful about this, though. Commas are perfectly valid characters in a filename, as are almost anything but NULLs. Know your data.
Related
This question already has answers here:
Replace whole line when match found with sed
(4 answers)
Closed 4 years ago.
How do I find line starts with and replace complete line?
File output:
xyz
abc
/dev/linux-test1/
Code:
output=/dev/sda/windows
sed 's/^/dev/linux*/$output/g' file.txt
I am getting below Error:
sed: -e expression #1, char 9: unknown option to `s'
File Output expected after replacement:
xyz
abc
/dev/sda/windows
Let's take this in small steps.
First we try changing "dev" to "other":
sed 's/dev/other/' file.txt
/other/linux-test1/
(Omitting the other lines.) So far, so good. Now "/dev/" => "/other/":
sed 's//dev///other//' file.txt
sed: 1: "s//dev///other//": bad flag in substitute command: '/'
Ah, it's confused, we're using '/' as both a command delimiter and literal text. So we use a different delimiter, like '|':
sed 's|/dev/|/other/|' file.txt
/other/linux-test1/
Good. Now we try to replace the whole line:
sed 's|^/dev/linux*|/other/|' file.txt
/other/-test1/
It didn't replace the whole line... Ah, in sed, '*' means the previous character repeated any number of times. So we precede it with '.', which means any character:
sed 's|^/dev/linux.*|/other/|' file.txt
/other/
Now to introduce the variable:
sed 's|^/dev/linux.*|$output|' file.txt
$output
The shell didn't expand the variable, because of the single quotes. We change to double quotes:
sed "s|^/dev/linux.*|$output|" file.txt
/dev/sda/windows
This might work for you (GNU sed):
output="/dev/sda/windows"; sed -i '\#/dev/linux.*/#c'"$output" file
Set the shell variable and change the line addressed by /dev/linux.*/ to it.
N.B. The shell variable needs to interpolated hence the ; i.e. the variable may be set on a line on its own. Also the the delimiter for the sed address must be changed so as not to interfere with the address, hence \#...#, and finally the shell variable should be enclosed in double quotes to allow full interpolation.
I'd recommend not doing it this way. Here's why.
Sed is not a programming language. It's a stream editor with some constructs that look and behave like a language, but it offers very little in the way of arbitrary string manipulation, format control, etc.
Sed only takes data from a file or stdin (also a file). Embedding strings within your sed script is asking for errors -- constructs like s/re/$output/ are destined to fail at some point, almost regardless of what workarounds you build into your sed script. The best solutions for making sed commands like this work is to do your input sanitization OUTSIDE of sed.
Which brings me to ... this may be the wrong tool for this job, or might be only one component of the toolset for the job.
The error you're getting is obviously because the sed command you're using is horribly busted. The substitute command is:
s/pattern/replacement/flags
but the command you're running is:
s/^/dev/linux*/$output/g
The pattern you're searching for is ^, the null at the beginning of the line. Your replacement pattern is dev, then you have a bunch of text that might be interpreted as flags. This plainly doesn't work, when your search string contains the same character that you're using as a delimiter to the options for the substitute command.
In regular expressions and in sed, you can escape things. You while you might get some traction with s/^\/dev\/linux.*/$output/, you'd still run into difficulty if $output contained slashes. If you're feeding this script to sed from bash, you could use ${output//\//\\\/}, but you can't handle those escapes within sed itself. Sed has no variables.
In a proper programming language, you'd have better separation of variable content and the commands used for the substitution.
output="/dev/sda/windows"
awk -v output="$output" '$1~/\/dev\/linux/ { $0=output } 1' file.txt
Note that I've used $1 here because in your question, your input lines (and output) appear to have a space at the beginning of each line. Awk automatically trims leading and trailing space when assigning field (positional) variables.
Or you could even do this in pure bash, using no external tools:
output="/dev/sda/windows"
while read -r line; do
[[ "$line" =~ ^/dev/linux ]] && line="$output"
printf '%s\n' "$line"
done < file.txt
This one isn't resilient in the face of leading whitespace. Salt to taste.
So .. yes, you can do this with sed. But the way commands get put together in sed makes something like this risky, and despite the available workarounds like switching your substitution command delimiter to another character, you'd almost certainly be better off using other tools.
I've read lots of posts to understand how to correctly escape white spaces and special characters inside strings using sed, but still i can't make it, here's what i'm trying to achieve.
I have a file containing the some strings like this one:
JAVA_OPTS="$JAVA_OPTS -Dorg.apache.catalina.jsessionid=some_value"
and i'm trying to replace 'some_value' using the following:
sed -i "s/^\(JAVA_OPTS=\"\$JAVA_OPTS[ \t]*-Dorg\.apache\.catalina\.jsessionid*=\s*\).*\$/\1$DORG_APACHE_CATALINA_JSESSIONID/" $JBOSS_CONFIGURATION/jboss.configuration
$JBOSS_CONFIGURATION is a variable containing an absolute Linux path.
jboss.configuration is a file i'm pointing as the target for replace
operations.
$DORG_APACHE_CATALINA_JSESSIONID contains the value i want instead
of 'some_value'.
Please note that the pattern:
JAVA_OPTS="$JAVA_OPTS -D
Is always present, and org.apache.catalina.jsessionid is an example of a variable value i'm trying to replace with this script.
What's missing/wrong ? i tried also escaping whitespaces using \s without success,
and echoing the whole gives me the following:
echo "s/^\(JAVA_OPTS=\"\$JAVA_OPTS[ \t]*-Dorg\.apache\.catalina\.jsessionid*=\s*\).*\$/\1$DORG_APACHE_CATALINA_JSESSIONID/"
s/^\(JAVA_OPTS="$JAVA_OPTS[ \t]*-Dorg\.apache\.catalina\.jsessionid*=\s*\).*$/\1/
is echo interpreting the search pattern as sed does ?
any info/help/alternative ways of doing it are highly welcome,
thank you all
echo 'JAVA_OPTS="$JAVA_OPTS -Dorg.apache.catalina.jsessionid=some_value"' | (export DORG_APACHE_CATALINA_JSESSIONID=FOO/BAR/FOOBAR; sed "s/^\(JAVA_OPTS=\"\$JAVA_OPTS[ \t]*-Dorg\.apache\.catalina\.jsessionid*=\s*\).*\$/\1${DORG_APACHE_CATALINA_JSESSIONID////\/}\"/")
Note the bash expansion (in order to escape any / that may trip up sed) and the extra \" after $DORG_APACHE_CATALINA_JSESSIONID in order to properly close the double quote. Other than that your sed expression works for me and the above command outputs the follwoing result:
JAVA_OPTS="$JAVA_OPTS -Dorg.apache.catalina.jsessionid=FOO/BAR/FOOBAR"
You can use sed like this:
sed -r '/\$JAVA_OPTS -D/{s/^(.+=).*$/\1'"$DORG_APACHE_CATALINA_JSESSIONID"'/;}' $JBOSS_CONFIGURATION/jboss.configuration
You can specify a pattern that'll match the desired string rather than trying to specify it exactly.
The following should work for you:
sed -i 's#^\(JAVA_OPTS.*Dorg.apache.catalina.jsessionid\)=\([^"]*\)"#\1='"$DORG_APACHE_CATALINA_JSESSIONID"'"#' $JBOSS_CONFIGURATION/jboss.configuration
sed 's/=\w.*$/='"$DORG_APACHE_CATALINA_JSESSIONID"'/' $JBOSS_CONFIGURATION/jboss.configuration
I need to replace within a little bash script a string inside a file but... I am getting weird results.
Let's say I want to replace:
<tag><![CDATA[text]]></tag>
With:
<tag><![CDATA[replaced_text]]></tag>
Should I use sed? I think due to / and [ ] I am getting weird results...
What would be the best way of approaching this?
Perl with -p option works almost as sed and it has \Q (quote) switch for its regexes:
perl -pe 's{\Q<tag><![CDATA[text]]></tag>}
{<tag><![CDATA[replaced_text]]></tag>}' YOUR_FILE
And in Perl you can use different punctuation to delimiter your expressions (s{...}{...} in my example).
Yes, you need to escape the brackets, and either escape slashes or use different delimiters.
sed 's,<tag><!\[CDATA\[text\]\]></tag>,<tag><!\[CDATA\[replaced)text\]\]></tag>,'
That said, SGML and XML are not actually any better than HTML when it comes to using regexes; don't expect this to generalize.
This should be enough:
$ echo '<tag><![CDATA[text]]></tag>' | sed 's/\[text\]/\[replaced_text\]/'
<tag><![CDATA[replaced_text]]></tag>
You can also change your / separator inside sed to a different character like ,, | or %.
Just use a delimiter other than /, here I use #:
sed -i 's#<tag><!\[CDATA\[text\]\]></tag>#<tag><![CDATA[replaced_text]]></tag>#g' filename
-i to have sed change the file instead of printing it out.
g is for matching more than once (global).
But do you know the exact string you want to match, both the tag and the text?
For instance, if you want to replace the text in all with your replaced_text:
perl -i -pe 's#(<tag><!\[CDATA\[)(.*?)(\]\]></tag>)#\1replaced_text\3#g' filename
Switched to perl because sed doesn't support non-greedy multipliers (the *?).
I am using sed -e "s/foo/$bar/" -e "s/some/$text/" file.whatever to replace a phrase in a certain file. The problem is that the $bar string contains multiple special characters like /. So when I try to replace something in a text file using the following code...
#!/bin/bash
bar="http://stackoverflow.com/"
sed -e "s/foo/$bar/" -e "s/some/$text/ file.whatever
...then I get an error saying : sed: unknown option to s is there anything I can do about it?
You can use any delimiter. s#some#SOME# for example. Another good delimiter is vertical-bar. Other chars can work but have special significance for some contexts such as regular expressions.
You can get this difficulty in sed regardless of what delimiters you use, especially if you don't know what the string contains. I'd pick a different method for passing the shell variables into the helper interpreter.
awk -v rep1="$bar" -v rep2="$text" '{sub(/foo/, rep1); sub(/some/, rep2); print}'
or
perl -spe 's/foo/$rep1/; s/some/$rep2/' -- -rep1="$bar" -rep2="$text"
Correctness trumps brevity in this case.
(reference for Perl example)
Alright, I know this is a simple question, but I can't seem to get this sed command to work. I'm trying to get a text file and replace one bit of it from placeholder text to a study code. The study code that it is going to replace it with is passed into the script using arguments when the script is first ran. The problem is, when I try to replace the placeholder text with the variable $study, it replaces it with a literally "$study".
Right now my arguments set like this:
export study=$1
export tag=$2
export mode=$3
export select=$4
My sed command looks like this:
sed -i.backup -e 's/thisisthestudycodereplacethiswiththestudycode/$study/' freq.spx
Is there some easy way of getting sed to not look at the literal $study, or would it be better at this point to do it another way?
Use double quotes instead of single quotes.
Because ' quoting prevents shell variable expansions, and " quoting does not.
You probably won't run into this issue, but just in case...
Paul's answer is slightly suboptimal if $study might contain slashes or other characters with special meaning to sed.
mv freq.spx freq.spx.backup && \
awk -v "study=$study" '{
gsub(/thisisthestudycodereplacethiswiththestudycode/, study);
print;
}' freq.spx.backup > freq.spx
Although awkward (hah, pun!), this will always work regardless of $study's contents.
try this...................
sed -e 's/%d/'$a'/g' amar.htm , amar.htm having the string "%d" which is indeed to be replaced and "a" is having the string to replace.