I'm trying to path a web address to a bash script run under cgi. I search a little and found this link. From what I understood, this line should separate the variable from it's value:
USERNAME=`echo "$QUERY_STRING" | sed -n 's/^.*username=\([^&]*\).*$/\1/p' | sed "s/%20/ /g"
So, reading about sed, I have concluded that this line should be sufficient for my needs:
url='echo "$QUERY_STRING" | sed "s/url=\(.*\)/\1/"'
where the input is
url=www.web.address.com
However, the variable get is the string is:
echo "$QUERY_STRING" | sed "s/url=(.*)/\1/"
if I tried to remove the apostrophes I get an empty variable. Note that if I simply to the echo command I the desired effect.
How can I separate the url value?
You can use parameter expansion and you won't need sed.
echo "${QUERY_STRING##*=}"
To set a variable to that value:
url=${QUERY_STRING##*=}
Related
I am using the below code for replacing a string
inside a shell script.
echo $LINE | sed -e 's/12345678/"$replace"/g'
but it's getting replaced with $replace instead of the value of that variable.
Could anybody tell what went wrong?
If you want to interpret $replace, you should not use single quotes since they prevent variable substitution.
Try:
echo $LINE | sed -e "s/12345678/${replace}/g"
Transcript:
pax> export replace=987654321
pax> echo X123456789X | sed "s/123456789/${replace}/"
X987654321X
pax> _
Just be careful to ensure that ${replace} doesn't have any characters of significance to sed (like / for instance) since it will cause confusion unless escaped. But if, as you say, you're replacing one number with another, that shouldn't be a problem.
you can use the shell (bash/ksh).
$ var="12345678abc"
$ replace="test"
$ echo ${var//12345678/$replace}
testabc
Not specific to the question, but for folks who need the same kind of functionality expanded for clarity from previous answers:
# create some variables
str="someFileName.foo"
find=".foo"
replace=".bar"
# notice the the str isn't prefixed with $
# this is just how this feature works :/
result=${str//$find/$replace}
echo $result
# result is: someFileName.bar
str="someFileName.sally"
find=".foo"
replace=".bar"
result=${str//$find/$replace}
echo $result
# result is: someFileName.sally because ".foo" was not found
Found a graceful solution.
echo ${LINE//12345678/$replace}
Single quotes are very strong. Once inside, there's nothing you can do to invoke variable substitution, until you leave. Use double quotes instead:
echo $LINE | sed -e "s/12345678/$replace/g"
Let me give you two examples.
Using sed:
#!/bin/bash
LINE="12345678HI"
replace="Hello"
echo $LINE | sed -e "s/12345678/$replace/g"
Without Using sed:
LINE="12345678HI"
str_to_replace="12345678"
replace_str="Hello"
result=${str//$str_to_replace/$replace_str}
echo $result
Hope you will find it helpful!
echo $LINE | sed -e 's/12345678/'$replace'/g'
you can still use single quotes, but you have to "open" them when you want the variable expanded at the right place. otherwise the string is taken "literally" (as #paxdiablo correctly stated, his answer is correct as well)
To let your shell expand the variable, you need to use double-quotes like
sed -i "s#12345678#$replace#g" file.txt
This will break if $replace contain special sed characters (#, \). But you can preprocess $replace to quote them:
replace_quoted=$(printf '%s' "$replace" | sed 's/[#\]/\\\0/g')
sed -i "s#12345678#$replace_quoted#g" file.txt
I had a similar requirement to this but my replace var contained an ampersand. Escaping the ampersand like this solved my problem:
replace="salt & pepper"
echo "pass the salt" | sed "s/salt/${replace/&/\&}/g"
use # if you want to replace things like /. $ etc.
result=$(echo $str | sed "s#$oldstr#$newstr#g")
the above code will replace all occurrences of the specified replacement term
if you want, remove the ending g which means that the only first occurrence will be replaced.
Use this instead
echo $LINE | sed -e 's/12345678/$replace/g'
this works for me just simply remove the quotes
I prefer to use double quotes , as single quptes are very powerful as we used them if dont able to change anything inside it or can invoke the variable substituion .
so use double quotes instaed.
echo $LINE | sed -e "s/12345678/$replace/g"
I'm trying to write a bash script to replace all occurrences of a placeholder in a file with an environment variable of the same name. As an example, if I have a file like the following...
This is an {{VAR1}} {{VAR2}}.
It should work across multiple lines in this {{VAR2}}.
... and I have the following environment variables set:
VAR1='example'
VAR2='file'
after running the script on my file, I should get the output:
This is an example file.
It should work across multiple lines in this file.
I'm sure there must be a solution using awk/sed, but so far the closest I've come can't handle if there's more than one variable on a line. Here's my attempt so far:
cat example.txt | grep -o '{{.*}}' > temp
while read placeholder; do
varName=$(echo "$placeholder" | tr -d '{}')
value="${!varName}"
sed -i "s/$placeholder/$value/g" "$file"
done < temp
rm -rf temp
I'd use Perl:
perl -pe 's/{{(.*?)}}/$ENV{$1}/g' filename
This assumes that VAR1 and VAR2 are environment variables (i.e., are exported), so that Perl can pick them out of its environment. This would be required of any approach that isn't pure shell; I just mention it to avoid confusion.
This works as follows:
s/pattern/replacement/g is a substitution command; you may recognize it from sed. The difference is that here we can use Perl's more powerful regex engine and variables. The g flag makes it so that all matches are replaced; without it, it would apply only to the first.
In the pattern, .*? matches non-greedily, so that in a line that contains foo {{VAR1}} bar {{VAR2}} baz, the pattern {{.*?}} matches only {{VAR1}} instead of {{VAR1}} bar {{VAR2}}.
The part between {{ and }} is captured because it is between () and can be reused as $1
$ENV{$1} in the replacement uses the special %ENV hash that contains the environment of the Perl process. $ENV{$1} is the value of the environment variable that has the name $1, which is the captured group from before.
Only bash and sed:
$ VAR1='example'
$ VAR2='file'
$ export VAR1 VAR2
$ sed -e '{s/{{\([^{]*\)}}/${\1}/g; s/^/echo "/; s/$/";/}' -e e filename
This is an example file.
It should work across multiple lines in this file.
sed -e '{s/{{\([^{]*\)}}/${\1}/g;}' filename:
This is an ${VAR1} ${VAR2}.
It should work across multiple lines in this ${VAR2}.
{{\([^{]*\)}} - Search for {{..}}
[^{] - Non greedy match
\1 - Access to the bracketed values \(...\).
sed -e '{s/{{\([^{]*\)}}/${\1}/g; s/^/echo "/; s/$/";/}' filename:
echo "This is an ${VAR1} ${VAR2}.";
echo "It should work across multiple lines in this ${VAR2}.";
s/^/echo "/ - Replace the beginning of the line with echo "
s/$/";/ - Replace the end of the line with ";
I was just playing with your original approach. Wouldn't adding another loop on $varName work?
cat example.txt | grep -o '{{.*}}' > temp
while read placeholder; do
varName=$(echo "$placeholder" | tr -d '{}')
for i in $varName; do
value="${!i}"
sed -i "s/{{$i}}/$value/g" example.txt
done
done < temp
rm -rf temp
With the help of SO I have been able to correct my sed command so I can use variables (was using ' instead of " and wasn't using g for global).
However, I am struggling to get the command to work correctly.
To give you some context, I have a file containing numerous lines of text and some line contain one or more tags. these tags are in the following format:
[#$key#] - i.e #[#] is used to indicate the presence of a key
Then I have an internal array object which stores a string value which contains each key and a corresponding value in the following format:
$key=$value
What I am trying to do is to cat my file containing the tags at the same time as using a global sed replace to swap the $key for the corresponding value within the array object.
The problem I have is that my grep/sed command is not replacing any of the text and I can't figure out why.
here is my code:
for x in "${prop[#]}"
do
key=`echo "${x}" | cut -d '=' -f 1`
value=`echo "${x}" | cut -d '=' -f 2`
# global replace on the $MY_FILE
cat $MY_FILE | sed "s|\[#${key}#\]|${value}|g" > ${TEMP_MY_FILE}
cat $TEMP_MY_FILE > $MY_FILE
done
I thought about using sed -i, but my version doesn't seem to support-i
Although [ doesn't have any special meaning in a double-quoted bash string, \[ still evaluates to [ since backslashes are processed to allow for escaping dollar signs. Try
sed "s|\\[#${key}#\\]|${value}|g"
as your sed command. The double backslash will causes a literal backslash to be sent to sed, which will use it to escape the [ to treat it literally as well.
I Want to search variable and replace with its absolute path in file.
setenv ABC /home/xyz
cat file.txt
${ABC}/Test/Folder_1
${ABC}/Test/Folder_2
I want to replace all occurance of the ${ABC} by /home/xyz.
I tried by the below mentioned way, but does not work,
sed -i 's/\$ABC/echo $ABC/g' file.txt
I can do by below mentioned way, but I do not want to do this way.(I have to put so many back slash)
$ echo $ABC | sed -i 's/\$ABC/\/home\/xyz/g' file.txt
Please give me some suggestion for this question.
Thank You.
If you really want to use the value from a variable in your replacement string, you could use
sed "s#\${ABC}#$ABC#g" file.txt
Character after s in sed is the delimiter and it can be any one character of your choice and it works as long as it's not in the string-to-be-matched and string-to-be-replaced.
Example :
sed 's:string-to-be-matched:string-to-be-replaced:g' file-to-be-edited
: is the delimiter
g means global replacement.
In your case, as the string-to-be-replaced contains the / , the same you are using as sed delimiter.
Simple Solution will be :
sed -i 's:${ABC}:'"$ABC"':g' fill.txt
'" is at either end of $ABC in the replacement string. Purpose is to expand shell variable to use with sed
Another way to get the absolute path is readlink -f ${ABC}/Test/Folder_2
Or the perl alternative to your slash hungry command
$ echo $ABC | sed -i 's/\$ABC/\/home\/xyz/g' file.txt
would be
$ echo $ABC | perl -p -i -e 's!\$ABC!/home/xyz!g'
the first character after the 's above will be used as the delimiter in the replacement expression (i.e. 's#foo#bar#g')
If I run these commands from a script:
#my.sh
PWD=bla
sed 's/xxx/'$PWD'/'
...
$ ./my.sh
xxx
bla
it is fine.
But, if I run:
#my.sh
sed 's/xxx/'$PWD'/'
...
$ ./my.sh
$ sed: -e expression #1, char 8: Unknown option to `s'
I read in tutorials that to substitute environment variables from shell you need to stop, and 'out quote' the $varname part so that it is not substituted directly, which is what I did, and which works only if the variable is defined immediately before.
How can I get sed to recognize a $var as an environment variable as it is defined in the shell?
Your two examples look identical, which makes problems hard to diagnose. Potential problems:
You may need double quotes, as in sed 's/xxx/'"$PWD"'/'
$PWD may contain a slash, in which case you need to find a character not contained in $PWD to use as a delimiter.
To nail both issues at once, perhaps
sed 's#xxx#'"$PWD"'#'
In addition to Norman Ramsey's answer, I'd like to add that you can double-quote the entire string (which may make the statement more readable and less error prone).
So if you want to search for 'foo' and replace it with the content of $BAR, you can enclose the sed command in double-quotes.
sed 's/foo/$BAR/g'
sed "s/foo/$BAR/g"
In the first, $BAR will not expand correctly while in the second $BAR will expand correctly.
Another easy alternative:
Since $PWD will usually contain a slash /, use | instead of / for the sed statement:
sed -e "s|xxx|$PWD|"
You can use other characters besides "/" in substitution:
sed "s#$1#$2#g" -i FILE
一. bad way: change delimiter
sed 's/xxx/'"$PWD"'/'
sed 's:xxx:'"$PWD"':'
sed 's#xxx#'"$PWD"'#'
maybe those not the final answer,
you can not known what character will occur in $PWD, / : OR #.
if delimiter char in $PWD, they will break the expression
the good way is replace(escape) the special character in $PWD.
二. good way: escape delimiter
for example:
try to replace URL as $url (has : / in content)
x.com:80/aa/bb/aa.js
in string $tmp
URL
A. use / as delimiter
escape / as \/ in var (before use in sed expression)
## step 1: try escape
echo ${url//\//\\/}
x.com:80\/aa\/bb\/aa.js #escape fine
echo ${url//\//\/}
x.com:80/aa/bb/aa.js #escape not success
echo "${url//\//\/}"
x.com:80\/aa\/bb\/aa.js #escape fine, notice `"`
## step 2: do sed
echo $tmp | sed "s/URL/${url//\//\\/}/"
URL
echo $tmp | sed "s/URL/${url//\//\/}/"
URL
OR
B. use : as delimiter (more readable than /)
escape : as \: in var (before use in sed expression)
## step 1: try escape
echo ${url//:/\:}
x.com:80/aa/bb/aa.js #escape not success
echo "${url//:/\:}"
x.com\:80/aa/bb/aa.js #escape fine, notice `"`
## step 2: do sed
echo $tmp | sed "s:URL:${url//:/\:}:g"
x.com:80/aa/bb/aa.js
With your question edit, I see your problem. Let's say the current directory is /home/yourname ... in this case, your command below:
sed 's/xxx/'$PWD'/'
will be expanded to
sed `s/xxx//home/yourname//
which is not valid. You need to put a \ character in front of each / in your $PWD if you want to do this.
Actually, the simplest thing (in GNU sed, at least) is to use a different separator for the sed substitution (s) command. So, instead of s/pattern/'$mypath'/ being expanded to s/pattern//my/path/, which will of course confuse the s command, use s!pattern!'$mypath'!, which will be expanded to s!pattern!/my/path!. I’ve used the bang (!) character (or use anything you like) which avoids the usual, but-by-no-means-your-only-choice forward slash as the separator.
Dealing with VARIABLES within sed
[root#gislab00207 ldom]# echo domainname: None > /tmp/1.txt
[root#gislab00207 ldom]# cat /tmp/1.txt
domainname: None
[root#gislab00207 ldom]# echo ${DOMAIN_NAME}
dcsw-79-98vm.us.oracle.com
[root#gislab00207 ldom]# cat /tmp/1.txt | sed -e 's/domainname: None/domainname: ${DOMAIN_NAME}/g'
--- Below is the result -- very funny.
domainname: ${DOMAIN_NAME}
--- You need to single quote your variable like this ...
[root#gislab00207 ldom]# cat /tmp/1.txt | sed -e 's/domainname: None/domainname: '${DOMAIN_NAME}'/g'
--- The right result is below
domainname: dcsw-79-98vm.us.oracle.com
VAR=8675309
echo "abcde:jhdfj$jhbsfiy/.hghi$jh:12345:dgve::" |\
sed 's/:[0-9]*:/:'$VAR':/1'
where VAR contains what you want to replace the field with
I had similar problem, I had a list and I have to build a SQL script based on template (that contained #INPUT# as element to replace):
for i in LIST
do
awk "sub(/\#INPUT\#/,\"${i}\");" template.sql >> output
done
If your replacement string may contain other sed control characters, then a two-step substitution (first escaping the replacement string) may be what you want:
PWD='/a\1&b$_' # these are problematic for sed
PWD_ESC=$(printf '%s\n' "$PWD" | sed -e 's/[\/&]/\\&/g')
echo 'xxx' | sed "s/xxx/$PWD_ESC/" # now this works as expected
for me to replace some text against the value of an environment variable in a file with sed works only with quota as the following:
sed -i 's/original_value/'"$MY_ENVIRNONMENT_VARIABLE"'/g' myfile.txt
BUT when the value of MY_ENVIRONMENT_VARIABLE contains a URL (ie https://andreas.gr) then the above was not working.
THEN use different delimiter:
sed -i "s|original_value|$MY_ENVIRNONMENT_VARIABLE|g" myfile.txt