With the help of SO I have been able to correct my sed command so I can use variables (was using ' instead of " and wasn't using g for global).
However, I am struggling to get the command to work correctly.
To give you some context, I have a file containing numerous lines of text and some line contain one or more tags. these tags are in the following format:
[#$key#] - i.e #[#] is used to indicate the presence of a key
Then I have an internal array object which stores a string value which contains each key and a corresponding value in the following format:
$key=$value
What I am trying to do is to cat my file containing the tags at the same time as using a global sed replace to swap the $key for the corresponding value within the array object.
The problem I have is that my grep/sed command is not replacing any of the text and I can't figure out why.
here is my code:
for x in "${prop[#]}"
do
key=`echo "${x}" | cut -d '=' -f 1`
value=`echo "${x}" | cut -d '=' -f 2`
# global replace on the $MY_FILE
cat $MY_FILE | sed "s|\[#${key}#\]|${value}|g" > ${TEMP_MY_FILE}
cat $TEMP_MY_FILE > $MY_FILE
done
I thought about using sed -i, but my version doesn't seem to support-i
Although [ doesn't have any special meaning in a double-quoted bash string, \[ still evaluates to [ since backslashes are processed to allow for escaping dollar signs. Try
sed "s|\\[#${key}#\\]|${value}|g"
as your sed command. The double backslash will causes a literal backslash to be sent to sed, which will use it to escape the [ to treat it literally as well.
Related
I have a config file where i need to replace the entire line with new values. It can be either a word or a URL. I am trying to write a script to replace only this particular parameter with new values.
I have tried using grep to extract the line using the parameter and divided the values separately and saved in two different variables.
Now I am trying to replace the whole line with the parameter along with new value or url usind sed
jaram=`grep -i "$a" app.properties`;
param=`grep -i "$a" app.properties |sed 's/\'$a'=*//'`;
sed -e 's~'$jaram'~'$a=''$changed_param'~g' app.properties
The config file contains:
abc1=http://howareyou:scema=olk
abc2=http://howareyou:scema=olk
Here I am trying to replace the url of only abc1. though both have same value. I need to replace the entire url with something different url or a word.
Here I am trying to find the line which contains abc1 and this line url after = is saved in a different variable.
I tried to replace the url with new one using sed:
sed -i 's~'$jaram'~'$a=''$param'~g' app.properties
sed: -e expression #1, char 0: no previous regular expression
Seems like I am doing wrong at some syntax when using sed
trying to replace something like
sed 's/jaram/'{$a=$param}/'
Expecting an output like
abc1=http://jalkek:kj/iuwerj
abc2=http://howareyou:scema=olk
The following script with comments:
# recrete the input config file
cat <<EOF >input
abc1=http://howareyou:scema=olk
abc2=http://howareyou:scema=olk
EOF
# some input variables
name="abc1"
newvalue='http://jalkek:kj/iuwerj'
# the sed script
sed -i 's'$'\01''^'"$name"'=.*'$'\01'"$name"'='"$newvalue"$'\x01' input
# and the output
cat input
produces the following output:
abc1=http://jalkek:kj/iuwerj
abc2=http://howareyou:scema=olk
Notes:
I used the 0x01 byte as the separator for s command inside sed. So it should work with all printable characters.
Remember about quoting. All variables should be inside " double quotes, but all the rest would be best if inside ' single quotes.
I match the name= with ^, so from the beginning of the line. So that for example blablaabc1= doesn't match.
I use ANSI-C Quoting from bash to generate the unredable 0x01 bytes to delimit sed command.
A little more readable version of the sed script could be this one, that just uses " for quoting and uses ~ as a separator:
sed "s~^${name}=.*~${name}=${newvalue}~"
I have a text file that contain a lot of mess text.
I used grep to get all the text that contains the string prod like this
cat textfile | grep "<host>prod*"
The result
<host>prod-reverse-proxy01</host>
<host>prod-reverse-proxy01</host>
<host>prod-reverse-proxy01</host>
Continually, i used sed with the intention to remove all the "host" part
cat textfile | grep "<host>prod*" | sed "s/<host>//g"; "s/</host>//g"
But only the first "host" was removed.
prod-reverse-proxy01</host>
prod-reverse-proxy01</host>
prod-reverse-proxy01</host>
How can i remove the other "/host" part?
sed -n -e "s/^<host>\(.*\)<\/host>/\1/p" textfile
sed can process your file directly. No need to grep or cat.
-n is there to suppress any lines that do not match. Last 'p' in the script will print all matching files.
Script dissection:
s/.../.../...
is the search/replace form. The bit between the first and the second '/' is what you search for. The bit between the second and third is what you replace it with. The last part is any commands you want to apply to the replacement.
Search:
^<host>\(.*\)<\/host>
finds all lines beginning with <host> followed by any text (.*) followed by </host>. Any text between <host> and </host> is stored into internal variable '1' using '(' and ')'. Note that (, ) and / (in </host>) have to be escaped.
Replace:
\1
Replace found text with contents of variable 1 (1 has to be escaped, otherwise, everything is replaced by character '1'.
Commands:
p
Print resulting line (after replacement).
Note: Your search involves removing two similar but not identical strings (<host> and </host>).
I think this sed is enough
sed 's/<[/]*host>//g' infile
I was trying to do these few operations/commands on a single line and assign it to a variable. I have it working about 90% of the way except for one part of it.
I was unaware you could do this, but I read that you can nest $(..) inside other $(..).... So I was trying to do that to get this working, but can't seem to get it the rest of the way.
So basically, what I want to do is:
1. Take the output of a file and assign it to a variable
2. Then pre-pend some text to the start of that output
3. Then append some text to the end of the output
4. And finally remove newlines and replace them with "\n" character...
I can do this just fine in multiple steps but I would like to try and get this working this way.
So far I have tried the following:
My 1st attempt, before reading about nested $(..):
MY_VAR=$(echo -n "<pre style=\"display:inline;\">"; cat input.txt | sed ':a;N;$!ba;s/\n/\\n/g'; echo -n "</pre>")
This one worked 99% of the way except there was a newline being added between the cat command's output and the last echo command. I'm guessing this is from the cat command since sed removed all newlines except for that one, maybe...?
Other tries:
MY_VAR=$( $(echo -n "<pre style=\"display:inline;\">"; cat input.txt; echo -n "</pre>") | sed ':a;N;$!ba;s/\n/\\n/g')
MY_VAR="$( echo $(echo -n "<pre style=\"display:inline;\">"; cat input.txt; echo "</pre>") | sed ':a;N;$!ba;s/\n/\\n/g' )"
MY_VAR="$( echo "$(echo -n "<pre style=\"display:inline;\">"; cat input.txt; echo "</pre>")" | sed ':a;N;$!ba;s/\n/\\n/g' )"
*Most these others were tried with and without the extra double-quotes surrounding the different $(..) parts...
I had a few other attempts, but they didn't have any luck either... On a few of the other attempts above, it seemed to work except sed was NOT inserting the replacement part of it. The output was correct for the most part, except instead of seeing "\n" between lines it just showed each of the lines smashed together into one line without anything to separate them...
I'm thinking there is something small I am missing here if anyone has any idea..?
*P.S. Does Bash have a name for the $(..) structure? It's hard trying to Google for that since it doesn't really search symbols...
You have no need to nest command substitutions here.
your_var='<pre style="display:inline;">'"$(<input.txt)"'</pre>'
your_var=${your_var//$'\n'/'\n'}
"$(<input.txt)" expands to the contents of input.txt, but without any trailing newline. (Command substitution always strips trailing newlines; printf '%s' "$(cat ...)" has the same effect, albeit less efficiently as it requires a subshell, whereas cat ... alone does not).
${foo//bar/baz} expands to the contents of the shell variable named foo, with all instances of bar replaced with baz.
$'\n' is bash syntax for a literal newline.
'\n' is bash syntax for a two-character string, beginning with a backslash.
Thus, tying all this together, it first generates a single string with the prefix, the contents of the file, and the suffix; then replaces literal newlines inside that combined string with '\n' two-character sequences.
Granted, this is multiple lines as implemented above -- but it's also much faster and more efficient than anything involving a command substitution.
However, if you really want a single, nested command substitution, you can do that:
your_var=$(printf '%s' '<pre style="display:inline;">' \
"$(sed '$ ! s/$/\\n/g' <input.txt | tr -d '\n')" \
'</pre>')
The printf %s combines its arguments without any delimiter between them
The sed operation adds a literal \n to the end of each line except the last
The tr -d '\n' operation removes literal newlines from the file
However, even this approach could be done more efficiently without the nesting:
printf -v your_var '%s' '<pre style="display:inline;">' \
"$(sed '$ ! s/$/\\n/g' <input.txt | tr -d '\n')" \
'</pre>')
...which has the printf assign its results directly to your_var, without any outer command substitution required (and thus saving the expense of the outer subshell).
I trying to find and replace items using bash. I was able to use sed to grab out some of the characters, but I think I might be using it in the wrong matter.
I am basically trying to remove the characters after ";" and before "," including removing ","
sed -e 's/\(;\).*\(,\)/\1\2/'
That is what I used to replace it with nothing. However, it ends up replacing everything in the middle so my output came out like this:
cmd2="BMC,./socflash_x64 if=B600G3_BMC_V0207.ima;,reboot -f"
This is the original text of what I need to replace
cmd2="BMC,./socflash_x64 if=B600G3_BMC_V0207.ima;X,sleep 120;after_BMC,./run-after-bmc-update.sh;hba_fw,./hba_fw.sh;X,sleep 5;DB,2;X,reboot -f"
Is there any way to make it look like this output?
./socflash_x64 if=B600G3_BMC_V0207.ima;sleep 120;./run-after-bmc-update.sh;./hba_fw.sh;sleep 5;reboot -f
Ff there is any way to make this happen other than bash I am fine with any type of language.
Non-greedy search can (mostly) be simulated in programs that don't support it by replacing match-any (dot .) with a negated character class.
Your original command is
sed -e 's/\(;\).*\(,\)/\1\2/'
You want to match everything in between the semi-colon and the comma, but not another comma (non-greedy). Replace .* with [^,]*
sed -e 's/\(;\)[^,]*\(,\)/\1\2/'
You may also want to exclude semi-colons themselves, making the expression
sed -e 's/\(;\)[^,;]*\(,\)/\1\2/'
Note this would treat a string like "asdf;zxcv;1234,qwer" differently, since one would match ;zxcv;1234, and the other would match only ;1234,
In perl:
perl -pe 's/;.*?,/;/g;' -pe 's/^[^,]*,//' foo.txt
will output:
./socflash_x64 if=B600G3_BMC_V0207.ima;sleep 120;./run-after-bmc-update.sh;./hba_fw.sh;sleep 5;2;reboot -f
The .*? is non greedy matching before the comma. The second command is to remove from the beginning to the comma.
Something like:
echo $cmd2 | tr ';' '\n' | cut -d',' -f2- | tr '\n' ';' ; echo
result is:
./socflash_x64 if=B600G3_BMC_V0207.ima;sleep 120;./run-after-bmc-update.sh;./hba_fw.sh;sleep 5;2;reboot -f;
however, I thing your requirements are a few more complex, because 'DB,2' seems a particular case. After "tr" command, insert a "grep" or "grep -v" to include/exclude these cases.
I have a file like
key\0value\n
akey\0value\n
key2\0value\n
I have to create a script that take as argument a word. I have to return every lines having a key exactly the same than the argument.
I tried
grep -aF "$key\x0"
but grep seems to do not understand the \x0 (\0 same result). Futhermore, I have to check that the line begins with "$key\0"
I only can use sed grep and tr and other no maching commands
To have the \0 taken into account try :
grep -Pa "^key\x0"
it works for me.
Using sed
sed will work:
$ sed -n '/^key1\x00/p' file
key1value
The use of \x00 to represent a hex character is a GNU extension to sed. Since this question is tagged linux, that is not a problem.
Since the null character does not display well, one might (or might not) want to improve the display with something like this:
$ sed -n 's/^\(akey\)\x00/\1-->/p' file
akey-->value
Using sed with keys that contain special characters
If the key itself can contain sed or shell active characters, then we must escape them first and then run sed against the input file:
#!/bin/bash
printf -v script '/^%s\\x00/p' "$(sed 's:[]\[^$.*/]:\\&:g' <<<"$1")"
sed -n "$script" file
To use this script, simply supply the key as the first argument on the command line, enclosed in single-quotes, of course, to prevent shell processing.
To see how it works, let's look at the pieces in turn:
sed 's:[]\[^$.*/]:\\&:g' <<<"$1"
This puts a backslash escape in front of all sed-active characters.
printf -v script '/^%s\\x00/p' "$(sed 's:[]\[^$.*/]:\\&:g' <<<"$1")"
This creates a sed command using the escaped key and stores it in the shell variable script.
sed -n "$script" file
This runs sed using the shell variable script as the sed command.
Using awk
The question states that awk is not an acceptable tool. For completeness, though, here is an awk solution:
$ awk -F'\x00' -v k=key1 '$1 == k' file
key1value
Explanation:
-F'\x00'
awk divides the input up into records (lines) and divides the records up into fields. Here, we set the field separator to the null character. Consequently, the first field, denoted $1, is the key.
-v k=key1
This creates an awk variable, called k, and sets it to the key that we are looking for.
$1 == k
This statement looks for records (lines) for which the first field matches our specified key. If a match is found, the line is printed.