How can i replace the following text in a file in linux with a different line
Current :
0 22 * * * /scripts/application_folder_backup.sh >> /var/log/application_folder_backup.log
Replacement line : #line_removed
I tried using sed but my text in the file already has a / which is causing problems. I tried storing the string in a variable too. But it doesn't work
#!/bin/bash
var="0 22 * * * /scripts/application_folder_backup.sh >> /var/log/application_folder_backup.log"
sed -i -e 's/$var/#line_removed/g' /tmp/k1.txt
exit
Just / is not a problem here, even * or all the special regex meta characters will be a problem for sed since it uses only regex for search patterns.
Better to use this non-regex based awk command:
awk -v var="$var" 'index($0, var) { $0 = "#line_removed" } 1' file
#line_removed
index function in awk uses plain text search instead of a regex based search.
I suggest previous to feed the contents of $var to the sed script, to escape all the / chars to be \/, as in:
var_esc=$(echo "$var" | sed 's/\//\\\//g')
But the sed expression gets very complicated (you must use single quotes, if you don't want to double \ chars, as double quotes do interpret also the backslashes.
Another thing that could simplify the expression is to use the possibility of change the regexp delimiter (with using a different char to begin it) as in:
var_esc=$(echo "$var" | sed 's:/:\/:g')
The idea is to, before substituting the $var variable, to escape all the possible interferring chars you can have (you don't know a priory if there are going to be, as you are using a variable for that purpose) and then substitute them in the final command:
sed "s/$var_esc/#line_removed/g"
Finally, if you don't want to substitute the line, but just to erase it, you can do something like this:
sed "/$var_esc/d"
and that will erase all lines that match your $var_esc contents. (this time I believe you cannot change the regexp delimiter, as the / introduces the matching line operator --- d is the command to delete line in the output)
Another way to delete the line in your file is to call ex directly and pass the editor command as input:
ex file <<EOF <-- edit "file" with the commands that follow until EOF is found.
/$var_esc <-- search for first occurrence of $var_esc
d <-- delete line
w <-- write file
EOF <-- eof marker.
Related
I have a config file where i need to replace the entire line with new values. It can be either a word or a URL. I am trying to write a script to replace only this particular parameter with new values.
I have tried using grep to extract the line using the parameter and divided the values separately and saved in two different variables.
Now I am trying to replace the whole line with the parameter along with new value or url usind sed
jaram=`grep -i "$a" app.properties`;
param=`grep -i "$a" app.properties |sed 's/\'$a'=*//'`;
sed -e 's~'$jaram'~'$a=''$changed_param'~g' app.properties
The config file contains:
abc1=http://howareyou:scema=olk
abc2=http://howareyou:scema=olk
Here I am trying to replace the url of only abc1. though both have same value. I need to replace the entire url with something different url or a word.
Here I am trying to find the line which contains abc1 and this line url after = is saved in a different variable.
I tried to replace the url with new one using sed:
sed -i 's~'$jaram'~'$a=''$param'~g' app.properties
sed: -e expression #1, char 0: no previous regular expression
Seems like I am doing wrong at some syntax when using sed
trying to replace something like
sed 's/jaram/'{$a=$param}/'
Expecting an output like
abc1=http://jalkek:kj/iuwerj
abc2=http://howareyou:scema=olk
The following script with comments:
# recrete the input config file
cat <<EOF >input
abc1=http://howareyou:scema=olk
abc2=http://howareyou:scema=olk
EOF
# some input variables
name="abc1"
newvalue='http://jalkek:kj/iuwerj'
# the sed script
sed -i 's'$'\01''^'"$name"'=.*'$'\01'"$name"'='"$newvalue"$'\x01' input
# and the output
cat input
produces the following output:
abc1=http://jalkek:kj/iuwerj
abc2=http://howareyou:scema=olk
Notes:
I used the 0x01 byte as the separator for s command inside sed. So it should work with all printable characters.
Remember about quoting. All variables should be inside " double quotes, but all the rest would be best if inside ' single quotes.
I match the name= with ^, so from the beginning of the line. So that for example blablaabc1= doesn't match.
I use ANSI-C Quoting from bash to generate the unredable 0x01 bytes to delimit sed command.
A little more readable version of the sed script could be this one, that just uses " for quoting and uses ~ as a separator:
sed "s~^${name}=.*~${name}=${newvalue}~"
I have the following sed commands that replace a block of text with the contents of a file between the start & end markers /**##+ and **##-* respectively:
sed -i -ne '/**##+/ {p; r block.txt' -e ':a; n; /**##-*/ {p; b}; ba}; p' -e '/**##+/d' test.txt && sed -i -e '/**##+/d' -e '/**##-*/d' test.txt
(Besides replacing text, the command also converts line endings.)
As it is, this leaves the start and end markers intact, but I want to get rid of those as well. My using the p command means that I can't have a d command in the same execution unit. I work around the problem by introducing a second set of commands that delete those markers, but I would like to have it all in one single sed command, if possible.
test.txt
start of file
/**##+
* the start marker is above
*/
this should get replaced
/**##-*/
end marker is above
block.txt
THIS IS THE REPLACEMENT
Results
Running the command should change test.txt like so:
start of file
THIS IS THE REPLACEMENT
end marker is above
I am looking for the shortest, single-line solution in sed.
This might work for you (GNU sed):
sed '/^\/\*\*##+/,/^\/\*\*##-\*/cThis is the replacement' file
This changes the lines between the range to the required string.
To replace a range with contents of a file use:
sed -e '/^\/\*\*##+/!b;:a;N;/^\/\*\*##-\*/M!ba;r replacementFile' -e 'd' file
On encountering the start of the range set up a loop to gather up the range in the pattern space, then read the replacement file into the standard output and delete the contents of the pattern space.
Your start and end tags contain regex meta characters and /. sed only searches an input by regex and you need to escape / and all of those meta-characters in sed.
It is much easier to handle this in awk as awk allows non-regex plain text search also:
awk -v st='/**##+' -v et='/**##-*/' -v repl="$(<block.txt)" '
$0 == st{del=1} $0 == et{$0 = repl; del=0} !del' file
start of file
THIS IS THE REPLACEMENT
end marker is above
This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 6 years ago.
I'm trying to replace text in a file with the output of another command. Unfortunately, the outputted text contains characters bash expands. For example, I'm running the following script to change the file (somestring references output that would break the sed command):
#!/bin/bash
somestring='$6$sPnfj/lnXwZVrec7$fCnL9uy1oWIMZduInKTHBAxhsQxGCsBpm2XfVFFqDPHKidrd93yfjbYvKgYexXHVcvkKdu9lbfy16Ek5GvKy/1'
sed '0,/^title/s/^title*/'"$somestring"'\n&/' $HOME/example.txt
sed fails with this error:
sed: -e expression #1, char 30: unknown option to `s'
I think bash is substuting the contents of $somestring when building the sed command, but is then trying to expand the resulting text. I can't put the entire sed script in single quotes, I need bash to expand it the first time, just not the second. Any suggestions? Thanks
here the forward slash / is the problem. If it's the only issue you can set sed to use a different delimiter.
for example
$ somestring="abc/def"; echo xxx | sed 's/xxx/'"$somestring"'/'
sed: -e expression #1, char 11: unknown option to `s'
$ somestring="abc/def"; echo xxx | sed 's_xxx_'"$somestring"'_'
abc/def
you also need to worry about & and \ chars and escape them if can appear in the replacement text.
If you can't control the the replacement string, either you have to sanitize with another sed script or, alternatively use r command to read it from a file. For example,
$ seq 5 | sed -e '/3/{r replace' -e 'd}'
1
2
3slashes///1ampersand&and2backslashes\\end
4
5
where
$ cat replace
3slashes///1ampersand&and2backslashes\\end
You have several errors here:
the string somestring has characters that are significative for sed command (the most important being '/' that you are using as a delimiter) You can escape it, by substituting it with a previous
somestring=$(echo "$somestring" | sed -e 's/\//\\\//g')
that will convert your / chars to \/ sequences.
you are using sed '0,/^title/s/^title*/'"$somestring"'\n&/' $HOME/example.txt which is looking to substitute the string titl followed by any number of e characters by that $somestring value, followed by a new line and the original one. Unfortunately, sed(1) doesn't allow you to use newline characters in the pattern substitution side of the s command, but you can afford the result by using the i command with a text consisting of you pattern (preceding any new line by a \ to interpret it as literal):
Finally the script leads to:
#!/bin/bash
somestring='$6$sPnfj/lnXwZVrec7$fCnL9uy1oWIMZduInKTHBAxhsQxGCsBpm2XfVFFqDPHKidrd93yfjbYvKgYexXHVcvkKdu9lbfy16Ek5GvKy/1'
somestring=$(echo "$somestring" | sed -e 's/\//\\\//g')
sed '/^title/i\
'"$somestring\\
" $HOME/example.txt
If your shell is Bash, you can use parameter substitution to replace the problematic /:
somestring="{somestring//\//\\/}"
That looks scary, but is easier to understand if you look at the version that replaces x with __:
somestring="${somestring//x/__}"
It might be easier to use (say) underscore as the delimiter for your sed s command, and then the substitution above would be
somestring="${somestring//_/\\_}"
If you already have backslashes, you'll need to first replace those:
somestring="${somestring//\\/\\\\}"
somestring="{somestring//\//\\/}"
If there were other characters that needed escaping (e.g. on the search side of s///), then you could extend the above appropriately.
This URL provides the cleanest answer:
Command to escape a string in bash
printf "%q" "$someVariable"
will escape any characters you need escaped for you.
I have a string like this one:
#
pap
which basically translates to a \t#\n\tpap and I want to replace it with:
#
pap
python
which translates to \t#\n\tpap\n\tpython.
Tried this with sed in a lot of ways but it's not working maybe because sed uses new lines in a different way. I tried with:
sed -i "s/\t#\n\tpap/\t#\tpython\n\tpap/" /etc/freeradius/sites-available/default
...and many different other ways with no result. Any idea how can I do my replace in this situation?
try this line with gawk:
awk -v RS="\0" -v ORS="" '{gsub(/\t#\n\tpap/,"yourNEwString")}7' file
if you want to let sed handle new lines, you have to read the whole file first:
sed ':a;N;$!ba;s/\t#\n\tpap/NewString/g' file
This might work for you (GNU sed):
sed '/^\t#$/{n;/^\tpap$/{p;s//\tpython/}}' file
If a line contains only \t# print it, then if the next line contains only \tpap print it too, then replace that line with \tpython and print that.
A GNU sed solution that doesn't require reading the entire file at once:
sed '/^\t#$/ {n;/^\tpap$/a\\tpython'$'\n''}' file
/^\t#$/ matches comment-only lines (matching \t# exactly), in which case (only) the entire {...} expression is executed:
n loads and prints the next line.
/^\tpap/ matches that next line against \tpap exactly.
in case of a match, a\\tpython will then output \n\tpython before the following line is read - note that the spliced-in newline ($'\n') is required to signal the end of the text passed to the a command (you can alternatively use multiple -e options).
(As an aside: with BSD sed (OS X), it gets cumbersome, because
Control chars. such as \n and \t aren't directly supported and must be spliced in as ANSI C-quoted literals.
Leading whitespace is invariably stripped from the text argument to the a command, so a substitution approach must be used: s//&\'$'\n\t'python'/ replaces the pap line with itself plus the line to append:
sed '/^'$'\t''#$/ {n; /^'$'\t''pap$/ s//&\'$'\n\t'python'/;}' file
)
An awk solution (POSIX-compliant) that also doesn't require reading the entire file at once:
awk '{print} /^\t#$/ {f=1;next} f && /^\tpap$/ {print "\tpython"} {f=0}' file
{print}: prints every input line
/^\t#$/ {f=1;next}: sets flag f (for 'found') to 1 if a comment-only line (matching \t# exactly) is found and moves on to the next line.
f && /^\tpap$/ {print "\tpython"}: if a line is preceded by a comment line and matches \tpap exactly, outputs extra line \tpython.
{f=0}: resets the flag that indicates a comment-only line.
A couple of pure bash solutions:
Concise, but somewhat fragile, using parameter expansion:
in=$'\t#\n\tpap\n' # input string
echo "${in/$'\t#\n\tpap\n'/$'\t#\n\tpap\n\tpython\n'}"
Parameter expansion only supports patterns (wildcard expressions) as search strings, which limits the matching abilities:
Here the assumption is made that pap is followed by \n, whereas no assumption is made about what precedes \t#, potentially resulting in false positives.
If the assumption could be made that \t#\n\tpap is always enclosed in \n, echo "${in/$'\n\t#\n\tpap\n'/$'\n\t#\n\tpap\n\tpython\n'}" would work robustly; otherwise, see below.
Robust, but verbose, using the =~ operator for regex matching:
The =~ operator supports extended regular expressions on the right-hand side and thus allows more flexible and robust matching:
in=$'\t#\n\tpap' # input string
# Search string and string to append after.
search=$'\t#\n\tpap'
append=$'\n\tpython'
out=$in # Initialize output string to input string.
if [[ $in =~ ^(.*$'\n')?("$search")($'\n'.*)?$ ]]; then # perform regex matching
out=${out/$search/$search$append} # replace match with match + appendage
fi
echo "$out"
You can just translate the character \n to another one, then apply sed, then apply the reverse translation. If tr is used, it must be a 1-byte character, for instance \v (vertical tabulation, nowadays almost unused).
cat FILE|tr '\n' '\v'|sed 's/\t#\v\tpap/&\v\tpython/'|tr '\v' '\n'|sponge FILE
or, without sponge:
cat FILE|tr '\n' '\v'|sed 's/\t#\v\tpap/&\v\tpython/'|tr '\v' '\n' >FILE.bak && mv FILE.bak FILE
I have a text file which has a particular line something like
sometext sometext sometext TEXT_TO_BE_REPLACED sometext sometext sometext
I need to replace the whole line above with
This line is removed by the admin.
The search keyword is TEXT_TO_BE_REPLACED
I need to write a shell script for this. How can I achieve this using sed?
You can use the change command to replace the entire line, and the -i flag to make the changes in-place. For example, using GNU sed:
sed -i '/TEXT_TO_BE_REPLACED/c\This line is removed by the admin.' /tmp/foo
You need to use wildcards (.*) before and after to replace the whole line:
sed 's/.*TEXT_TO_BE_REPLACED.*/This line is removed by the admin./'
The Answer above:
sed -i '/TEXT_TO_BE_REPLACED/c\This line is removed by the admin.' /tmp/foo
Works fine if the replacement string/line is not a variable.
The issue is that on Redhat 5 the \ after the c escapes the $. A double \\ did not work either (at least on Redhat 5).
Through hit and trial, I discovered that the \ after the c is redundant if your replacement string/line is only a single line. So I did not use \ after the c, used a variable as a single replacement line and it was joy.
The code would look something like:
sed -i "/TEXT_TO_BE_REPLACED/c $REPLACEMENT_TEXT_STRING" /tmp/foo
Note the use of double quotes instead of single quotes.
The accepted answer did not work for me for several reasons:
my version of sed does not like -i with a zero length extension
the syntax of the c\ command is weird and I couldn't get it to work
I didn't realize some of my issues are coming from unescaped slashes
So here is the solution I came up with which I think should work for most cases:
function escape_slashes {
sed 's/\//\\\//g'
}
function change_line {
local OLD_LINE_PATTERN=$1; shift
local NEW_LINE=$1; shift
local FILE=$1
local NEW=$(echo "${NEW_LINE}" | escape_slashes)
# FIX: No space after the option i.
sed -i.bak '/'"${OLD_LINE_PATTERN}"'/s/.*/'"${NEW}"'/' "${FILE}"
mv "${FILE}.bak" /tmp/
}
So the sample usage to fix the problem posed:
change_line "TEXT_TO_BE_REPLACED" "This line is removed by the admin." yourFile
All of the answers provided so far assume that you know something about the text to be replaced which makes sense, since that's what the OP asked. I'm providing an answer that assumes you know nothing about the text to be replaced and that there may be a separate line in the file with the same or similar content that you do not want to be replaced. Furthermore, I'm assuming you know the line number of the line to be replaced.
The following examples demonstrate the removing or changing of text by specific line numbers:
# replace line 17 with some replacement text and make changes in file (-i switch)
# the "-i" switch indicates that we want to change the file. Leave it out if you'd
# just like to see the potential changes output to the terminal window.
# "17s" indicates that we're searching line 17
# ".*" indicates that we want to change the text of the entire line
# "REPLACEMENT-TEXT" is the new text to put on that line
# "PATH-TO-FILE" tells us what file to operate on
sed -i '17s/.*/REPLACEMENT-TEXT/' PATH-TO-FILE
# replace specific text on line 3
sed -i '3s/TEXT-TO-REPLACE/REPLACEMENT-TEXT/'
for manipulation of config files
i came up with this solution inspired by skensell answer
configLine [searchPattern] [replaceLine] [filePath]
it will:
create the file if not exists
replace the whole line (all lines) where searchPattern matched
add replaceLine on the end of the file if pattern was not found
Function:
function configLine {
local OLD_LINE_PATTERN=$1; shift
local NEW_LINE=$1; shift
local FILE=$1
local NEW=$(echo "${NEW_LINE}" | sed 's/\//\\\//g')
touch "${FILE}"
sed -i '/'"${OLD_LINE_PATTERN}"'/{s/.*/'"${NEW}"'/;h};${x;/./{x;q100};x}' "${FILE}"
if [[ $? -ne 100 ]] && [[ ${NEW_LINE} != '' ]]
then
echo "${NEW_LINE}" >> "${FILE}"
fi
}
the crazy exit status magic comes from https://stackoverflow.com/a/12145797/1262663
In my makefile I use this:
#sed -i '/.*Revision:.*/c\'"`svn info -R main.cpp | awk '/^Rev/'`"'' README.md
PS: DO NOT forget that the -i changes actually the text in the file... so if the pattern you defined as "Revision" will change, you will also change the pattern to replace.
Example output:
Abc-Project written by John Doe
Revision: 1190
So if you set the pattern "Revision: 1190" it's obviously not the same as you defined them as "Revision:" only...
bash-4.1$ new_db_host="DB_HOSTNAME=good replaced with 122.334.567.90"
bash-4.1$
bash-4.1$ sed -i "/DB_HOST/c $new_db_host" test4sed
vim test4sed
'
'
'
DB_HOSTNAME=good replaced with 122.334.567.90
'
it works fine
To do this without relying on any GNUisms such as -i without a parameter or c without a linebreak:
sed '/TEXT_TO_BE_REPLACED/c\
This line is removed by the admin.
' infile > tmpfile && mv tmpfile infile
In this (POSIX compliant) form of the command
c\
text
text can consist of one or multiple lines, and linebreaks that should become part of the replacement have to be escaped:
c\
line1\
line2
s/x/y/
where s/x/y/ is a new sed command after the pattern space has been replaced by the two lines
line1
line2
cat find_replace | while read pattern replacement ; do
sed -i "/${pattern}/c ${replacement}" file
done
find_replace file contains 2 columns, c1 with pattern to match, c2 with replacement, the sed loop replaces each line conatining one of the pattern of variable 1
To replace whole line containing a specified string with the content of that line
Text file:
Row: 0 last_time_contacted=0, display_name=Mozart, _id=100, phonebook_bucket_alt=2
Row: 1 last_time_contacted=0, display_name=Bach, _id=101, phonebook_bucket_alt=2
Single string:
$ sed 's/.* display_name=\([[:alpha:]]\+\).*/\1/'
output:
100
101
Multiple strings delimited by white-space:
$ sed 's/.* display_name=\([[:alpha:]]\+\).* _id=\([[:digit:]]\+\).*/\1 \2/'
output:
Mozart 100
Bach 101
Adjust regex to meet your needs
[:alpha] and [:digit:]
are Character Classes and Bracket Expressions
This worked for me:
sed -i <extension> 's/.*<Line to be replaced>.*/<New line to be added>/'
An example is:
sed -i .bak -e '7s/.*version.*/ version = "4.33.0"/'
-i: The extension for the backup file after the replacement. In this case, it is .bak.
-e: The sed script. In this case, it is '7s/.*version.*/ version = "4.33.0"/'. If you want to use a sed file use the -f flag
s: The line number in the file to be replaced. In this case, it is 7s which means line 7.
Note:
If you want to do a recursive find and replace with sed then you can grep to the beginning of the command:
grep -rl --exclude-dir=<directory-to-exclude> --include=\*<Files to include> "<Line to be replaced>" ./ | sed -i <extension> 's/.*<Line to be replaced>.*/<New line to be added>/'
The question asks for solutions using sed, but if that's not a hard requirement then there is another option which might be a wiser choice.
The accepted answer suggests sed -i and describes it as replacing the file in-place, but -i doesn't really do that and instead does the equivalent of sed pattern file > tmp; mv tmp file, preserving ownership and modes. This is not ideal in many circumstances. In general I do not recommend running sed -i non-interactively as part of an automatic process--it's like setting a bomb with a fuse of an unknown length. Sooner or later it will blow up on someone.
To actually edit a file "in place" and replace a line matching a pattern with some other content you would be well served to use an actual text editor. This is how it's done with ed, the standard text editor.
printf '%s\n' '/TEXT_TO_BE_REPLACED/' d i 'This line is removed by the admin' . w q | \
ed -s /tmp/foo > /dev/null
Note that this only replaces the first matching line, which is what the question implied was wanted. This is a material difference from most of the other answers.
That disadvantage aside, there are some advantages to using ed over sed:
You can replace the match with one or multiple lines without any extra effort.
The replacement text can be arbitrarily complex without needing any escaping to protect it.
Most importantly, the original file is opened, modified, and saved. A copy is not made.
How it works
How it works:
printf will use its first argument as a format string and print each of its other arguments using that format, effectively meaning that each argument to printf becomes a line of output, which is all sent to ed on stdin.
The first line is a regex pattern match which causes ed to move its notion of "the current line" forward to the first line that matches (if there is no match the current line is set to the last line of the file).
The next is the d command which instructs ed to delete the entire current line.
After that is the i command which puts ed into insert mode;
after that all subsequent lines entered are written to the current line (or additional lines if there are any embedded newlines). This means you can expand a variable (e.g. "$foo") containing multiple lines here and it will insert all of them.
Insert mode ends when ed sees a line consisting of .
The w command writes the content of the file to disk, and
the q command quits.
The ed command is given the -s switch, putting it into silent mode so it doesn't echo any information as it runs,
the file to be edited is given as an argument to ed,
and, finally, stdout is thrown away to prevent the line matching the regex from being printed.
Some Unix-like systems may (inappropriately) ship without an ed installed, but may still ship with an ex; if so you can simply use it instead. If have vim but no ex or ed you can use vim -e instead. If you have only standard vi but no ex or ed, complain to your sysadmin.
It is as similar to above one..
sed 's/[A-Za-z0-9]*TEXT_TO_BE_REPLACED.[A-Za-z0-9]*/This line is removed by the admin./'
Below command is working for me. Which is working with variables
sed -i "/\<$E\>/c $D" "$B"
I very often use regex to extract data from files I just used that to replace the literal quote \" with // nothing :-)
cat file.csv | egrep '^\"([0-9]{1,3}\.[0-9]{1,3}\.)' | sed s/\"//g | cut -d, -f1 > list.txt