I'm new at bash scripting and I have a problem. I want to replace one line in a file with another one. This is my file:
/home/iosub/linux/fis2 b7c839bf081421804e15c51d65a6f8fc -
/home/iosub/linux/e212CIosub/doi 7edd241b1f05a9ea346c085b8c9e95c5 -
/home/iosub/linux/e212CIosub/test2 7edd241b1f05a9ea346c085b8c9e95c5 -
/home/iosub/linux/e212CIosub/xab ed4940ef42ec8f72fa8bdcf506a13f3d -
/home/iosub/linux/e212CIosub/test1 8af599cfb361efb9cd4c2ad7b4c4e53a -
/home/iosub/linux/e212CIosub/xaa 8cf57351b5fc952ec566ff865c6ed6bd -
/home/iosub/linux/e212CIosub/test3 74420c2c4b90871894aa51be38d33f2c -
Without the blank lines. I want to replace a line with another one.
for example /home/iosub/linux/e212CIosub/test1 8af599cfb361efb9cd4c2ad7b4c4e53a -
with /home/iosub/linux/e212CIosub/test1 d2199e312ecd00ff5f9c12b7d54c97f1 -
I have /home/iosub/linux/e212CIosub/test1 in a variable and the new code in another variable.
I know that I must use sed. How can I do it?
I've tried a lot of combinations like:
sed "/$1/d" cont > cont2;
where $1 is /home/iosub/linux/e212CIosub/test1
And after that I concatenate the new string to the file. This places the new line at the end of the file, it would be a good solution, if it would work . . . But it doesn't.It gives an error: sed: -e expression #1, char 5: extra characters after command
I've also tried the replacement method, but I didn't get any result.
Any solution would be good. Thank's
sed "s#$1#$var#g" -i filename
This will replace all occurences of $1 with $var in file filename inplace.
edit added '\' in front of # chars for initial regex delimiters
when you have a '/' char in your search pattern, it breaks the use of that character ('/') as a reg-exp delimiter. Use a Regex delimiter that is NOT in your regex, i.e.
sed "\#$1#d" cont > cont2
Then continue with your processing.
But given your example, why don't you do something like
sed "s\#\($1\)(.*$)#\1 $2 -#" cont > cont2
This matches your value from $1, essentially deletes the text after your $1 value, and prints the $1 value followed by a space, your $2 value (d2....), a space and the ending '-' character.
Edit (Finally), I'll mention that some legacy Unix sed's (aix in particular) seem to not accept escaping the regex delimiter char (but I don't have access to a system to verify that now). In that case, you have to escape all of the any-and-all '/'s in your regex search pattern, like \/home\/iosub\/linux\/e212CIosub\/test1, yikes!
I hope this helps.
Related
I got a file with ^$ as delimiter, the text is like :
tony^$36^$developer^$20210310^$CA
I want to replace the datetime.
I tried awk -F '\^\$' '{print $4}' file.txt | sed -i '/20210310/20221210/' , but it returns nothing. Then I tried the awk part, it returns nothing, I guess it still treat the line as a whole and the delimiter doesn't work. Wondering why and how to solve it?
A simple solution would be:
sed 's/\^\$/\n/g; s/20210310/20221210/g' -i file.txt
which will modify the file to separate each section to a new line.
If you need a different delimiter, change the \n in the command to maybe space or , .. up to you.
And it will also replace the date in the file.
If you want to see the changes, and really modify the file, remove the -i from the command.
When I run your awk command, I get these warnings:
awk: warning: escape sequence `\^' treated as plain `^'
awk: warning: escape sequence `\$' treated as plain `$'
That explains why your output is blank: the field delimiter is interpreted as the regular expression '^$', which matches a completely blank line (only). As a result, each non-blank line of input is without any field separators, and therefore has only a single field. $4 can be non-empty only if there are at least four fields.
You can fix that by escaping the backslashes:
awk -F '\\^\\$' '{print $4}' file.txt
If all you want to do is print the modified datecodes py themselves, then that should get you going. However, the question ...
How to extract and replace columns with a multi-character delimiter?
... sounds like you may want actually to replace the datecode within each line, keeping the rest intact. In that case, it is a non-starter for the awk command to discard the other parts of the line. You have several options here, but two of the more likely would be
instead of sending field 4 out to sed for substitution, do the sub in the awk script, and then reconstitute the input line by printing all fields, with the expected delimiters. (This is left as an exercise.) OR
do the whole thing in sed:
sed -E 's/^((([^^]|\^[^$])*\^\$){3})20210310(\^\$.*)/\120221210\4/' file.txt
If you wanted to modify file.txt in-place then you could add the -i flag (which, on the other hand, is not useful in your original command, where sed's input is coming from a pipe rather than a file).
The -E option engages the POSIX extended regex dialect, which allows the given regex to be more readable (the alternative would require a bunch more \ characters).
Overall, presuming that there are five or more fields delimited by literal '^$' strings, and the fourth contains exactly "20210310", that matches the first three fields, including their trailing delimiters, and captures them all as group 1; matches the leading delimiter of the fifth field and all the remainder of the line and captures it as group 4; and substitutes replaces the whole line with group 1 followed by the new datecode followed by group 4.
How can i replace the following text in a file in linux with a different line
Current :
0 22 * * * /scripts/application_folder_backup.sh >> /var/log/application_folder_backup.log
Replacement line : #line_removed
I tried using sed but my text in the file already has a / which is causing problems. I tried storing the string in a variable too. But it doesn't work
#!/bin/bash
var="0 22 * * * /scripts/application_folder_backup.sh >> /var/log/application_folder_backup.log"
sed -i -e 's/$var/#line_removed/g' /tmp/k1.txt
exit
Just / is not a problem here, even * or all the special regex meta characters will be a problem for sed since it uses only regex for search patterns.
Better to use this non-regex based awk command:
awk -v var="$var" 'index($0, var) { $0 = "#line_removed" } 1' file
#line_removed
index function in awk uses plain text search instead of a regex based search.
I suggest previous to feed the contents of $var to the sed script, to escape all the / chars to be \/, as in:
var_esc=$(echo "$var" | sed 's/\//\\\//g')
But the sed expression gets very complicated (you must use single quotes, if you don't want to double \ chars, as double quotes do interpret also the backslashes.
Another thing that could simplify the expression is to use the possibility of change the regexp delimiter (with using a different char to begin it) as in:
var_esc=$(echo "$var" | sed 's:/:\/:g')
The idea is to, before substituting the $var variable, to escape all the possible interferring chars you can have (you don't know a priory if there are going to be, as you are using a variable for that purpose) and then substitute them in the final command:
sed "s/$var_esc/#line_removed/g"
Finally, if you don't want to substitute the line, but just to erase it, you can do something like this:
sed "/$var_esc/d"
and that will erase all lines that match your $var_esc contents. (this time I believe you cannot change the regexp delimiter, as the / introduces the matching line operator --- d is the command to delete line in the output)
Another way to delete the line in your file is to call ex directly and pass the editor command as input:
ex file <<EOF <-- edit "file" with the commands that follow until EOF is found.
/$var_esc <-- search for first occurrence of $var_esc
d <-- delete line
w <-- write file
EOF <-- eof marker.
I have a string like this one:
#
pap
which basically translates to a \t#\n\tpap and I want to replace it with:
#
pap
python
which translates to \t#\n\tpap\n\tpython.
Tried this with sed in a lot of ways but it's not working maybe because sed uses new lines in a different way. I tried with:
sed -i "s/\t#\n\tpap/\t#\tpython\n\tpap/" /etc/freeradius/sites-available/default
...and many different other ways with no result. Any idea how can I do my replace in this situation?
try this line with gawk:
awk -v RS="\0" -v ORS="" '{gsub(/\t#\n\tpap/,"yourNEwString")}7' file
if you want to let sed handle new lines, you have to read the whole file first:
sed ':a;N;$!ba;s/\t#\n\tpap/NewString/g' file
This might work for you (GNU sed):
sed '/^\t#$/{n;/^\tpap$/{p;s//\tpython/}}' file
If a line contains only \t# print it, then if the next line contains only \tpap print it too, then replace that line with \tpython and print that.
A GNU sed solution that doesn't require reading the entire file at once:
sed '/^\t#$/ {n;/^\tpap$/a\\tpython'$'\n''}' file
/^\t#$/ matches comment-only lines (matching \t# exactly), in which case (only) the entire {...} expression is executed:
n loads and prints the next line.
/^\tpap/ matches that next line against \tpap exactly.
in case of a match, a\\tpython will then output \n\tpython before the following line is read - note that the spliced-in newline ($'\n') is required to signal the end of the text passed to the a command (you can alternatively use multiple -e options).
(As an aside: with BSD sed (OS X), it gets cumbersome, because
Control chars. such as \n and \t aren't directly supported and must be spliced in as ANSI C-quoted literals.
Leading whitespace is invariably stripped from the text argument to the a command, so a substitution approach must be used: s//&\'$'\n\t'python'/ replaces the pap line with itself plus the line to append:
sed '/^'$'\t''#$/ {n; /^'$'\t''pap$/ s//&\'$'\n\t'python'/;}' file
)
An awk solution (POSIX-compliant) that also doesn't require reading the entire file at once:
awk '{print} /^\t#$/ {f=1;next} f && /^\tpap$/ {print "\tpython"} {f=0}' file
{print}: prints every input line
/^\t#$/ {f=1;next}: sets flag f (for 'found') to 1 if a comment-only line (matching \t# exactly) is found and moves on to the next line.
f && /^\tpap$/ {print "\tpython"}: if a line is preceded by a comment line and matches \tpap exactly, outputs extra line \tpython.
{f=0}: resets the flag that indicates a comment-only line.
A couple of pure bash solutions:
Concise, but somewhat fragile, using parameter expansion:
in=$'\t#\n\tpap\n' # input string
echo "${in/$'\t#\n\tpap\n'/$'\t#\n\tpap\n\tpython\n'}"
Parameter expansion only supports patterns (wildcard expressions) as search strings, which limits the matching abilities:
Here the assumption is made that pap is followed by \n, whereas no assumption is made about what precedes \t#, potentially resulting in false positives.
If the assumption could be made that \t#\n\tpap is always enclosed in \n, echo "${in/$'\n\t#\n\tpap\n'/$'\n\t#\n\tpap\n\tpython\n'}" would work robustly; otherwise, see below.
Robust, but verbose, using the =~ operator for regex matching:
The =~ operator supports extended regular expressions on the right-hand side and thus allows more flexible and robust matching:
in=$'\t#\n\tpap' # input string
# Search string and string to append after.
search=$'\t#\n\tpap'
append=$'\n\tpython'
out=$in # Initialize output string to input string.
if [[ $in =~ ^(.*$'\n')?("$search")($'\n'.*)?$ ]]; then # perform regex matching
out=${out/$search/$search$append} # replace match with match + appendage
fi
echo "$out"
You can just translate the character \n to another one, then apply sed, then apply the reverse translation. If tr is used, it must be a 1-byte character, for instance \v (vertical tabulation, nowadays almost unused).
cat FILE|tr '\n' '\v'|sed 's/\t#\v\tpap/&\v\tpython/'|tr '\v' '\n'|sponge FILE
or, without sponge:
cat FILE|tr '\n' '\v'|sed 's/\t#\v\tpap/&\v\tpython/'|tr '\v' '\n' >FILE.bak && mv FILE.bak FILE
I have a file withe following data
:1:aaaaa:aaa:aaa
and i want to remove the leading colon using bash to be like
1:aaaaa:aaa:aaa
You could use sed:
sed 's/^://' filename
^ denotes the start of line, so ^: would match a colon at the beginning of a line. Replace it by nothing!
str=':1:aaaaa:aaa:aaa'
echo ${str:1} #=> 1:aaaaa:aaa:aaa
Resources: Bash string manipulation
As well as sed, you may get better performance for such a simple operation on a large file by using cut:
cut myfile -d : -f 2-
You can also extract additional fields this way with other -f values.
If you want to remove the leading colon from data in a variable, such as in a loop, you can also do
myvar=":1:aaaaa:aaa:aaa"
echo ${myvar#:}
I'm trying to write a bash script to configure a server and I need to change the line:
listen = /var/run/php5-fpm.sock
to equal the following:
listen = 127.0.0.1:9000
in the file:
/etc/php5/fpm/pool.d/www.conf
So I've been looking at tutorials for using sed and I've tried the following command to no avail:
$~: sed -i 's//var/run/php5-fpm.sock/127.0.0.1:9000/g' /etc/php5/fpm/pool.d/www.conf
$~: sed: -e expression #1, char 8: unknown option to `s'
I've tried escaping the forward slash with a backslash: '/' but I think I'm on the wrong track. There must be a better way to do this?
Thanks for your help.
This is because you are trying to replace the character '/' in the pattern, and this character is used to delimit the 's///' expression. You have two choices, you can escape every '/' character with '/'or - and this is the one I prefer, use a different character to delimit the pattern and replacement string - I tend to use '!'
The character immediately after the 's' is used to delimit the expressions.
sed -i 's!/var/run/php5-fpm.sock!127.0.0.1:9000!g' /etc/php5/fpm/pool.d/www.conf
I actually have got into the habit of ALWAYS using '!' for sed, and perl - as you end up having to escape less characters and ultimately save time.