Sed variable expansion and \n match when trying to append some text - linux

Question

I've the following template file
# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
#http_access allow localnet
#http_access allow localhost
I want to append some text after the line bellow using sed (this is a requirement, I can't use another tool)
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
I'm tried to use the following command:
sed "/# INSERT YOUR OWN RULE/aTEXTTOAPPEND" test.txt
The result:
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
TEXTTOAPPEND
#
The old code used perl to do the job
perl -i -0pe 's/(#( |\t)*\n# INSERT YOUR OWN RULE.*\n#( |\t)*\n)/\1\n'"$REPLACE"'\n/g' /etc/squid/squid.conf
I'm facing two problems:
1 - Not being able to use a varible in this append command
I tried so far:
$ echo $REPLACE
TEXTTOAPPEND
$ sed "/# INSERT YOUR OWN RULE/a${REPLACE}" test.txt
sed: -e expression #1, char 53: unknown command: `B'
sed "/# INSERT YOUR OWN RULE/a\${REPLACE}" test.txt
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
${REPLACE}
#
2 - Not being able to match the hash character after the line I want to append the text (when I add the \n the command stop matching, so, nothing is added in this case)
sed "/# INSERT YOUR OWN RULE.*\n#/aTEXTTOAPPEND" test.txt
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
Expected output:
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
TEXTTOAPPEND
Can someone point what I'm missing here? For variable expansion I thought using double quotes and ${var} would do the job.
Update:
I was trying everything using git-bash, when trying in a real linux machine the command bellow worked:
echo $REPLACE
ABC
sed "/# INSERT YOUR OWN RULE.*/a${REPLACE}" test.txt
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
ABC
#
the only problem now is the 2. how to work with the \n in this case?
sed "/# INSERT YOUR OWN RULE.*\n#.*/a${REPLACE}" test.txt
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

Answer 1

Can someone point what I'm missing here?
Commands in sed are separated by a newline. Sed sees a newline - it assumes the command ends here. I can reproduce your sed: -e expression #1, char 53: unknown command: 'B' with simple variable that has a newline and the next line starts with B:
replace="something
B something"
sed "/# INSERT YOUR OWN RULE/a${replace}"
sed sees asomething and appends something to the output and newline terminates the a command. Then it sees B something and tries to parse that as a command, but B is invalid.
The most safest way to append content of the variable with sed is to use a temporary file with r command. Note, that you need a newline after r command after filename, because any ; will be parsed as part of the filename! In bash, you can be smart and combine process substitution with a here string to create a temporary file descriptor*. To output current pattern space and read next line into pattern space in sed use n command. Like this:
sed '/# INSERT YOUR OWN RULE/{n;r'<(cat <<<"$replace")$'\n}'
It will append the content of replace after the next line after the regex. Note that after r there is $'\n' - a newline.
* Just a <(echo "$replace") would work too, but I somehow feel the cat <<<"$replace" will be better in memory consumption for big strings, I didn't check that in any way.
The following script:
replace="anything
can
be here!"
cat <<EOF |
#http_access deny to_localhost
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
# Example rule allowing access from your local networks.
EOF
sed '/# INSERT YOUR OWN RULE/{n;r'<(cat <<<"$replace")$'\n}'
outputs on repl:
#http_access deny to_localhost
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
anything
can
be here!
# Example rule allowing access from your local networks.
how to work with the \n in this case?
Sed reads one line at a time. As this is very similar, I would just point to this answer I did just yesterday that deals with the same problem. The script in sed this case would need to buffer two lines at a time with N command:
sed '
: restart
N # buffer two lines
: loop
# match two lines
/# INSERT YOUR OWN RULE.*\n#/{
r'<(cat <<<"$replace")'
# print and start over
n ; b restart
}
# hold, print leading line, change, remove leading line
h ; s/\n.*// ; p ; x ; s/[^\n]*\n//
# append next line and loop
N
b loop
'
But you can't do rsomething or asomething with sed -z because then records would be separated by zero, so sed reads the whole file, so asomething would be displayed after the whole file. Well, you can test it sed -z '/# INSERT YOUR OWN RULE.*\n#.*/r'<(cat <<<$replace) and it will just print the content of $replace on the end of the file.

Answer 2

This might work for you (GNU sed):
cat <<! | sed 'N;/^# INSERT YOUR OWN RULE.*\n#/!{P;D};r /dev/stdin' file
Text to be appended or a variable
$var
or both $var
!
Construct a here document to be appended and pipe it through to the sed invocation.
The sed invocation uses the N and the P;D commands to open a two line window throughout the length of the file but on matching the required two lines invokes the r command which appends the former here document.
An alternative:
sed '1{x;s/^/'"$var"'/;x};N;/^# INSERT YOUR OWN RULE.*\n#/!{P;D};G' file
But,this should work too:
sed 'N;/^# INSERT YOUR OWN RULE.*\n#/!{P;D};a\'"$var" file

Answer 3

Sometimes the simplest way is the best, maybe this is the case?
sed '8aTEXTTOAPPEND' -i test.txt
This will add 'TEXTTOAPPEND' after 8-th string.
And automate this
N=$(grep -n 'INSERT YOUR OWN RULE' test.txt) # get line number
N=${N%%:*} # remove all except line number
((N++)) # inc line number coz we got this line with #
sed "${N}aTEXTTOAPPEND" -i test.txt # add text after $Nth line
Oneliner
N=$(grep -n 'INSERT YOUR OWN RULE' test.txt); N=${N%%:*}; ((N++)); sed "${N}aTEXTTOAPPEND" -i test.txt

Answer 4

sed "/INSERT YOUR OWN RULE/a '\n'TEXTTOAPPEND" file-for-change |
sed "/INSERT YOUR OWN RULE/{n;s/'/#/;n;s/^'//;n;d"}
Second sed is for change and removing "'" leading symbol at start of next two lines
Or, more simple variant:
sed -r '/INSERT YOUR/,+1{s/(#$)/\1\nTEXTTOAPPEND/}'
It works with GNU sed.