Shell execute grep command within sed replacement not as expected - linux

Question

I am working on some deployment script for Kubernetes, and want to execute a single command to replace some "Template Variables" in yaml file using sed.
I have the following (example, shortened yaml file) input file where i want to do the replacements:
input.txt
CONTAINER_ADDITIONAL
spec:
selector:
app: CONTAINER_NAME
type: NodePort
ports:
- protocol: TCP
port: EXPOSED_PORT
Also, I've got the following Dockerfile where i want to get the EXPOSE port number from to be inserted into the EXPOSED_PORT:
Dockerfile
FROM node:latest
EXPOSE 3000
WORKDIR /app
I have now tried to use different approaches to get the port number 3000 (from Dockerfile) inserted into the EXPOSED_PORT
(CONTAINER_NAME and CONTAINER_ADDITIONAL are working, they are in the example file for completenes with the command below).
The following commands can be executed directly in the shell and give the wanted result (3000):
cat Dockerfile | grep EXPOSE | cut -d" " -f2 -> there may be still the \n
cat Dockerfile | grep EXPOSE | cut -d" " -f2 | tr -d "\n" -> previously mentionned \n removed
grep EXPOSE Dockerfile | cut -d" " -f2 -> with \n
grep EXPOSE Dockerfile | cut -d" " -f2 | tr -d "\n" -> without \n
grep EXPOSE Dockerfile | awk '{print $2}' -> with \n, uses single quote - not an option?
grep EXPOSE Dockerfile|tr -d -c 0-9 -> without \n, not prefered (when situation would exist of multiple port numbers separated by spaces)
grep -Po "(?<=^EXPOSE )\w*$" Dockerfile -> with \n, not prefered (multiple port numbers)
grep -Po "(?<=^EXPOSE )\w*$" Dockerfile | tr -d "\n" -> without \n
HOWEVER:
sed -e 's#CONTAINER_NAME#some-container-name#g;s#CONTAINER_ADDITIONAL#cat some_config.txt#e;s#EXPOSED_PORT#grep EXPOSE Dockerfile | cut -d" " -f2 | tr -d "\n"#e' input.txt
Does not work for the EXPOSED_PORT. The other two variables CONTAINER_NAME and CONTAINER_ADDITIONAL work (the cat gets executed, content of some_config.txt is being put in there)
No matter which of the above mentioned commands that are working and giving the correct result directly in shell, they do not work when executed in sed (the awk for sure not, because of single quotes).
The output I get is:
inserted_content_of: some_config.txt
some_more_inserted_content_from: some_config.txt
spec:
selector:
app: some-container-name
type: NodePort
ports:
- protocol: TCP
sh: 1: port:: not found
The expected output that i want to have:
inserted_content_of: some_config.txt
some_more_inserted_content_from: some_config.txt
spec:
selector:
app: some-container-name
type: NodePort
ports:
- protocol: TCP
port: 3000
Is there anything that I am doing wrong with the sed command?
Is there some explanation what goes wrong?
How can i solve this issue?

Answer 1

I would think, that
grep EXPOSE input.txt | cut -d" " -f2
is a good way to get the portnumber (the \n is not appended when used in another command). Perhaps you should save it first before your next step.
portnumber=$(grep EXPOSE input.txt | cut -d" " -f2)
echo "The portnumber will be [${portnumber}]."
I will replace your grep command with echo "4444", showing the problem with your nested command.
With #e you ask sed to execute the resulting string after processing s#EXPOSED_PORT#echo "4444"#. The line with EXPOSED_PORT is
port: EXPOSED_PORT
So sed is trying to execute port echo "4444", and complains about the command port.
When you want to use the #e, you should use something like
sed -r 's#(.*)(EXPOSED_PORT)(.*)#echo "\1$(echo "4444")\3"#e' input.txt
And you thought it would be easy? Try like this:
sed 's#EXPOSED_PORT#'${portnumber}'#' input.txt
# or when you really want to squeeze your command into this line
sed 's#EXPOSED_PORT#'$(echo "4444")'#' input.txt
Or look at awk:
awk -v port=${portnumber} '/EXPOSED_PORT/ {$2=port} 1' input.txt
# or nested
awk -v port=$(echo "4444") '/EXPOSED_PORT/ {$2=port} 1' input.txt

Answer 2

The issue is that you are not changing the whole line when changing the port but rather just a section of the line. Because the leading "port:" has been left, sed tries to execute this along with the grep command and hence the error. To overcome, this, search and replace the whole line and so:
sed -e 's#CONTAINER_NAME#some-container-name#g;s#CONTAINER_ADDITIONAL#cat some_config.txt#e;s#^.*EXPOSED_PORT#echo " port: $(grep EXPOSE input.txt | cut -d" " -f2 | tr -d "\n")"#e' input.txt
Echo out the leading port: along with the expanded grep command.