This question already has answers here:
How to cat <<EOF >> a file containing code?
(5 answers)
Closed 5 years ago.
I'm trying to create a script file using substitution string from ENV but want also to prevent some from escaping
export PLACEHOLDER1="myPlaceholder1Value"
sudo /bin/su -c "cat << EOF > /etc/init.d/my-script
#!/bin/bash
myvariable_1=toto$PLACEHOLDER1
myvariable_final=\"dynamicvar=\${myvariable_1},\${myvariable_2}\"
EOF
"
It results in which is not good as the myvariable_final are not escaped and substituted as the one from the init script dependencies ($remote_fs, $syslog, $network, $time)
#!/bin/bash
myvariable_1=totomyPlaceholder1Value
myvariable_2=titimyPlaceholder2Value
myvariable_final="dynamicvar=,"
If i try to put a backslash \ behind the dollars $, I manage to avoid the substitution but I getting an unwanted backslash \:
export PLACEHOLDER1="myPlaceholder1Value"
export PLACEHOLDER2="myPlaceholder2Value"
sudo /bin/su -c "cat << EOF > /etc/init.d/my-script
#!/bin/bash
myvariable_1=toto$PLACEHOLDER1
myvariable_2=titi$PLACEHOLDER2
myvariable_final=\"dynamicvar=\$\{myvariable_1},\$\{myvariable_2}\"
EOF
"
results in:
#!/bin/bash
myvariable_1=totomyPlaceholder1Value
myvariable_2=titimyPlaceholder2Value
myvariable_final="dynamicvar=$\{myvariable_1},$\{myvariable_2}"
Wanted/attended result whould have been :
#!/bin/bash
myvariable_1=totomyPlaceholder1Value
myvariable_2=titimyPlaceholder2Value
myvariable_final="dynamicvar=${myvariable_1},${myvariable_2}"
solved by putting quote around the EOF as below and using backslash to control the escaping when needed
export PLACEHOLDER1="myPlaceholder1Value"
export PLACEHOLDER2="myPlaceholder2Value"
sudo /bin/su -c "cat << 'EOF' > /etc/init.d/my-script
#!/bin/bash
myvariable_1=toto$PLACEHOLDER1
myvariable_2=titi$PLACEHOLDER2
myvariable_final=\"dynamicvar=\${myvariable_1},\${myvariable_2}\"
EOF
"
Just use 'EOF' to prevent the variable from expanding:
sudo /bin/su -c "cat << 'EOF' > /etc/init.d/my-script
# ^ ^
From man bash:
Here Documents
This type of redirection instructs the shell to read input from the
current source until a line containing only delimiter (with no
trailing blanks) is seen. All of the lines read up to that point are
then used as the standard input for a command.
The format of here-documents is:
<<[-]word
here-document
delimiter
No parameter expansion, command substitution, arithmetic expansion,
or pathname expansion is performed on word. If any characters in word
are quoted, the delimiter is the result of quote removal on word, and
the lines in the here-document are not expanded. If word is
unquoted, all lines of the here-document are subjected to parameter
expansion, command substitution, and arithmetic expansion. In the
latter case, the character sequence \<newline> is ignored, and \
must be used to quote the characters \, $, and `.
when using the su command put the command itself in sigle quotes and just escape the $ with a backslash. the placeholder variables has to set in command bash context (here after su). so you need to do sth like
su -c 'ph="ph"; cat << EOF > script
varinscript=$ph
var=\${var}
EOF'
Related
Is it possible to create a heredoc that does not become subject to variable expansion?
e.g.
cat <<-EOF > somefile.sh
Do not print current value of $1 instead evaluate it later.
EOF
Update I am aware of escaping by \. My actual heredoc has many variables in it - and it is error prone and tedious to escape all of them.
Quote the delimiter:
cat <<-"EOF" > somefile.sh
Do not print current value of $1 instead evaluate it later.
EOF
This results in:
$ cat somefile.sh
Do not print current value of $1 instead evaluate it later.
Documentation
The format of here-documents is:
<<[-]word
here-document
delimiter
No parameter and variable expansion, command substitution, arithmetic expansion, or pathname expansion is performed on word. If
any characters in word are quoted, the delimiter is the result of
quote removal on word, and the lines in the here-document are
not expanded. If word is unquoted, all lines of the here-document are subjected to parameter expansion, command
substitution, and arithmetic expansion, the character sequence
\ is ignored, and \ must be used to quote the characters \,
$, and `.
If the redirection operator is <<-, then all leading tab characters are stripped from input lines and the line containing
delimiter. This allows here-documents within shell scripts to be
indented in a natural fashion. [Emphasis added.]
Put backlash before the $ sign
$ VAR=XXX
$ cat << END
> dk
> \$VAR
> END
dk
$VAR
I have a command that is returning a string with a leading whitespace that gets eaten when I use command substitution
> echo " test"
test
> echo $(echo " test")
test
Why is this happening and what can I do about it?
You can double-quote command substitution to preserve that,
echo "$(echo " test")"
test
From the man bash on page,
Command Substitution
Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command, with any trailing newlines
deleted. Embedded newlines are not deleted, but they may be removed during word splitting.
If the substitution appears within double quotes, word splitting and pathname expansion are not performed on the results.
I am writing the bash script, which has to perform some of the commands on remote server via ssh. The script has two major parts:
Part 1:
Using local variables $A and $B
Part 2:
Execute commands on remote server as follows:
ssh -T user#servername << 'EOF'
...
Using local variables $A and $B
...
EOF
The problem is that local variables $A and $B are not available within scope of ssh commands on remote server. As far as I understand the variables $A and $B outside and within ssh scope are not the same.
So my question is how to pass local variables from bash script to ssh scope?
One more note, the part 2 is pretty big so I can't use "one liner" after ssh.
Thanks
The problem has nothing to do with ssh but is only related to here documents in bash or any other Posix shell.
Man page for bash says in the here document paragraph:
The format of here-documents is:
<<[-]word
here-document
delimiter
No parameter expansion, command substitution, arithmetic expansion, or pathname expansion is performed on word. If any characters in word are quoted, the delimiter is the result of quote removal on word, and the lines in the here-document are not expanded. If word is unquoted, all lines of the here-document are subjected to parameter expansion, command substitution, and arithmetic expansion. In the latter case, the character sequence \<newline> is ignored, and \ must be used to quote the characters \, $, and ` .
As you quote EOF, you explicitely ask the shell not to replace the $1 and $2 variables.
The most robust way is to not quote EOF and consistently quote all others special characters in the here document.
For example, if your here document contained something like:
ssh -T user#servername << 'EOF'
for f in /var/log/messages*
do echo "Filename: " $f
done
EOF
you could rewrite it with no quotes around EOF but with one inside $:
ssh -T user#servername << EOF
for f in /var/log/messages*
do echo "Filename: " \$f
done
EOF
That way all unquoted varibles would be interpolated.
Alternatively, if the server allows it, you can try to pass the 2 parameters as environment variables.
Say you want to use the names PARAM1 and PARAM2. The sshd_config file on the server should contain the line AcceptEnv PARAM1 PARAM2 because by default and for security reasons no environment variable is accepted.
You can then use:
export PARAM1=$1
export PARAM2=$2
ssh -T -o SendEnv=PARAM1 -o SenEnv=PARAM2 user#servername << 'EOF'
...
Using variables $PARAM1 and $PARAM2
...
EOF
There could be a way to directly tell ssh to use local variables but I'll quickly answer and mention that you could use a script that wraps the ssh with code that inserts the variables remotely where you will have access to them once you get control of the command prompt.
I am working on a bash script that needs to create a file in this location:
/etc/yum.repos.d/nginx.repo
with the following contents:
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1
So, I have tried to do it like this:
cat >/etc/yum.repos.d/nginx.repo <<EOL
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/\$releasever/\$basearch/
gpgcheck=0
enabled=1
EOL
When I check the contents of the file, I see the following:
As you can see, the dollar sign weren't getting escaped, so the variable was evaluated to null/empty string and the contents do not look correct. Because, when I try to install nginx, I get this error:
http://nginx.org/packages/centos///repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found
Any ideas?
In principle, it suffices to use a syntax
cat >file <<EOL
$my_var
EOL
That is, use the vars as they are, without escaping $.
So instead of
baseurl=http://nginx.org/packages/centos/\$releasever/\$basearch/
^ ^
say
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
From man bash:
Here Documents
This type of redirection instructs the shell to read input from the
current source until a line containing only delimiter (with no
trailing blanks) is seen. All of the lines read up to that point are
then used as the standard input for a command.
The format of here-documents is:
<<[-]word
here-document
delimiter
No parameter expansion, command substitution, arithmetic expansion,
or pathname expansion is performed on word. If any characters in word
are quoted, the delimiter is the result of quote removal on word, and
the lines in the here-document are not expanded. If word is
unquoted, all lines of the here-document are subjected to parameter
expansion, command substitution, and arithmetic expansion. In the
latter case, the character sequence \ is ignored, and \
must be used to quote the characters \, $, and `.
See an example:
$ cat a.sh
r="hello"
cat - <<EOL
hello
$r
EOL
echo "double quotes"
cat - <<"EOL"
hello
$r
EOL
echo "single quotes"
cat - <<'EOL'
hello
$r
EOL
And let's run it:
$ bash a.sh
hello
hello <-- it expands when unquoted
double quotes
hello
$r <-- it does not expand with "EOL"
single quotes
hello
$r <-- it does not expand with 'EOL'
There's an here-doc generic syntax to prevent the content to be expanded like when you put single quotes around variables :
cat<<'EOF'
:
cat<<'EOF' > /path/to/file
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1
EOF
From
man bash | less +/here-doc
If any characters in word are quoted, the delimiter is the result of quote removal on word, and the lines in the here-document are not expanded.
Just wrap that string into single quotes
baseurl='http://nginx.org/packages/centos/$releasever/$basearch/'
Then the dollar sign would be treated as a usual character.
[root#xxx ~]# cat test
baseurl='http://nginx.org/packages/centos/$releasever/$basearch/'
I want to print code into a file using cat <<EOF >>:
cat <<EOF >> brightup.sh
!/bin/bash
curr=`cat /sys/class/backlight/intel_backlight/actual_brightness`
if [ $curr -lt 4477 ]; then
curr=$((curr+406));
echo $curr > /sys/class/backlight/intel_backlight/brightness;
fi
EOF
but when I check the file output, I get this:
!/bin/bash
curr=1634
if [ -lt 4477 ]; then
curr=406;
echo > /sys/class/backlight/intel_backlight/brightness;
fi
I tried putting single quotes but the output also carries the single quotes with it. How can I avoid this issue?
You only need a minimal change; single-quote the here-document delimiter after <<.
cat <<'EOF' >> brightup.sh
or equivalently backslash-escape it:
cat <<\EOF >>brightup.sh
Without quoting, the here document will undergo variable substitution, backticks will be evaluated, etc, like you discovered.
If you need to expand some, but not all, values, you need to individually escape the ones you want to prevent.
cat <<EOF >>brightup.sh
#!/bin/sh
# Created on $(date # : <<-- this will be evaluated before cat;)
echo "\$HOME will not be evaluated because it is backslash-escaped"
EOF
will produce
#!/bin/sh
# Created on Fri Feb 16 11:00:18 UTC 2018
echo "$HOME will not be evaluated because it is backslash-escaped"
As suggested by #fedorqui, here is the relevant section from man bash:
Here Documents
This type of redirection instructs the shell to read input from the
current source until a line containing only delimiter (with no
trailing blanks) is seen. All of the lines read up to that point are
then used as the standard input for a command.
The format of here-documents is:
<<[-]word
here-document
delimiter
No parameter expansion, command substitution, arithmetic expansion,
or pathname expansion is performed on word. If any characters in word
are quoted, the delimiter is the result of quote removal on word, and
the lines in the here-document are not expanded. If word is
unquoted, all lines of the here-document are subjected to parameter
expansion, command substitution, and arithmetic expansion. In the
latter case, the character sequence \<newline> is ignored, and \
must be used to quote the characters \, $, and `.
This should work, I just tested it out and it worked as expected: no expansion, substitution, or what-have-you took place.
cat <<< '
#!/bin/bash
curr=`cat /sys/class/backlight/intel_backlight/actual_brightness`
if [ $curr -lt 4477 ]; then
curr=$((curr+406));
echo $curr > /sys/class/backlight/intel_backlight/brightness;
fi' > file # use overwrite mode so that you don't keep on appending the same script to that file over and over again, unless that's what you want.
Using the following also works.
cat <<< ' > file
... code ...'
Also, it's worth noting that when using heredocs, such as << EOF, substitution and variable expansion and the like takes place. So doing something like this:
cat << EOF > file
cd "$HOME"
echo "$PWD" # echo the current path
EOF
will always result in the expansion of the variables $HOME and $PWD. So if your home directory is /home/foobar and the current path is /home/foobar/bin, file will look like this:
cd "/home/foobar"
echo "/home/foobar/bin"
instead of the expected:
cd "$HOME"
echo "$PWD"
Or, using your EOF markers, you need to quote the initial marker so expansion won't be done:
#-----v---v------
cat <<'EOF' >> brightup.sh
#!/bin/bash
curr=`cat /sys/class/backlight/intel_backlight/actual_brightness`
if [ $curr -lt 4477 ]; then
curr=$((curr+406));
echo $curr > /sys/class/backlight/intel_backlight/brightness;
fi
EOF
IHTH
I know this is a two year old question, but this is a quick answer for those searching for a 'how to'.
If you don't want to have to put quotes around anything you can simply write a block of text to a file, and escape variables you want to export as text (for instance for use in a script) and not escape one's you want to export as the value of the variable.
#!/bin/bash
FILE_NAME="test.txt"
VAR_EXAMPLE="\"string\""
cat > ${FILE_NAME} << EOF
\${VAR_EXAMPLE}=${VAR_EXAMPLE} in ${FILE_NAME}
EOF
Will write '"${VAR_EXAMPLE}="string" in test.txt' into test.txt
This can also be used to output blocks of text to the console with the same rules by omitting the file name
#!/bin/bash
VAR_EXAMPLE="\"string\""
cat << EOF
\${VAR_EXAMPLE}=${VAR_EXAMPLE} to console
EOF
Will output '"${VAR_EXAMPLE}="string" to console'
cat with <<EOF>> will create or append the content to the existing file, won't overwrite. whereas cat with <<EOF> will create or overwrite the content.
cat test.txt
hello
cat <<EOF>> test.txt
> hi
> EOF
cat test.txt
hello
hi
cat <<EOF> test.txt
> haiiiii
> EOF
cat test.txt
haiiiii