How can I write a here document to a file in Bash script?
Read the Advanced Bash-Scripting Guide Chapter 19. Here Documents.
Here's an example which will write the contents to a file at /tmp/yourfilehere
cat << EOF > /tmp/yourfilehere
These contents will be written to the file.
This line is indented.
EOF
Note that the final 'EOF' (The LimitString) should not have any whitespace in front of the word, because it means that the LimitString will not be recognized.
In a shell script, you may want to use indentation to make the code readable, however this can have the undesirable effect of indenting the text within your here document. In this case, use <<- (followed by a dash) to disable leading tabs (Note that to test this you will need to replace the leading whitespace with a tab character, since I cannot print actual tab characters here.)
#!/usr/bin/env bash
if true ; then
cat <<- EOF > /tmp/yourfilehere
The leading tab is ignored.
EOF
fi
If you don't want to interpret variables in the text, then use single quotes:
cat << 'EOF' > /tmp/yourfilehere
The variable $FOO will not be interpreted.
EOF
To pipe the heredoc through a command pipeline:
cat <<'EOF' | sed 's/a/b/'
foo
bar
baz
EOF
Output:
foo
bbr
bbz
... or to write the the heredoc to a file using sudo:
cat <<'EOF' | sed 's/a/b/' | sudo tee /etc/config_file.conf
foo
bar
baz
EOF
Instead of using cat and I/O redirection it might be useful to use tee instead:
tee newfile <<EOF
line 1
line 2
line 3
EOF
It's more concise, plus unlike the redirect operator it can be combined with sudo if you need to write to files with root permissions.
Note:
the following condenses and organizes other answers in this thread, esp the excellent work of Stefan Lasiewski and Serge Stroobandt
Lasiewski and I recommend Ch 19 (Here Documents) in the Advanced Bash-Scripting Guide
The question (how to write a here document (aka heredoc) to a file in a bash script?) has (at least) 3 main independent dimensions or subquestions:
Do you want to overwrite an existing file, append to an existing file, or write to a new file?
Does your user or another user (e.g., root) own the file?
Do you want to write the contents of your heredoc literally, or to have bash interpret variable references inside your heredoc?
(There are other dimensions/subquestions which I don't consider important. Consider editing this answer to add them!) Here are some of the more important combinations of the dimensions of the question listed above, with various different delimiting identifiers--there's nothing sacred about EOF, just make sure that the string you use as your delimiting identifier does not occur inside your heredoc:
To overwrite an existing file (or write to a new file) that you own, substituting variable references inside the heredoc:
cat << EOF > /path/to/your/file
This line will write to the file.
${THIS} will also write to the file, with the variable contents substituted.
EOF
To append an existing file (or write to a new file) that you own, substituting variable references inside the heredoc:
cat << FOE >> /path/to/your/file
This line will write to the file.
${THIS} will also write to the file, with the variable contents substituted.
FOE
To overwrite an existing file (or write to a new file) that you own, with the literal contents of the heredoc:
cat << 'END_OF_FILE' > /path/to/your/file
This line will write to the file.
${THIS} will also write to the file, without the variable contents substituted.
END_OF_FILE
To append an existing file (or write to a new file) that you own, with the literal contents of the heredoc:
cat << 'eof' >> /path/to/your/file
This line will write to the file.
${THIS} will also write to the file, without the variable contents substituted.
eof
To overwrite an existing file (or write to a new file) owned by root, substituting variable references inside the heredoc:
cat << until_it_ends | sudo tee /path/to/your/file
This line will write to the file.
${THIS} will also write to the file, with the variable contents substituted.
until_it_ends
To append an existing file (or write to a new file) owned by user=foo, with the literal contents of the heredoc:
cat << 'Screw_you_Foo' | sudo -u foo tee -a /path/to/your/file
This line will write to the file.
${THIS} will also write to the file, without the variable contents substituted.
Screw_you_Foo
To build on #Livven's answer, here are some useful combinations.
variable substitution, leading tab retained, overwrite file, echo to stdout
tee /path/to/file <<EOF
${variable}
EOF
no variable substitution, leading tab retained, overwrite file, echo to stdout
tee /path/to/file <<'EOF'
${variable}
EOF
variable substitution, leading tab removed, overwrite file, echo to stdout
tee /path/to/file <<-EOF
${variable}
EOF
variable substitution, leading tab retained, append to file, echo to stdout
tee -a /path/to/file <<EOF
${variable}
EOF
variable substitution, leading tab retained, overwrite file, no echo to stdout
tee /path/to/file <<EOF >/dev/null
${variable}
EOF
the above can be combined with sudo as well
sudo -u USER tee /path/to/file <<EOF
${variable}
EOF
When root permissions are required
When root permissions are required for the destination file, use |sudo tee instead of >:
cat << 'EOF' |sudo tee /tmp/yourprotectedfilehere
The variable $FOO will *not* be interpreted.
EOF
cat << "EOF" |sudo tee /tmp/yourprotectedfilehere
The variable $FOO *will* be interpreted.
EOF
For future people who may have this issue the following format worked:
(cat <<- _EOF_
LogFile /var/log/clamd.log
LogTime yes
DatabaseDirectory /var/lib/clamav
LocalSocket /tmp/clamd.socket
TCPAddr 127.0.0.1
SelfCheck 1020
ScanPDF yes
_EOF_
) > /etc/clamd.conf
For those looking for a pure bash solution (or a need for speed), here's a simple solution without cat:
# here-doc tab indented
{ read -r -d '' || printf >file '%s' "$REPLY"; } <<-EOF
foo bar
EOF
or for an easy "mycat" function (and avoid leaving REPLY in environment):
mycat() {
local REPLY
read -r -d '' || printf '%s' "$REPLY"
}
mycat >file <<-EOF
foo bar
EOF
Quick speed comparison of "mycat" vs OS cat (1000 loops >/dev/null on my OSX laptop):
mycat:
real 0m1.507s
user 0m0.108s
sys 0m0.488s
OS cat:
real 0m4.082s
user 0m0.716s
sys 0m1.808s
NOTE: mycat doesn't handle file arguments, it just handles the problem "write a heredoc to a file"
As instance you could use it:
First(making ssh connection):
while read pass port user ip files directs; do
sshpass -p$pass scp -o 'StrictHostKeyChecking no' -P $port $files $user#$ip:$directs
done <<____HERE
PASS PORT USER IP FILES DIRECTS
. . . . . .
. . . . . .
. . . . . .
PASS PORT USER IP FILES DIRECTS
____HERE
Second(executing commands):
while read pass port user ip; do
sshpass -p$pass ssh -p $port $user#$ip <<ENDSSH1
COMMAND 1
.
.
.
COMMAND n
ENDSSH1
done <<____HERE
PASS PORT USER IP
. . . .
. . . .
. . . .
PASS PORT USER IP
____HERE
Third(executing commands):
Script=$'
#Your commands
'
while read pass port user ip; do
sshpass -p$pass ssh -o 'StrictHostKeyChecking no' -p $port $user#$ip "$Script"
done <<___HERE
PASS PORT USER IP
. . . .
. . . .
. . . .
PASS PORT USER IP
___HERE
Forth(using variables):
while read pass port user ip fileoutput; do
sshpass -p$pass ssh -o 'StrictHostKeyChecking no' -p $port $user#$ip fileinput=$fileinput 'bash -s'<<ENDSSH1
#Your command > $fileinput
#Your command > $fileinput
ENDSSH1
done <<____HERE
PASS PORT USER IP FILE-OUTPUT
. . . . .
. . . . .
. . . . .
PASS PORT USER IP FILE-OUTPUT
____HERE
If you want to keep the heredoc indented for readability:
$ perl -pe 's/^\s*//' << EOF
line 1
line 2
EOF
The built-in method for supporting indented heredoc in Bash only supports leading tabs, not spaces.
Perl can be replaced with awk to save a few characters, but the Perl one is probably easier to remember if you know basic regular expressions.
In addition, if you're writing to a file, it can be a good idea to check whether or not your write succeeded for failed. For example:
if ! echo "contents" > ./file ; then
echo "ERROR: failed to write to file" >& 2
exit 1
fi
To do the same with heredoc, there are two possible approaches.
1)
if ! cat > ./file << EOF
contents
EOF
then
echo "ERROR: failed to write to file" >& 2
exit 1
fi
if ! cat > ./file ; then
echo "ERROR: failed to write to file" >& 2
exit 1
fi << EOF
contents
EOF
You can test the error case in the above code by replacing the destination file ./file with /file (assuming you're not running as root).
I like the following method of basic redirection for its concision, readability and presentation in an indented script:
<<-End_of_file >file
→ foo bar
End_of_file
Where → is a tab character.
This is standard Bourne shell redirection without forking any cat or tee process.
But it is not working with current bash even when called through /bin/sh.
It is still working with /bin/zsh since more than 20 years.
This question already has answers here:
echo "#!" fails -- "event not found"
(5 answers)
Closed 7 months ago.
echo "#!/bin/bash\nls -l /home/" > /home/myscript.sh
bash: !/bin/bash\nls: event not found
My script should be:
#!/bin/bash
ls -l /home/
Why does it ignore the echo "" string and think that there is some sort of event? Why does it not recognize #!/bin/bash as a special word?
the same thing happens when I
echo "#!/bin/bash" > /home/myscript.sh
so it's not the new line!
echo -e "#\!/bin/bash" > /home/myscript.sh
writes the file content as:
#\!/bin/bash
Why is this simple action going miserably wrong?
From the bash manpage:
Enclosing characters in double quotes preserves the literal value of
all characters within the quotes, with the exception of $, `, \,
and, when history expansion is
enabled, !.
So either use single quotes, or disable history expansion with set +o history.
But don't use echo. Instead, do :
printf '%s\n' '#!/bin/bash' 'ls -l /home/' > /home/myscript
or
cat > /home/myscript << 'EOF'
#!/bin/bash
ls -l /home/
EOF
echo -e '#!/bin/bash\nls -l /home/' > /home/myscript.sh
a combination of -e and using single quote fixed it.
I’ve noticed that we can use curly braces to make some of the commands much shorter as it is evaluated into list of arguments.
Input:
echo a{,b,c}
Output:
a ab ac
How do I force the same behaviour when the arguments are passed from the file?
Input:
cat file.txt | xargs echo
Output:
a{,b,c}
Expected output - same as in the previous example.
That {} expansion is a bash / zsh feature, as such then you need to explicitly run it thru any of these shells, in your case would be (using -I<STRING> to let xargs replace it in the string before running it):
cat file.txt |xargs -I# bash -c 'echo #'
xargs calls the echo as found in the $PATH, not the shell's builtin echo.
check the list of bash expansions: brace expansion happens first, so it won't get a chance to expand in that pipeline.
You'll have to do something like
while read -r line; do eval echo "$line"; done < file.txt
which exposes you to all kinds of nasty attacks if someone puts something malicious in that file.
Other than asking why would you want to do this... I offer the following:
add the string to a file:
echo 'a{,b,c}' > /tmp/foo
put the string in a variable:
export thing=`cat /tmp/foo`
eval the string:
eval $thing
If you had a bunch of these in a file then run the file through a loop and eval the loop value:
echo 'a{,b,c}' >> /tmp/foo
echo 'a{,b,c}' >> /tmp/foo
echo 'a{,b,c}' >> /tmp/foo
for i in `cat /tmp/foo`; do eval echo $i; done
Right now I have this file so far...
#!/usr/bin/env bash
DIRECTORY=$1
ssh root#example.com "$( cat <<'EOT'
cd /web/$DIRECTORY || exit
pwd
unset GIT_DIR
git log --oneline -n 10 --decorate
git branch
EOT
)";
Why does the pwd just print out "/web/"? It doesn't actually seem to be using my variable. Then all the git commands throw errors about being in the web directory.
If I have it echo $DIRECTORY out before ssh-ing, it echos out my variable, but refuses to pass it through to the ssh command?
This happens because you quote the here doc delimiter. Here's POSIX:
If any character in word is quoted, the delimiter is formed by performing quote removal on word, and the here-document lines will not be expanded. Otherwise, the delimiter is the word itself.
And here's an example:
#!/bin/bash
var=42
cat << 'end'
With quotes: $var and \$var
end
cat << end
Without quotes: $var and \$var
end
When executed:
With quotes: $var and \$var
Without quotes: 42 and $var
I have a file configure.sh with following (it creates a test.sh file with configuration, so that i can use that test.sh finally as main configuration task). But it does not work
cat > /var/tmp/test.sh << EOF
regex='value=(.*)'
for i in $(cat /var/tmp/test.ini);
do
if [[ $i =~ $regex ]];
then
echo ${BASH_REMATCH[1]}
#or
curl -v ${BASH_REMATCH[1]}
fi
done
EOF
When the configure.sh is executed it makes the test.sh file completely wrong such as
reged='value(.*)'
for i in
original line1
original line1
original line1
original line1
do
if [[ =~ ]];
then
fi
done
EOF
The EOF block is not writing exactly how i set above. How do you write such string inside EOF?
Change << EOF to << 'EOF' and leave the EOF at the end unchanged. Quoting the EOF in this way will stop Bash from executing expansions on the here-document.
From man bash, section Here Documents:
No parameter expansion, command substitution, arithmetic
expansion, or pathname expansion is performed on word. If any characters in word are
quoted, the delimiter is the result of quote
removal on word, and the lines in the here-document are not expanded.
So write << \EOF instead of << EOF.