I'm trying to configure a file with a bash script. And the variables in the bash script are not written in file as it is written in script.
Ex:
#!/bin/bash
printf "%s" "template("$DATE\t$HOST\t$PRIORITY\t$MSG\n")" >> /file.txt
exit 0
This results to template('tttn') instead of template("$DATE\t$HOST\t$PRIORITY\t$MSG\n in file.
How do I write in the script so that the result is template("$DATE\t$HOST\t$PRIORITY\t$MSG\n in the configured file?
Is it possible to write variable as it looks in script to file?
Enclose the strings you want to write within single quotes to avoid variable replacement.
> FOO=bar
> echo "$FOO"
bar
> echo '$FOO'
$FOO
>
Using printf in any shell script is uncommon, just use echo with the -e option.
It allows you to use ANSI C metacharacters, like \t or \n. The \n at the end however isn't necessary, as echo will add one itself.
echo -e "template(${DATE}\t${HOST}\t${PRIORITY}\t${MSG})" >> file.txt
The problem with what you've written is, that ANSI C metacharacters, like \t can only be used in the first parameter to printf.
So it would have to be something like:
printf 'template(%s\t%s\t%s\t%s)\n' ${DATE} ${HOST} ${PRIORITY} ${MSG} >> file.txt
But I hope we both agree, that this is very hard on the eyes.
There are several escaping issues and the power of printf has not been used, try
printf 'template(%s\t%s\t%s\t%s)\n' "${DATE}" "${HOST}" "${PRIORITY}" "${MSG}" >> file.txt
Reasons for this separate answer:
The accepted answer does not fit the title of the question (see comment).
The post with the right answer
contains wrong claims about echo vs printf as of this post and
is not robust against whitespace in the values.
The edit queue is full at the moment.
Related
How do I print a newline? This merely prints \n:
$ echo -e "Hello,\nWorld!"
Hello,\nWorld!
Use printf instead:
printf "hello\nworld\n"
printf behaves more consistently across different environments than echo.
Make sure you are in Bash.
$ echo $0
bash
All these four ways work for me:
echo -e "Hello\nworld"
echo -e 'Hello\nworld'
echo Hello$'\n'world
echo Hello ; echo world
echo $'hello\nworld'
prints
hello
world
$'' strings use ANSI C Quoting:
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard.
You could always do echo "".
For example,
echo "Hello,"
echo ""
echo "World!"
On the off chance that someone finds themselves beating their head against the wall trying to figure out why a coworker's script won't print newlines, look out for this:
#!/bin/bash
function GET_RECORDS()
{
echo -e "starting\n the process";
}
echo $(GET_RECORDS);
As in the above, the actual running of the method may itself be wrapped in an echo which supersedes any echos that may be in the method itself. Obviously, I watered this down for brevity. It was not so easy to spot!
You can then inform your comrades that a better way to execute functions would be like so:
#!/bin/bash
function GET_RECORDS()
{
echo -e "starting\n the process";
}
GET_RECORDS;
Simply type
echo
to get a new line
POSIX 7 on echo
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html
-e is not defined and backslashes are implementation defined:
If the first operand is -n, or if any of the operands contain a <backslash> character, the results are implementation-defined.
unless you have an optional XSI extension.
So I recommend that you should use printf instead, which is well specified:
format operand shall be used as the format string described in XBD File Format Notation [...]
the File Format Notation:
\n <newline> Move the printing position to the start of the next line.
Also keep in mind that Ubuntu 15.10 and most distros implement echo both as:
a Bash built-in: help echo
a standalone executable: which echo
which can lead to some confusion.
str='hello\nworld'
$ echo | sed "i$str"
hello
world
You can also do:
echo "hello
world"
This works both inside a script and from the command line.
On the command line, press Shift+Enter to do the line break inside the string.
This works for me on my macOS and my Ubuntu 18.04 (Bionic Beaver) system.
For only the question asked (not special characters etc) changing only double quotes to single quotes.
echo -e 'Hello,\nWorld!'
Results in:
Hello,
World!
There is a new parameter expansion added in Bash 4.4 that interprets escape sequences:
${parameter#operator} - E operator
The expansion is a string that is the value of parameter with
backslash escape sequences expanded as with the $'…' quoting
mechanism.
$ foo='hello\nworld'
$ echo "${foo#E}"
hello
world
I just use echo without any arguments:
echo "Hello"
echo
echo "World"
To print a new line with echo, use:
echo
or
echo -e '\n'
This could better be done as
x="\n"
echo -ne $x
-e option will interpret backslahes for the escape sequence
-n option will remove the trailing newline in the output
PS: the command echo has an effect of always including a trailing newline in the output so -n is required to turn that thing off (and make it less confusing)
My script:
echo "WARNINGS: $warningsFound WARNINGS FOUND:\n$warningStrings
Output:
WARNING : 2 WARNINGS FOUND:\nWarning, found the following local orphaned signature file:
On my Bash script I was getting mad as you until I've just tried:
echo "WARNING : $warningsFound WARNINGS FOUND:
$warningStrings"
Just hit Enter where you want to insert that jump. The output now is:
WARNING : 2 WARNINGS FOUND:
Warning, found the following local orphaned signature file:
If you're writing scripts and will be echoing newlines as part of other messages several times, a nice cross-platform solution is to put a literal newline in a variable like so:
newline='
'
echo "first line${newline}second line"
echo "Error: example error message n${newline}${usage}" >&2 #requires usage to be defined
If the previous answers don't work, and there is a need to get a return value from their function:
function foo()
{
local v="Dimi";
local s="";
.....
s+="Some message here $v $1\n"
.....
echo $s
}
r=$(foo "my message");
echo -e $r;
Only this trick worked on a Linux system I was working on with this Bash version:
GNU bash, version 2.2.25(1)-release (x86_64-redhat-linux-gnu)
You could also use echo with braces,
$ (echo hello; echo world)
hello
world
This got me there....
outstuff=RESOURCE_GROUP=[$RESOURCE_GROUP]\\nAKS_CLUSTER_NAME=[$AKS_CLUSTER_NAME]\\nREGION_NAME=[$REGION_NAME]\\nVERSION=[$VERSION]\\nSUBNET-ID=[$SUBNET_ID]
printf $outstuff
Yields:
RESOURCE_GROUP=[akswork-rg]
AKS_CLUSTER_NAME=[aksworkshop-804]
REGION_NAME=[eastus]
VERSION=[1.16.7]
SUBNET-ID=[/subscriptions/{subidhere}/resourceGroups/makeakswork-rg/providers/Microsoft.Network/virtualNetworks/aks-vnet/subnets/aks-subnet]
Sometimes you can pass multiple strings separated by a space and it will be interpreted as \n.
For example when using a shell script for multi-line notifcations:
#!/bin/bash
notify-send 'notification success' 'another line' 'time now '`date +"%s"`
With jq:
$ jq -nr '"Hello,\nWorld"'
Hello,
World
Additional solution:
In cases, you have to echo a multiline of the long contents (such as code/ configurations)
For example:
A Bash script to generate codes/ configurations
echo -e,
printf might have some limitation
You can use some special char as a placeholder as a line break (such as ~) and replace it after the file was created using tr:
echo ${content} | tr '~' '\n' > $targetFile
It needs to invoke another program (tr) which should be fine, IMO.
I have a file stv.txt containing some names
For example stv.txt is as follows:
hello
world
I want to generate another file by using these names and adding some extra text to them.I have written a script as follows
for i in `cat stvv.txt`;
do printf 'if(!strcmp("$i",optarg))' > my_file;
done
output
if(!strcmp("$i",optarg))
desired output
if(!strcmp("hello",optarg))
if(!strcmp("world",optarg))
how can I get the correct result?
This is a working solution.
1 All symbols inside single quotes is considered a string. 2 When using printf, do not surround the variable with quotes. (in this example)
The code below should fix it,
for i in `cat stvv.txt`;
printf 'if(!strcmp('$i',optarg))' > my_file;
done
basically, break the printf statement into three parts.
1: the string 'if(!strcmp('
2: $i (no quotes)
3: the string ',optarg))'
hope that helps!
To insert a string into a printf format, use %s in the format string:
$ for line in $(cat stvv.txt); do printf 'if(!strcmp("%s",optarg))\n' "$line"; done
if(!strcmp("hello",optarg))
if(!strcmp("world",optarg))
The code $(cat stvv.txt) will perform word splitting and pathname expansion on the contents of stvv.txt. You probably don't want that. It is generally safer to use a while read ... done <stvv.txt loop such as this one:
$ while read -r line; do printf 'if(!strcmp("%s",optarg))\n' "$line"; done <stvv.txt
if(!strcmp("hello",optarg))
if(!strcmp("world",optarg))
Aside on cat
If you are using bash, then $(cat stvv.txt) could be replaced with the more efficient $(<stvv.txt). This question, however, is tagged shell not bash. The cat form is POSIX and therefore portable to all POSIX shells while the bash form is not.
I have a script that needs to dynamically assemble another script for later execution, however I'm experiencing problems with characters being substituted, specifically any that are escaped such as \\, \t, \n and so-on. While I can work around this issue with variables, it's extremely annoying, especially as the code is provided in segments wrapped as here documents in quoted form, i.e - such that they should not be processed at all.
Even more annoying, on some platforms this substitution seems to extend to other characters as well, such as \1, which has been causing havoc with my testing of regular expressions.
Here's a simple example:
#!/bin/sh
script=$(cat << 'SCRIPT'
#!/bin/sh
printf '\t%s' "$1"
SCRIPT
)
DIR=$(dirname "$PWD")
echo "$script" > "$DIR/test_script.sh"
I would expect this to produce a simple script with the line printf '\t%s' "$1", however instead the line is produced as printf ' %s' "$1", when no substitution is expected.
Can anyone explain why this is happening, and ideally, how to prevent this from happening? Like I say, I can work around this with variables for substituted characters, but it's destroying the readability of my script (and is hell to debug).
It would appear your version of echo processes some escape characters when you dump the script to a file. echo is not well-standardized, since any standard would have conflicted with some previous implementation. Use printf instead.
#!/bin/sh
script=$(cat << 'SCRIPT'
#!/bin/sh
printf '\t%s' "$1"
SCRIPT
)
DIR=$(dirname "$PWD")
printf "%s\n" "$script" > "$DIR/test_script.sh"
I would like to copy the contents of a variable (here called var) into a file.
The name of the file is stored in another variable destfile.
I'm having problems doing this. Here's what I've tried:
cp $var $destfile
I've also tried the same thing with the dd command... Obviously the shell thought that $var was referring to a directory and so told me that the directory could not be found.
How do I get around this?
Use the echo command:
var="text to append";
destdir=/some/directory/path/filename
if [ -f "$destdir" ]
then
echo "$var" > "$destdir"
fi
The if tests that $destdir represents a file.
The > appends the text after truncating the file. If you only want to append the text in $var to the file existing contents, then use >> instead:
echo "$var" >> "$destdir"
The cp command is used for copying files (to files), not for writing text to a file.
echo has the problem that if var contains something like -e, it will be interpreted as a flag. Another option is printf, but printf "$var" > "$destdir" will expand any escaped characters in the variable, so if the variable contains backslashes the file contents won't match. However, because printf only interprets backslashes as escapes in the format string, you can use the %s format specifier to store the exact variable contents to the destination file:
printf "%s" "$var" > "$destdir"
None of the answers above work if your variable:
starts with -e
starts with -n
starts with -E
contains a \ followed by an n
should not have an extra newline appended after it
and so they cannot be relied upon for arbitrary string contents.
In bash, you can use "here strings" as:
cat <<< "$var" > "$destdir"
As noted in the comment by Ash below, #Trebawa's answer (formulated in the same room as mine!) using printf is a better approach than cat.
All of the above work, but also have to work around a problem (escapes and special characters) that doesn't need to occur in the first place: Special characters when the variable is expanded by the shell. Just don't do that (variable expansion) in the first place. Use the variable directly, without expansion.
Also, if your variable contains a secret and you want to copy that secret into a file, you might want to not have expansion in the command line as tracing/command echo of the shell commands might reveal the secret. Means, all answers which use $var in the command line may have a potential security risk by exposing the variable contents to tracing and logging of the shell.
For variables that are already exported, use this:
printenv var >file
That means, in case of the OP question:
printenv var >"$destfile"
Note: variable names are case sensitive.
Warning: It is not a good idea to export a variable just for the sake of printing it with printenv. If you have a non-exported script variable that contains a secret, exporting it will expose it to all future child processes (unless unexported, for example using export -n).
If I understood you right, you want to copy $var in a file (if it's a string).
echo $var > $destdir
When you say "copy the contents of a variable", does that variable contain a file name, or does it contain a name of a file?
I'm assuming by your question that $var contains the contents you want to copy into the file:
$ echo "$var" > "$destdir"
This will echo the value of $var into a file called $destdir. Note the quotes. Very important to have "$var" enclosed in quotes. Also for "$destdir" if there's a space in the name. To append it:
$ echo "$var" >> "$destdir"
you may need to edit a conf file in a build process:
echo "db-url-host=$POSTGRESQL_HOST" >> my-service.conf
You can test this solution with running before export POSTGRESQL_HOST="localhost"
I'm writing a shell script that should be somewhat secure, i.e., does not pass secure data through parameters of commands and preferably does not use temporary files. How can I pass a variable to the standard input of a command?
Or, if it's not possible, how can I correctly use temporary files for such a task?
Passing a value to standard input in Bash is as simple as:
your-command <<< "$your_variable"
Always make sure you put quotes around variable expressions!
Be cautious, that this will probably work only in bash and will not work in sh.
Simple, but error-prone: using echo
Something as simple as this will do the trick:
echo "$blah" | my_cmd
Do note that this may not work correctly if $blah contains -n, -e, -E etc; or if it contains backslashes (bash's copy of echo preserves literal backslashes in absence of -e by default, but will treat them as escape sequences and replace them with corresponding characters even without -e if optional XSI extensions are enabled).
More sophisticated approach: using printf
printf '%s\n' "$blah" | my_cmd
This does not have the disadvantages listed above: all possible C strings (strings not containing NULs) are printed unchanged.
(cat <<END
$passwd
END
) | command
The cat is not really needed, but it helps to structure the code better and allows you to use more commands in parentheses as input to your command.
Note that the 'echo "$var" | command operations mean that standard input is limited to the line(s) echoed. If you also want the terminal to be connected, then you'll need to be fancier:
{ echo "$var"; cat - ; } | command
( echo "$var"; cat - ) | command
This means that the first line(s) will be the contents of $var but the rest will come from cat reading its standard input. If the command does not do anything too fancy (try to turn on command line editing, or run like vim does) then it will be fine. Otherwise, you need to get really fancy - I think expect or one of its derivatives is likely to be appropriate.
The command line notations are practically identical - but the second semi-colon is necessary with the braces whereas it is not with parentheses.
This robust and portable way has already appeared in comments. It should be a standalone answer.
printf '%s' "$var" | my_cmd
or
printf '%s\n' "$var" | my_cmd
Notes:
It's better than echo, reasons are here: Why is printf better than echo?
printf "$var" is wrong. The first argument is format where various sequences like %s or \n are interpreted. To pass the variable right, it must not be interpreted as format.
Usually variables don't contain trailing newlines. The former command (with %s) passes the variable as it is. However tools that work with text may ignore or complain about an incomplete line (see Why should text files end with a newline?). So you may want the latter command (with %s\n) which appends a newline character to the content of the variable. Non-obvious facts:
Here string in Bash (<<<"$var" my_cmd) does append a newline.
Any method that appends a newline results in non-empty stdin of my_cmd, even if the variable is empty or undefined.
I liked Martin's answer, but it has some problems depending on what is in the variable. This
your-command <<< """$your_variable"""
is better if you variable contains " or !.
As per Martin's answer, there is a Bash feature called Here Strings (which itself is a variant of the more widely supported Here Documents feature):
3.6.7 Here Strings
A variant of here documents, the format is:
<<< word
The word is expanded and supplied to the command on its standard
input.
Note that Here Strings would appear to be Bash-only, so, for improved portability, you'd probably be better off with the original Here Documents feature, as per PoltoS's answer:
( cat <<EOF
$variable
EOF
) | cmd
Or, a simpler variant of the above:
(cmd <<EOF
$variable
EOF
)
You can omit ( and ), unless you want to have this redirected further into other commands.
Try this:
echo "$variable" | command
If you came here from a duplicate, you are probably a beginner who tried to do something like
"$variable" >file
or
"$variable" | wc -l
where you obviously meant something like
echo "$variable" >file
echo "$variable" | wc -l
(Real beginners also forget the quotes; usually use quotes unless you have a specific reason to omit them, at least until you understand quoting.)