I need to output a variable value to a file in a unix script. My problem it that the variable contains multiple lines. I need those to be output as '\n' literals in the file (a java options file), but I'm using echo and they always get processed into real new lines.
echo "-dmyproperty=$MULTILINE_VAR" >> jvm.options
I've tried echo options like -e o -E but they don't seem to do anything. Can anyone help?
You can use bash parameter substitution with an ANSI-C quoted newline
$ var="line1
line2
line3"
$ echo "${var//$'\n'/\\n}"
line1\nline2\nline3
Related
I'm trying to figure out how to use a variable containing an ANSI-C quoting string as an argument for a subsequent bash command.
The string in the variable itself is a list of files (can virtually be a list of anything).
For example, I have a file containing a list of other files, for example test.lst containing :
>$ cat test.lst
a.txt
b.txt
c.txt
I need to pass the file content as a single string so I'm doing :
test_str=$(cat test.lst)
then converts to ANSI-C quoting string:
test_str=${test_str#Q}
So at the end I have :
>$ test_str=$(cat test.lst)
>$ test_str=${test_str#Q}
>$ echo $test_str
$'a.txt\nb.txt\nc.txt'
which is what I'm looking for.
Then problem arises when I try to reuse this variable as a string list in another bash command.
For example direct use into a for loop :
>$ for str in $test_str; do echo $str; done
$'a.txt\nb.txt\nc.txt'
What I expect at this step is that it prints the same thing as the content of the original test.lst
I also tried expanding it back but it leaves leading $' and trailing '
>$ str=${test_str#E}
>$ echo $str
$'a.txt b.txt c.txt'
I also tried printf and some other stuffs to no avail. What is the correct way to use such ANSI-C quoting variable into a bash command ?
How about:
eval echo "${test_str}"
I believe that an ANSI-C quoted string is meant to be evaluated by bash command line parser.
In the first place, why do you need quoting? Just keep the data untouched stored as elements of an array:
mapfile -t filelist < test.lst
# iterate through the list
for file in "${filelist[#]}"; do printf '%s\n' "$file"; done
I have a bash script which asks for two arguments with a space between them. Now I would like to automate filling out the prompt in the command line with reading from a text file. The text file contains a list with the argument combinations.
So something like this in the command line I think;
for line in 'cat text.file' ; do script.sh ; done
Can this be done? What am I missing/doing wrong?
Thanks for the help.
A while loop is probably what you need. Put the space separated strings in the file text.file :
cat text.file
bingo yankee
bravo delta
Then write the script in question like below.
#!/bin/bash
while read -r arg1 arg2
do
/path/to/your/script.sh "$arg1" "$arg2"
done<text.file
Don't use for to read files line by line
Try something like this:
#!/bin/bash
ARGS=
while IFS= read -r line; do
ARGS="${ARGS} ${line}"
done < ./text.file
script.sh "$ARGS"
This would add each line to a variable which then is used as the arguments of your script.
'cat text.file' is a string literal, $(cat text.file) would expand to output of command however cat is useless because bash can read file using redirection, also with quotes it will be treated as a single argument and without it will split at space tab and newlines.
Bash syntax to read a file line by line, but will be slow for big files
while IFS= read -r line; do ... "$line"; done < text.file
unsetting IFS for read command preserves leading spaces
-r option preserves \
another way, to read whole file is content=$(<file), note the < inside the command substitution. so a creative way to read a file to array, each element a non-empty line:
read_to_array () {
local oldsetf=${-//[^f]} oldifs=$IFS
set -f
IFS=$'\n' array_content=($(<"$1")) IFS=$oldifs
[[ $oldsetf ]]||set +f
}
read_to_array "file"
for element in "${array_content[#]}"; do ...; done
oldsetf used to store current set -f or set +f setting
oldifs used to store current IFS
IFS=$'\n' to split on newlines (multiple newlines will be treated as one)
set -f avoid glob expansion for example in case line contains single *
note () around $() to store the result of splitting to an array
If I were to create a solution determined by the literal of what you ask for (using a for loop and parsing lines from a file) I would use iterations determined by the number of lines in the file (if it isn't too large).
Assuming each line has two strings separated by a single space (to be used as positional parameters in your script:
file="$1"
f_count="$(wc -l < $file)"
for line in $(seq 1 $f_count)
do
script.sh $(head -n $line $file | tail -n1) && wait
done
You may have a much better time using sjsam's solution however.
I'm trying to configure a file with a bash script. And the variables in the bash script are not written in file as it is written in script.
Ex:
#!/bin/bash
printf "%s" "template("$DATE\t$HOST\t$PRIORITY\t$MSG\n")" >> /file.txt
exit 0
This results to template('tttn') instead of template("$DATE\t$HOST\t$PRIORITY\t$MSG\n in file.
How do I write in the script so that the result is template("$DATE\t$HOST\t$PRIORITY\t$MSG\n in the configured file?
Is it possible to write variable as it looks in script to file?
Enclose the strings you want to write within single quotes to avoid variable replacement.
> FOO=bar
> echo "$FOO"
bar
> echo '$FOO'
$FOO
>
Using printf in any shell script is uncommon, just use echo with the -e option.
It allows you to use ANSI C metacharacters, like \t or \n. The \n at the end however isn't necessary, as echo will add one itself.
echo -e "template(${DATE}\t${HOST}\t${PRIORITY}\t${MSG})" >> file.txt
The problem with what you've written is, that ANSI C metacharacters, like \t can only be used in the first parameter to printf.
So it would have to be something like:
printf 'template(%s\t%s\t%s\t%s)\n' ${DATE} ${HOST} ${PRIORITY} ${MSG} >> file.txt
But I hope we both agree, that this is very hard on the eyes.
There are several escaping issues and the power of printf has not been used, try
printf 'template(%s\t%s\t%s\t%s)\n' "${DATE}" "${HOST}" "${PRIORITY}" "${MSG}" >> file.txt
Reasons for this separate answer:
The accepted answer does not fit the title of the question (see comment).
The post with the right answer
contains wrong claims about echo vs printf as of this post and
is not robust against whitespace in the values.
The edit queue is full at the moment.
I have a file with several lines.
When using cat/more/less [file] in the shell,
the content is shown line by line
When doing the following commands:
temp=`cat [file]`
echo $temp
the content is shown in one line.
Is there a way to preserve the line endings when setting to environment variable and then echo it?
Thanks
Yes:
temp=`cat [file]`
echo "$temp"
The magic is in the quotes around $temp; without them, echo gets these arguments:
echo line1\nline2\nlin3
The shell parsing algorithm will split the command line at white space, so echo sees three arguments. If you quote the variable, echo will see a single argument and the shell parsing won't touch the whitespace between the quotes.
Here's a super-precise answer: Process substitution into a variable will not preserve:
Any ASCII NULs
Any number of trailing newlines
Only the latter can be worked around:
temp=$(realprocess; echo x) ## Add x to the end to preserve trailing newlines
temp=${temp%x} ## Remove the x again (keeping originally trailing newlines)
When you want to display the true contents of a variable, use printf. echo adds an additional newline, and is not reliable (when the input starts with the string -n for example).
In any case, always quote your variables, or the shell will split them on whitespace to any number of arguments!
printf %s "$temp"
Generally, keeping the complete contents of a file in a shell variable is not what you want. There is files for that.
If I do the following the newlines are preserved:
echo a >> test
echo b >> test
temp=`cat test`
echo $temp
I would like to copy the contents of a variable (here called var) into a file.
The name of the file is stored in another variable destfile.
I'm having problems doing this. Here's what I've tried:
cp $var $destfile
I've also tried the same thing with the dd command... Obviously the shell thought that $var was referring to a directory and so told me that the directory could not be found.
How do I get around this?
Use the echo command:
var="text to append";
destdir=/some/directory/path/filename
if [ -f "$destdir" ]
then
echo "$var" > "$destdir"
fi
The if tests that $destdir represents a file.
The > appends the text after truncating the file. If you only want to append the text in $var to the file existing contents, then use >> instead:
echo "$var" >> "$destdir"
The cp command is used for copying files (to files), not for writing text to a file.
echo has the problem that if var contains something like -e, it will be interpreted as a flag. Another option is printf, but printf "$var" > "$destdir" will expand any escaped characters in the variable, so if the variable contains backslashes the file contents won't match. However, because printf only interprets backslashes as escapes in the format string, you can use the %s format specifier to store the exact variable contents to the destination file:
printf "%s" "$var" > "$destdir"
None of the answers above work if your variable:
starts with -e
starts with -n
starts with -E
contains a \ followed by an n
should not have an extra newline appended after it
and so they cannot be relied upon for arbitrary string contents.
In bash, you can use "here strings" as:
cat <<< "$var" > "$destdir"
As noted in the comment by Ash below, #Trebawa's answer (formulated in the same room as mine!) using printf is a better approach than cat.
All of the above work, but also have to work around a problem (escapes and special characters) that doesn't need to occur in the first place: Special characters when the variable is expanded by the shell. Just don't do that (variable expansion) in the first place. Use the variable directly, without expansion.
Also, if your variable contains a secret and you want to copy that secret into a file, you might want to not have expansion in the command line as tracing/command echo of the shell commands might reveal the secret. Means, all answers which use $var in the command line may have a potential security risk by exposing the variable contents to tracing and logging of the shell.
For variables that are already exported, use this:
printenv var >file
That means, in case of the OP question:
printenv var >"$destfile"
Note: variable names are case sensitive.
Warning: It is not a good idea to export a variable just for the sake of printing it with printenv. If you have a non-exported script variable that contains a secret, exporting it will expose it to all future child processes (unless unexported, for example using export -n).
If I understood you right, you want to copy $var in a file (if it's a string).
echo $var > $destdir
When you say "copy the contents of a variable", does that variable contain a file name, or does it contain a name of a file?
I'm assuming by your question that $var contains the contents you want to copy into the file:
$ echo "$var" > "$destdir"
This will echo the value of $var into a file called $destdir. Note the quotes. Very important to have "$var" enclosed in quotes. Also for "$destdir" if there's a space in the name. To append it:
$ echo "$var" >> "$destdir"
you may need to edit a conf file in a build process:
echo "db-url-host=$POSTGRESQL_HOST" >> my-service.conf
You can test this solution with running before export POSTGRESQL_HOST="localhost"