I have a file stv.txt containing some names
For example stv.txt is as follows:
hello
world
I want to generate another file by using these names and adding some extra text to them.I have written a script as follows
for i in `cat stvv.txt`;
do printf 'if(!strcmp("$i",optarg))' > my_file;
done
output
if(!strcmp("$i",optarg))
desired output
if(!strcmp("hello",optarg))
if(!strcmp("world",optarg))
how can I get the correct result?
This is a working solution.
1 All symbols inside single quotes is considered a string. 2 When using printf, do not surround the variable with quotes. (in this example)
The code below should fix it,
for i in `cat stvv.txt`;
printf 'if(!strcmp('$i',optarg))' > my_file;
done
basically, break the printf statement into three parts.
1: the string 'if(!strcmp('
2: $i (no quotes)
3: the string ',optarg))'
hope that helps!
To insert a string into a printf format, use %s in the format string:
$ for line in $(cat stvv.txt); do printf 'if(!strcmp("%s",optarg))\n' "$line"; done
if(!strcmp("hello",optarg))
if(!strcmp("world",optarg))
The code $(cat stvv.txt) will perform word splitting and pathname expansion on the contents of stvv.txt. You probably don't want that. It is generally safer to use a while read ... done <stvv.txt loop such as this one:
$ while read -r line; do printf 'if(!strcmp("%s",optarg))\n' "$line"; done <stvv.txt
if(!strcmp("hello",optarg))
if(!strcmp("world",optarg))
Aside on cat
If you are using bash, then $(cat stvv.txt) could be replaced with the more efficient $(<stvv.txt). This question, however, is tagged shell not bash. The cat form is POSIX and therefore portable to all POSIX shells while the bash form is not.
Related
I'm trying to figure out how to use a variable containing an ANSI-C quoting string as an argument for a subsequent bash command.
The string in the variable itself is a list of files (can virtually be a list of anything).
For example, I have a file containing a list of other files, for example test.lst containing :
>$ cat test.lst
a.txt
b.txt
c.txt
I need to pass the file content as a single string so I'm doing :
test_str=$(cat test.lst)
then converts to ANSI-C quoting string:
test_str=${test_str#Q}
So at the end I have :
>$ test_str=$(cat test.lst)
>$ test_str=${test_str#Q}
>$ echo $test_str
$'a.txt\nb.txt\nc.txt'
which is what I'm looking for.
Then problem arises when I try to reuse this variable as a string list in another bash command.
For example direct use into a for loop :
>$ for str in $test_str; do echo $str; done
$'a.txt\nb.txt\nc.txt'
What I expect at this step is that it prints the same thing as the content of the original test.lst
I also tried expanding it back but it leaves leading $' and trailing '
>$ str=${test_str#E}
>$ echo $str
$'a.txt b.txt c.txt'
I also tried printf and some other stuffs to no avail. What is the correct way to use such ANSI-C quoting variable into a bash command ?
How about:
eval echo "${test_str}"
I believe that an ANSI-C quoted string is meant to be evaluated by bash command line parser.
In the first place, why do you need quoting? Just keep the data untouched stored as elements of an array:
mapfile -t filelist < test.lst
# iterate through the list
for file in "${filelist[#]}"; do printf '%s\n' "$file"; done
I'm trying to configure a file with a bash script. And the variables in the bash script are not written in file as it is written in script.
Ex:
#!/bin/bash
printf "%s" "template("$DATE\t$HOST\t$PRIORITY\t$MSG\n")" >> /file.txt
exit 0
This results to template('tttn') instead of template("$DATE\t$HOST\t$PRIORITY\t$MSG\n in file.
How do I write in the script so that the result is template("$DATE\t$HOST\t$PRIORITY\t$MSG\n in the configured file?
Is it possible to write variable as it looks in script to file?
Enclose the strings you want to write within single quotes to avoid variable replacement.
> FOO=bar
> echo "$FOO"
bar
> echo '$FOO'
$FOO
>
Using printf in any shell script is uncommon, just use echo with the -e option.
It allows you to use ANSI C metacharacters, like \t or \n. The \n at the end however isn't necessary, as echo will add one itself.
echo -e "template(${DATE}\t${HOST}\t${PRIORITY}\t${MSG})" >> file.txt
The problem with what you've written is, that ANSI C metacharacters, like \t can only be used in the first parameter to printf.
So it would have to be something like:
printf 'template(%s\t%s\t%s\t%s)\n' ${DATE} ${HOST} ${PRIORITY} ${MSG} >> file.txt
But I hope we both agree, that this is very hard on the eyes.
There are several escaping issues and the power of printf has not been used, try
printf 'template(%s\t%s\t%s\t%s)\n' "${DATE}" "${HOST}" "${PRIORITY}" "${MSG}" >> file.txt
Reasons for this separate answer:
The accepted answer does not fit the title of the question (see comment).
The post with the right answer
contains wrong claims about echo vs printf as of this post and
is not robust against whitespace in the values.
The edit queue is full at the moment.
I'd like to convert a list separated with '\n' in another one separated with space.
Ex:
Get a dictionary like ispell english dictionary. http://downloads.sourceforge.net/wordlist/ispell-enwl-3.1.20.zip
My initial idea was using a variable as accumulator:
a=""; cat american.0 | while read line; do a="$a $line"; done; echo $a
... but it results '\n' string!!!
Questions:
Why is it not working?
What is the correct way to do that?
Thanks.
The problem is that when you have a pipeline:
command_1 | command_2
each command is run in a separate subshell, with a separate copy of the parent environment. So any variables that the command creates, or any modifications it makes to existing variables, will not be perceived by the containing shell.
In your case, you don't really need the pipeline, because this:
cat filename | command
is equivalent, in every way that you need, to this:
command < filename
So you can write:
a=""; while read line; do a="$a $line"; done < american.0; echo $a
to avoid creating any subshells.
That said, according to this StackOverflow answer, you can't really rely on a shell variable being able to hold more than about 1–4KB of data, so you probably need to rethink your overall approach. Storing the entire word-list in a shell variable likely won't work, and even if it does, it likely won't work well.
Edited to add: To create a temporary file named /tmp/american.tmp that contains what the variable $a would have, you can write:
while IFS= read -r line; do
printf %s " $line"
done < american.0 > /tmp/american.tmp
If you want to replace '\n' with a space, you can simply use tr as follows:
a=$(tr '\n' ' ' < american.0)
I have a variable in a shell script in which I'd like to format the data. The variable stores new data during every iteration of a loop. Each time the new data is stored, I'd like to insert a new line character. Here is how I'm trying to store the data into the variable.
VARIABLE="$VARIABLE '\n' SomeData"
Unfortunately, the output includes the literal '\n' Any help would be appreciative.
Try $'\n':
VAR=a
VAR="$VAR"$'\n'b
echo "$VAR"
gives me
a
b
A common technique is:
nl='
'
VARIABLE="PreviousData"
VARIABLE="$VARIABLE${nl}SomeData"
echo "$VARIABLE"
PreviousData
SomeData
Also common, to prevent inadvertently having your string start with a newline:
VARIABLE="$VARIABLE${VARIABLE:+$nl}SomeData"
(The expression ${VARIABLE:+$nl} will expand to a newline if and only if VARIABLE is set and non-empty.)
VAR="one"
VAR="$VAR.\n.two"
echo -e $VAR
Output:
one.
.two
Other than $'\n' you can use printf also like this:
VARIABLE="Foo Bar"
VARIABLE=$(printf "${VARIABLE}\nSomeData")
echo "$VARIABLE"
OUTPUT:
Foo Bar
SomeData
I had a problem with all the other solutions: when using a # followed by SPACE (quite common when writing in Markdown) both would get split onto a new line.
So, another way of doing it would involve using single quotes so that the "\n" get rendered.
FOO=$'# Markdown Title #\n'
BAR=$'Be *brave* and **bold**.'
FOOBAR="$FOO$BAR"
echo "$FOOBAR"
Output:
# Markdown Title #
Be *brave* and **bold**.
Single quote All special characters between these quotes lose their
special meaning.https://www.tutorialspoint.com/unix/unix-quoting-mechanisms.htm
So the syntax you use does something different that you want to achieve.
This is what you need:
The $'\X' construct makes the -e option in echo unnecessary.
https://linux.die.net/abs-guide/escapingsection.html
echo -e "something\nsomething"
or
echo "something"$'\n'"something"
It's a lot simpler than you think:
VARIABLE="$VARIABLE
SomeData"
Building upon the first two solutions, I'd do like shown below. Concatenating strings with the '+=' operator, somehow looks clearer to me.
Also rememeber to use printf as opposed to echo, you will save yourself so much trouble
sometext="This is the first line"
sometext+=$'\n\n'
sometext+="This is the second line AFTER the inserted new lines"
printf '%s' "${sometext}"
Outputs:
This is the first line
This is the third line AFTER the inserted new line
Your problem is in the echo command, in ash you have to use the option -e to expand special characters. This should work for you:
VAR="First line"
VAR="$VAR\nSecond line"
echo -e $VAR
This outputs
First line
Second line
I'm writing a shell script that should be somewhat secure, i.e., does not pass secure data through parameters of commands and preferably does not use temporary files. How can I pass a variable to the standard input of a command?
Or, if it's not possible, how can I correctly use temporary files for such a task?
Passing a value to standard input in Bash is as simple as:
your-command <<< "$your_variable"
Always make sure you put quotes around variable expressions!
Be cautious, that this will probably work only in bash and will not work in sh.
Simple, but error-prone: using echo
Something as simple as this will do the trick:
echo "$blah" | my_cmd
Do note that this may not work correctly if $blah contains -n, -e, -E etc; or if it contains backslashes (bash's copy of echo preserves literal backslashes in absence of -e by default, but will treat them as escape sequences and replace them with corresponding characters even without -e if optional XSI extensions are enabled).
More sophisticated approach: using printf
printf '%s\n' "$blah" | my_cmd
This does not have the disadvantages listed above: all possible C strings (strings not containing NULs) are printed unchanged.
(cat <<END
$passwd
END
) | command
The cat is not really needed, but it helps to structure the code better and allows you to use more commands in parentheses as input to your command.
Note that the 'echo "$var" | command operations mean that standard input is limited to the line(s) echoed. If you also want the terminal to be connected, then you'll need to be fancier:
{ echo "$var"; cat - ; } | command
( echo "$var"; cat - ) | command
This means that the first line(s) will be the contents of $var but the rest will come from cat reading its standard input. If the command does not do anything too fancy (try to turn on command line editing, or run like vim does) then it will be fine. Otherwise, you need to get really fancy - I think expect or one of its derivatives is likely to be appropriate.
The command line notations are practically identical - but the second semi-colon is necessary with the braces whereas it is not with parentheses.
This robust and portable way has already appeared in comments. It should be a standalone answer.
printf '%s' "$var" | my_cmd
or
printf '%s\n' "$var" | my_cmd
Notes:
It's better than echo, reasons are here: Why is printf better than echo?
printf "$var" is wrong. The first argument is format where various sequences like %s or \n are interpreted. To pass the variable right, it must not be interpreted as format.
Usually variables don't contain trailing newlines. The former command (with %s) passes the variable as it is. However tools that work with text may ignore or complain about an incomplete line (see Why should text files end with a newline?). So you may want the latter command (with %s\n) which appends a newline character to the content of the variable. Non-obvious facts:
Here string in Bash (<<<"$var" my_cmd) does append a newline.
Any method that appends a newline results in non-empty stdin of my_cmd, even if the variable is empty or undefined.
I liked Martin's answer, but it has some problems depending on what is in the variable. This
your-command <<< """$your_variable"""
is better if you variable contains " or !.
As per Martin's answer, there is a Bash feature called Here Strings (which itself is a variant of the more widely supported Here Documents feature):
3.6.7 Here Strings
A variant of here documents, the format is:
<<< word
The word is expanded and supplied to the command on its standard
input.
Note that Here Strings would appear to be Bash-only, so, for improved portability, you'd probably be better off with the original Here Documents feature, as per PoltoS's answer:
( cat <<EOF
$variable
EOF
) | cmd
Or, a simpler variant of the above:
(cmd <<EOF
$variable
EOF
)
You can omit ( and ), unless you want to have this redirected further into other commands.
Try this:
echo "$variable" | command
If you came here from a duplicate, you are probably a beginner who tried to do something like
"$variable" >file
or
"$variable" | wc -l
where you obviously meant something like
echo "$variable" >file
echo "$variable" | wc -l
(Real beginners also forget the quotes; usually use quotes unless you have a specific reason to omit them, at least until you understand quoting.)