I am extracting a part from an existing file and storing it as a string in a variable.The string looks something like this.
var="*a<br>*b<br>*c"
Now as * is a special character in unix it doesnot work in further operations(like sed,grep) until I put an escape character infront of every *
Thats why,I am doing something like this -
echo $var | sed 's/\*/\\*/g'
On running this command in bash we get
echo $var | sed 's/\*/\\*/g'
\*a<br>\*b<br>\*c
which is the desired output,but when I try to store this in a variable, I am getting back my original variable like so
var=`echo $var | sed 's/\*/\\*/g'`
echo $var
*a<br>*b<br>*c
I am assuming this happens because the variable ignores the backslashes interpreting them as escape characters. How can I retain the backslashes and store them as in a variable?
The problem is caused by backticks. Use $( ) instead, and it goes away:
var="*a<br>*b<br>*c"
var=$(printf '%s\n' "$var" | sed 's/\*/\\*/g')
printf '%s\n' "$var"
(Why is this problem caused by backticks? Because the only way to nest them is to escape the inner ones with backslashes, so they necessarily change how backslashes behave; whereas $( ), because it uses different starting and ending sigils, can be nested natively).
That said, if your shell is one (like bash) with ksh-inspired extensions, you don't need sed at all here, as the shell can perform simple string replacements natively via parameter expansion:
var="*a<br>*b<br>*c"
printf '%s\n' "${var//'*'/'\*'}"
For background on why this answer uses printf instead of echo, see Why is printf better than echo? at [unix.se], or the APPLICATION USAGE section of the POSIX specification for echo.
How do I print a newline? This merely prints \n:
$ echo -e "Hello,\nWorld!"
Hello,\nWorld!
Use printf instead:
printf "hello\nworld\n"
printf behaves more consistently across different environments than echo.
Make sure you are in Bash.
$ echo $0
bash
All these four ways work for me:
echo -e "Hello\nworld"
echo -e 'Hello\nworld'
echo Hello$'\n'world
echo Hello ; echo world
echo $'hello\nworld'
prints
hello
world
$'' strings use ANSI C Quoting:
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard.
You could always do echo "".
For example,
echo "Hello,"
echo ""
echo "World!"
On the off chance that someone finds themselves beating their head against the wall trying to figure out why a coworker's script won't print newlines, look out for this:
#!/bin/bash
function GET_RECORDS()
{
echo -e "starting\n the process";
}
echo $(GET_RECORDS);
As in the above, the actual running of the method may itself be wrapped in an echo which supersedes any echos that may be in the method itself. Obviously, I watered this down for brevity. It was not so easy to spot!
You can then inform your comrades that a better way to execute functions would be like so:
#!/bin/bash
function GET_RECORDS()
{
echo -e "starting\n the process";
}
GET_RECORDS;
Simply type
echo
to get a new line
POSIX 7 on echo
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html
-e is not defined and backslashes are implementation defined:
If the first operand is -n, or if any of the operands contain a <backslash> character, the results are implementation-defined.
unless you have an optional XSI extension.
So I recommend that you should use printf instead, which is well specified:
format operand shall be used as the format string described in XBD File Format Notation [...]
the File Format Notation:
\n <newline> Move the printing position to the start of the next line.
Also keep in mind that Ubuntu 15.10 and most distros implement echo both as:
a Bash built-in: help echo
a standalone executable: which echo
which can lead to some confusion.
str='hello\nworld'
$ echo | sed "i$str"
hello
world
You can also do:
echo "hello
world"
This works both inside a script and from the command line.
On the command line, press Shift+Enter to do the line break inside the string.
This works for me on my macOS and my Ubuntu 18.04 (Bionic Beaver) system.
For only the question asked (not special characters etc) changing only double quotes to single quotes.
echo -e 'Hello,\nWorld!'
Results in:
Hello,
World!
There is a new parameter expansion added in Bash 4.4 that interprets escape sequences:
${parameter#operator} - E operator
The expansion is a string that is the value of parameter with
backslash escape sequences expanded as with the $'…' quoting
mechanism.
$ foo='hello\nworld'
$ echo "${foo#E}"
hello
world
I just use echo without any arguments:
echo "Hello"
echo
echo "World"
To print a new line with echo, use:
echo
or
echo -e '\n'
This could better be done as
x="\n"
echo -ne $x
-e option will interpret backslahes for the escape sequence
-n option will remove the trailing newline in the output
PS: the command echo has an effect of always including a trailing newline in the output so -n is required to turn that thing off (and make it less confusing)
My script:
echo "WARNINGS: $warningsFound WARNINGS FOUND:\n$warningStrings
Output:
WARNING : 2 WARNINGS FOUND:\nWarning, found the following local orphaned signature file:
On my Bash script I was getting mad as you until I've just tried:
echo "WARNING : $warningsFound WARNINGS FOUND:
$warningStrings"
Just hit Enter where you want to insert that jump. The output now is:
WARNING : 2 WARNINGS FOUND:
Warning, found the following local orphaned signature file:
If you're writing scripts and will be echoing newlines as part of other messages several times, a nice cross-platform solution is to put a literal newline in a variable like so:
newline='
'
echo "first line${newline}second line"
echo "Error: example error message n${newline}${usage}" >&2 #requires usage to be defined
If the previous answers don't work, and there is a need to get a return value from their function:
function foo()
{
local v="Dimi";
local s="";
.....
s+="Some message here $v $1\n"
.....
echo $s
}
r=$(foo "my message");
echo -e $r;
Only this trick worked on a Linux system I was working on with this Bash version:
GNU bash, version 2.2.25(1)-release (x86_64-redhat-linux-gnu)
You could also use echo with braces,
$ (echo hello; echo world)
hello
world
This got me there....
outstuff=RESOURCE_GROUP=[$RESOURCE_GROUP]\\nAKS_CLUSTER_NAME=[$AKS_CLUSTER_NAME]\\nREGION_NAME=[$REGION_NAME]\\nVERSION=[$VERSION]\\nSUBNET-ID=[$SUBNET_ID]
printf $outstuff
Yields:
RESOURCE_GROUP=[akswork-rg]
AKS_CLUSTER_NAME=[aksworkshop-804]
REGION_NAME=[eastus]
VERSION=[1.16.7]
SUBNET-ID=[/subscriptions/{subidhere}/resourceGroups/makeakswork-rg/providers/Microsoft.Network/virtualNetworks/aks-vnet/subnets/aks-subnet]
Sometimes you can pass multiple strings separated by a space and it will be interpreted as \n.
For example when using a shell script for multi-line notifcations:
#!/bin/bash
notify-send 'notification success' 'another line' 'time now '`date +"%s"`
With jq:
$ jq -nr '"Hello,\nWorld"'
Hello,
World
Additional solution:
In cases, you have to echo a multiline of the long contents (such as code/ configurations)
For example:
A Bash script to generate codes/ configurations
echo -e,
printf might have some limitation
You can use some special char as a placeholder as a line break (such as ~) and replace it after the file was created using tr:
echo ${content} | tr '~' '\n' > $targetFile
It needs to invoke another program (tr) which should be fine, IMO.
I need to output a variable value to a file in a unix script. My problem it that the variable contains multiple lines. I need those to be output as '\n' literals in the file (a java options file), but I'm using echo and they always get processed into real new lines.
echo "-dmyproperty=$MULTILINE_VAR" >> jvm.options
I've tried echo options like -e o -E but they don't seem to do anything. Can anyone help?
You can use bash parameter substitution with an ANSI-C quoted newline
$ var="line1
line2
line3"
$ echo "${var//$'\n'/\\n}"
line1\nline2\nline3
Found this error in other questions, but I can't see how the solutions relate to this.
Assume a file test containing:
one
twoX
three
I can correct twoX with:
perl -0777 -i -pe 's/twoX/two/igm' test
I can make a function to do this:
replace_str(){ perl -0777 -i -pe 's/'$2'/'$3'/igm' $1; }
replace_str test twoX two
But this fails when the replacement contains a space (possibly other chars):
replace_str test two 'two frogs'
Substitution replacement not terminated at -e line 1.
The perl line works with the space. Why not when called in a function? I've tried with other quotes and e.g. $(echo two frogs) (with and without quotes).
It's because you end the string you pass as argument to Perl for your variable expansions. That makes the regex become two arguments.
Instead just put the whole regex, including variables, inside double-quotes and the shell should expand the variables properly.
So use "s/$2/$3/igm" instead.
I am using an external command to populate my bash prompt, which is run each time PS1 is evaluated. However, I have a problem when this command outputs non-printable characters (like color escape codes).
Here is an example:
$ cat green_cheese.sh
#!/bin/bash
echo -e "\033[32mcheese\033[0m"
$ export PS1="\$(./green_cheese.sh) \$"
cheese $ # <- cheese is green!
cheese $ <now type really long command>
The canonical way of dealing with non-printing characters in the PS1 prompt is to enclose them in \[ and \] escape sequences. The problem is that if you do this from the external command those escapes are not parsed by the PS1 interpreter:
$ cat green_cheese.sh
#!/bin/bash
echo -e "\[\033[32m\]cheese\[\033[0m\]"
$ export PS1="\$(./green_cheese.sh) \$"
\[\]cheese\[\] $ # <- FAIL!
Is there a particular escape sequence I can use from the external command to achieve the desired result? Or is there a way I can manually tell the prompt how many characters to set the prompt width to?
Assume that I can print anything I like from the external command, and that this command can be quite intelligent (for example, counting characters in the output). I can also make the export PS1=... command as complicated as required. However, the escape codes for the colors must come from the external command.
Thanks in advance!
I couldn't tell you exactly why this works, but replace \[ and \] with the actual characters that bash generates from them in your prompt:
echo -e "\001\033[32m\002cheese\001\033[0m\002"
[I learned this from some Stack Overflow post that I cannot find now.]
If I had to guess, it's that bash replaces \[ and \] with the two ASCII characters before executing the command that's embedded in the prompt, so that by the time green_cheese.sh completes, it's too late for bash to process the wrappers correctly, and so they are treated literally. One way to avoid this is to use PROMPT_COMMAND to build your prompt dynamically, rather than embedding executable code in the value of PS1.
prompt_cmd () {
PS1="$(green_cheese.sh)"
PS1+=' \$ '
}
PROMPT_COMMAND=prompt_cmd
This way, the \[ and \] are added to PS1 when it is defined, not when it is evaluated, so you don't need to use \001 and \002 directly.
If you can't edit the code generating the string containing ANSI color / control codes, you can wrap them after the fact.
The following will enclose ANSI control sequences in ASCII SOH (^A) and STX (^B) which are equivalent to \[ and \] respectively:
function readline_ANSI_escape() {
if [[ $# -ge 1 ]]; then
echo "$*"
else
cat # Read string from STDIN
fi | \
perl -pe 's/(?:(?<!\x1)|(?<!\\\[))(\x1b\[[0-9;]*[mG])(?!\x2|\\\])/\x1\1\x2/g'
}
Use it like:
$ echo $'\e[0;1;31mRED' | readline_ANSI_escape
Or:
$ readline_ANSI_escape "$string"
As a bonus, running the function multiple times will not re-escape already escaped control codes.
I suspect that if you echo the value of $PS1 after your first example, you’ll find that its value is the word “cheese” in green. (At least, that’s what I see when I run your example.) At first glance, this is what you want — the word “cheese” in green! Except that what you really wanted was the word cheese preceded by the escape codes that produce green. What you did by using the -e flag for echo is produce a value with the escape codes already evaluated.
That happens to work for the specification of colors, but as you’ve found, it mangles the “non-printing sequence” markers into something the $PS1 interpreter doesn’t properly understand.
Fortunately, the solution is simple: drop the -e flag. echo will then leave the escape sequences untouched, and the $PS1 interpreter will Do The Right Thing™.