I'm in doubt of the diference and which one is the better quote to execute a command in shell script.
For example, I have this two examples:
echo "The name of the computer is `uname -n`"
echo "The name of the computer is $(uname -n)"
Which one is better? Or there is no diference?
The $(...) one is generally recommended because it nests easier. Compare:
date -d "1970-01-01 $(echo "$(date +%s)-3600"|bc) sec UTC"
date -d "1970-01-01 `echo \"\`date +%s\`-3600\"|bc` sec UTC "
Related
I have a bash script and at certain points I am using echo to put some messages in a log file. The problem that I have is related to the DATE variable which will be static throughout the entire execution of the script.
I have this basic script below to illustrate the problem:
#!/bin/bash
DATE=`date +"%Y-%m-%dT%H:%M:%S%:z"`
echo "script started at $DATE"
echo "doing something"
sleep 2
echo "script finished at $DATE"
If I execute this script, the output of the $DATE variable is the same in both lines. Is there some bash magic that could nicely resolve this without having to replace $DATE with the command itself on each line?
Thanks in advance
Newer versions of the bash/printf builtin have support for generating datetime stamps without the need to spawn a subprocess to call date:
$ builtin printf --help
...snip...
Options:
-v var assign the output to shell variable VAR rather than
display it on the standard output
...snip...
In addition to the standard format specifications described in printf(1),
printf interprets:
%b expand backslash escape sequences in the corresponding argument
%q quote the argument in a way that can be reused as shell input
%(fmt)T output the date-time string resulting from using FMT as a format
string for strftime(3)
... snip ...
Instead of spawning a subprocess to call date, eg:
logdt=`date +"%Y-%m-%dT%H:%M:%S:%z"`
The same can be accomplished via printf -v by wrapping the desired format in %(...)T, eg:
printf -v logdt '%(%Y-%m-%dT%H:%M:%S:%z)T'
NOTE: assuming %:z should be :%z
Assuming you'll be tagging a lot of lines with datetime stamps then the savings from eliminating the subproces date calls could be huge.
Running a test of 1000 datetime stamp generations:
$ time for ((i=1;i<=1000;i++)); do { printf -v logdt '%(...)T' | logdate=$(date ...) }; done
Timings for printf -v logdt '%(...)T':
real 0m0.182s # ~130 times faster than $(date ...)
user 0m0.171s
sys 0m0.000s
Timings for logdt=$(date ...):
real 0m24.443s # ~130 times slower than printf -v
user 0m5.533s
sys 0m16.724s
With bash version 4.3+ , you can use the builtin printf to format datetimes. -1 below is a magic value that means "now".
#!/bin/bash
datefmt='%Y-%m-%dT%H:%M:%S%z'
printf "script started at %($datefmt)T\n" -1
echo "doing something"
sleep 2
printf "script finished at %($datefmt)T\n" -1
bash didn't recognize %:z for me.
This can help you:
#!/bin/bash
echo "script started at $(date +'%Y-%m-%dT%H:%M:%S%:z')"
echo "doing something"
sleep 2
echo "script finished at $(date +'%Y-%m-%dT%H:%M:%S%:z')"
You might want to create an alias if calling the full command looks clumsy to you.
#! /bin/sh
DB_USER='aaa';
DB_PASSWD='aaa1';
DB_NAME='data';
TABLE='datalog';
mysql --local-infile=1 --user=$DB_USER --password=$DB_PASSWD $DB_NAME -e "load data local infile '/home/demo/data1.csv' into table datalog fields terminated by ',' lines terminated by '\n';" -e echo "script executed successfully " | date "+%h %e"
My aim is to print a success message after the above script executes successfully. I have written the above command to do so but it is printing the date, not the echo statement.
The arguments to mysql -e should be SQL commands, not shell script.
The solution is much simpler than what you are trying.
#!/bin/sh
# Terminate immediately if a command fails
set -e
# Don't put useless semicolons at the end of each assignment
# Use lower case for your private variables
db_user='aaa'
db_passwd='aaa1'
db_name='data'
table='datalog'
# Quote strings
mysql --local-infile=1 --user="$db_user" --password="$db_passwd" "$db_name" \
-e "load data local infile '/home/demo/data1.csv' into table $table fields terminated by ',' lines terminated by '\n';"
# Just use date
# Print diagnostics to standard error, not standard output
date "+%h %e script executed successfully" >&2
The use of set -e is somewhat cumbersome, but looks like the simplest solution to your basic script. For a more complex script, maybe instead use something like
mysql -e "... stuff ..." || exit
to terminate on failure of this individual command, but allow other failures in the script. Perhaps see also What does set -e mean in a bash script?
If you want to preserve the exit code from mysql and always print a message to show what happened, probably take out the set -e and do something like
if mysql -e "... whatever ..."
then
date "+%h %e script completed successfully" >&2
else
rc=$?
date "+%h %e script failed: $rc" >&2
exit $rc
fi
As an aside, date does not read its standard input for anything, so you can't pipe echo to it. The script above simply uses only date, but here are a few different ways you could solve that.
# Merge this output with the next output
date "+%h %e" | tr '\n' ' ' >&2
echo script executed successfully >&2
or
# Use a command substitution to interpolate the output from date into
# the arguments for echo
echo "$(date "+%h %e") script executed successfully >&2
For more complex situations, maybe also look into xargs, though it's absolutely horribly overkill here.
date "+%h %e" | xargs -I {} echo script executed successfully {} >&2
If you use only date, you will need to be mindful of your use of % in any literal message; to print a literal per-cent sign from date, use %%.
As a further stylistic aside, you should avoid upper case for your private variables.
Background Info:
I'm trying to follow the example posted here: http://www.cyberciti.biz/faq/bash-for-loop/
I would like loop 9 times using a control variable called "i".
Problem Description
My code looks like this:
for i in {0..8..1}
do
echo "i is $i"
tmpdate=$(date -d "$i days" "+%b %d")
echo $tmpdate
done
When I run this code, the debug prints show me:
"i is {0..8..1}"
instead of being a value between 0 and 8.
What I've Checked So Far:
I've tried to check my version of bash to make sure it supports this type of syntax. I'm running version 4,2,25(1)
I also tried using C like syntax where you do for (i=0;i<=8;i++) but that doesn't work either.
Any suggestions would be appreciated.
Thanks.
EDIT 1
I've also tried the following code:
for i in {0..8};
do
echo "i is $i"
tmpdate=$(date -d "$i days" "+%b %d")
echo $tmpdate
done
And...
for i in {0..8}
do
echo "i is $i"
tmpdate=$(date -d "$i days" "+%b %d")
echo $tmpdate
done
They all fail with the same results.
I also tried:
#!/bin/bash
for ((i=0;i<9;i++));
do
echo "i is $i"
tmpdate=$(date -d "$i days" "+%b %d")
echo $tmpdate
done
And that gives me the error:
test.sh: 4: test.sh: Syntax error: Bad for loop variable
FYI. I'm running on ubuntu 12
EDIT 2
Ok... so i think Weberick tipped me off to the issue...
To execute the script, I was running "sh test.sh"
when in the code I had defined it as a BASH script! My bad!
But here's the thing. Ultimately, I need it to work in both bash and sh.
so now that I'm being careful to make sure that I invoke the script the right way... I've noticed the following results:
when defined as a bash script and i execute using bash, the C-style version works!
when defined as an sh script and i execute using sh, the C-style version fails
me#devbox:~/tmp/test$ sh test.sh
test.sh: 5: test.sh: Syntax error: Bad for loop variable
when defined as an sh script and i execute using sh the NON c style version ( aka for i in {n ..x}), I get the "i is {0..8}" output.
PS. The ";" doesn't make a difference if you have the do on the next line...just FYI.
Ubuntu's default shell is dash, which doesn't recognise either of the bashisms (brace expansion, C-style for loop) you tried. Try running your script using bash explicitly:
bash myscript.sh
or by setting the shebang to #!/bin/bash. Make sure NOT to run the script with sh myscript.sh.
dash should work if you use seq:
for i in $(seq 0 1 8); do
echo "$i"
done
Just {0..8} should work in bash, the default increment is 1. If you want to use a C-style for loop in bash:
for ((i=0;i<9;i++)); do
echo "$i"
done
I'm confident that
#!/bin/bash
for ((i=0;i<9;i++))
do
echo "i is $i"
tmpdate=$(date -d "$i days" "+%b %d")
echo $tmpdate
done
work on Ubuntu 12.04
If you still have an error, can you please try running
chmod +x test.sh
then
./test.sh
And the result is
i is 0
Apr 04
i is 1
Apr 05
i is 2
Apr 06
i is 3
Apr 07
i is 4
Apr 08
i is 5
Apr 09
i is 6
Apr 10
i is 7
Apr 11
i is 8
Apr 12
I'm no expert at bash but according to tdlp you need a ; after the for statement. There are many ways to to a range. This is one of them.
#!/bin/bash
for i in `seq 1 8`; do
echo $i
done
The site you quote says
Bash v4.0+ has inbuilt support for setting up a step value using {START..END..INCREMENT} syntax:
So you can just use {0..8..1} when you have a bash version greater than 4.0, which I guess is not the case (try bash --version in your terminal). Instead of {0..8..1} you can also use {0..8}.
If you have an older version you can use instead of {START..END..INCREMENT} the command $(seq START INCREMENT END) in the for loop.
I am trying to find out the number of occurrences of pattern in a file with following code:
#!/bin/sh
var='grep -c 'abc' file1'
if [ "$var" -lt 10 ]; then
echo "less than 10"
fi
I am getting the error: Illegal number: grep -c abc file1
Can someone please help.
Thanks.
Use backticks (`) instead of apostrophes ('):
#!/bin/sh
var=`grep -c 'abc' file1`
if [ "$var" -lt 10 ]; then
echo "less than 10"
fi
You probably want backticks (`) rather than single-quotes ('). i.e.:
var=`grep -c 'abc' file1`
You've used a single quote rather than a backtick so your var variable is actually set to a string literal rather than the result of that command. You'd see that if you echoed the variable first:
pax$ var='grep -c 'abc' file1'
pax$ echo "[$var]"
[grep -c abc file1]
The backtick version would be:
var=`grep -c 'abc' file1`
But I'd like to suggest using bash where possible for scripting. You'll be hard-pressed finding a mainstream distro that doesn't have it by default and it's considered by some to be more powerful than other shells. In fact, on some systems, /bin/sh is bash.
If you can go that rute, the $() construct is usually a better idea since you can nest them without pain:
var=$(grep -c 'abc' file1)
Try posting the followings to get better answer:
grep --version
bash --version if your shell is Bash or let us know which shell you are using.
use grep within back-ticks or as shown below
[[ is more versatile in Bash than [.
Finally, the following works on my machine without any error:
#!/bin/bash
var=$(grep -c "abc" file1)
if [[ "$var" -lt 10 ]]
then
echo "less than 10"
fi
Execution:
user#machine:~$ cat file1
abc
abcd
abcde
abcdef
user#machine:~$
user#machine:~$ ./t.sh
less than 10
user#machine:~$
I would like to store a command to use at a later time in a variable (not the output of the command, but the command itself).
I have a simple script as follows:
command="ls";
echo "Command: $command"; #Output is: Command: ls
b=`$command`;
echo $b; #Output is: public_html REV test... (command worked successfully)
However, when I try something a bit more complicated, it fails. For example, if I make
command="ls | grep -c '^'";
The output is:
Command: ls | grep -c '^'
ls: cannot access |: No such file or directory
ls: cannot access grep: No such file or directory
ls: cannot access '^': No such file or directory
How could I store such a command (with pipes/multiple commands) in a variable for later use?
Use eval:
x="ls | wc"
eval "$x"
y=$(eval "$x")
echo "$y"
Do not use eval! It has a major risk of introducing arbitrary code execution.
BashFAQ-50 - I'm trying to put a command in a variable, but the complex cases always fail.
Put it in an array and expand all the words with double-quotes "${arr[#]}" to not let the IFS split the words due to Word Splitting.
cmdArgs=()
cmdArgs=('date' '+%H:%M:%S')
and see the contents of the array inside. The declare -p allows you see the contents of the array inside with each command parameter in separate indices. If one such argument contains spaces, quoting inside while adding to the array will prevent it from getting split due to Word-Splitting.
declare -p cmdArgs
declare -a cmdArgs='([0]="date" [1]="+%H:%M:%S")'
and execute the commands as
"${cmdArgs[#]}"
23:15:18
(or) altogether use a bash function to run the command,
cmd() {
date '+%H:%M:%S'
}
and call the function as just
cmd
POSIX sh has no arrays, so the closest you can come is to build up a list of elements in the positional parameters. Here's a POSIX sh way to run a mail program
# POSIX sh
# Usage: sendto subject address [address ...]
sendto() {
subject=$1
shift
first=1
for addr; do
if [ "$first" = 1 ]; then set --; first=0; fi
set -- "$#" --recipient="$addr"
done
if [ "$first" = 1 ]; then
echo "usage: sendto subject address [address ...]"
return 1
fi
MailTool --subject="$subject" "$#"
}
Note that this approach can only handle simple commands with no redirections. It can't handle redirections, pipelines, for/while loops, if statements, etc
Another common use case is when running curl with multiple header fields and payload. You can always define args like below and invoke curl on the expanded array content
curlArgs=('-H' "keyheader: value" '-H' "2ndkeyheader: 2ndvalue")
curl "${curlArgs[#]}"
Another example,
payload='{}'
hostURL='http://google.com'
authToken='someToken'
authHeader='Authorization:Bearer "'"$authToken"'"'
now that variables are defined, use an array to store your command args
curlCMD=(-X POST "$hostURL" --data "$payload" -H "Content-Type:application/json" -H "$authHeader")
and now do a proper quoted expansion
curl "${curlCMD[#]}"
var=$(echo "asdf")
echo $var
# => asdf
Using this method, the command is immediately evaluated and its return value is stored.
stored_date=$(date)
echo $stored_date
# => Thu Jan 15 10:57:16 EST 2015
# (wait a few seconds)
echo $stored_date
# => Thu Jan 15 10:57:16 EST 2015
The same with backtick
stored_date=`date`
echo $stored_date
# => Thu Jan 15 11:02:19 EST 2015
# (wait a few seconds)
echo $stored_date
# => Thu Jan 15 11:02:19 EST 2015
Using eval in the $(...) will not make it evaluated later:
stored_date=$(eval "date")
echo $stored_date
# => Thu Jan 15 11:05:30 EST 2015
# (wait a few seconds)
echo $stored_date
# => Thu Jan 15 11:05:30 EST 2015
Using eval, it is evaluated when eval is used:
stored_date="date" # < storing the command itself
echo $(eval "$stored_date")
# => Thu Jan 15 11:07:05 EST 2015
# (wait a few seconds)
echo $(eval "$stored_date")
# => Thu Jan 15 11:07:16 EST 2015
# ^^ Time changed
In the above example, if you need to run a command with arguments, put them in the string you are storing:
stored_date="date -u"
# ...
For Bash scripts this is rarely relevant, but one last note. Be careful with eval. Eval only strings you control, never strings coming from an untrusted user or built from untrusted user input.
For bash, store your command like this:
command="ls | grep -c '^'"
Run your command like this:
echo $command | bash
Not sure why so many answers make it complicated!
use alias [command] 'string to execute'
example:
alias dir='ls -l'
./dir
[pretty list of files]
I tried various different methods:
printexec() {
printf -- "\033[1;37m$\033[0m"
printf -- " %q" "$#"
printf -- "\n"
eval -- "$#"
eval -- "$*"
"$#"
"$*"
}
Output:
$ printexec echo -e "foo\n" bar
$ echo -e foo\\n bar
foon bar
foon bar
foo
bar
bash: echo -e foo\n bar: command not found
As you can see, only the third one, "$#" gave the correct result.
I faced this problem with the following command:
awk '{printf "%s[%s]\n", $1, $3}' "input.txt"
I need to build this command dynamically:
The target file name input.txt is dynamic and may contain space.
The awk script inside {} braces printf "%s[%s]\n", $1, $3 is dynamic.
Challenge:
Avoid extensive quote escaping logic if there are many " inside the awk script.
Avoid parameter expansion for every $ field variable.
The solutions bellow with eval command and associative arrays do not work. Due to bash variable expansions and quoting.
Solution:
Build bash variable dynamically, avoid bash expansions, use printf template.
# dynamic variables, values change at runtime.
input="input file 1.txt"
awk_script='printf "%s[%s]\n" ,$1 ,$3'
# static command template, preventing double-quote escapes and avoid variable expansions.
awk_command=$(printf "awk '{%s}' \"%s\"\n" "$awk_script" "$input")
echo "awk_command=$awk_command"
awk_command=awk '{printf "%s[%s]\n" ,$1 ,$3}' "input file 1.txt"
Executing variable command:
bash -c "$awk_command"
Alternative that also works
bash << $awk_command
As you don't specify any scripting language, I would recommand tcl, the Tool Command Language for this kind of purpose.
Then in the first line, add the appropriate shebang:
#!/usr/local/bin/tclsh
with appropriate location you can retrieve with which tclsh.
In tcl scripts, you can call operating system commands with exec.
#!/bin/bash
#Note: this script works only when u use Bash. So, don't remove the first line.
TUNECOUNT=$(ifconfig |grep -c -o tune0) #Some command with "Grep".
echo $TUNECOUNT #This will return 0
#if you don't have tune0 interface.
#Or count of installed tune0 interfaces.
First of all, there are functions for this. But if you prefer variables then your task can be done like this:
$ cmd=ls
$ $cmd # works
file file2 test
$ cmd='ls | grep file'
$ $cmd # not works
ls: cannot access '|': No such file or directory
ls: cannot access 'grep': No such file or directory
file
$ bash -c $cmd # works
file file2 test
$ bash -c "$cmd" # also works
file
file2
$ bash <<< $cmd
file
file2
$ bash <<< "$cmd"
file
file2
Or via a temporary file
$ tmp=$(mktemp)
$ echo "$cmd" > "$tmp"
$ chmod +x "$tmp"
$ "$tmp"
file
file2
$ rm "$tmp"
Be careful registering an order with the: X=$(Command)
This one is still executed. Even before being called. To check and confirm this, you can do:
echo test;
X=$(for ((c=0; c<=5; c++)); do
sleep 2;
done);
echo note the 5 seconds elapsed
It is not necessary to store commands in variables even as you need to use it later. Just execute it as per normal. If you store in variables, you would need some kind of eval statement or invoke some unnecessary shell process to "execute your variable".