I always believed that a sub-shell was not a child process, but another
shell environment in the same process.
I use a basic set of built-ins:
(echo "Hello";read)
On another terminal:
ps -t pts/0
PID TTY TIME CMD
20104 pts/0 00:00:00 ksh
So, no child process in kornShell (ksh).
Enter bash, it appears to behave differently, given the same command:
PID TTY TIME CMD
3458 pts/0 00:00:00 bash
20067 pts/0 00:00:00 bash
So, a child process in bash.
From reading the man pages for bash, it is obvious that another process is created for a sub-shell,
however it fakes $$, which is sneeky.
Is this difference between bash and ksh expected, or am I reading the symptoms incorrectly?
Edit: additional information:
Running strace -f on bash and ksh on Linux shows that bash calls clone twice for the sample command (it does not call fork). So bash might be using threads (I tried ltrace but it core dumped!).
KornShell calls neither fork, vfork, nor clone.
In ksh, a subshell might or might not result in a new process. I don't know what the conditions are, but the shell was optimized for performance on systems where fork() was more expensive than it typically is on Linux, so it avoids creating a new process whenever it can. The specification says a "new environment", but that environmental separation may be done in-process.
Another vaguely-related difference is the use of new processes for pipes. In ksh and zsh, if the last command in a pipeline is a builtin, it runs in the current shell process, so this works:
$ unset x
$ echo foo | read x
$ echo $x
foo
$
In bash, all pipeline commands after the first are run in subshells, so the above doesn't work:
$ unset x
$ echo foo | read x
$ echo $x
$
As #dave-thompson-085 points out, you can get the ksh/zsh behavior in bash versions 4.2 and newer if you turn off job control (set +o monitor) and turn on the lastpipe option (shopt -s lastpipe). But my usual solution is to use process substitution instead:
$ unset x
$ read x < <(echo foo)
$ echo $x
foo
ksh93 works unusually hard to avoid subshells. Part of the reason is the avoidance of stdio and extensive use of sfio which allows builtins to communicate directly. Another reason is ksh can in theory have so many builtins. If built with SHOPT_CMDLIB_DIR, all of the cmdlib builtins are included and enabled by default. I can't give a comprehensive list of places where subshells are avoided, but it's typically in situations where only builtins are used, and where there are no redirects.
#!/usr/bin/env ksh
# doCompat arr
# "arr" is an indexed array name to be assigned an index corresponding to the detected shell.
# 0 = Bash, 1 = Ksh93, 2 = mksh
function doCompat {
${1:+:} return 1
if [[ ${BASH_VERSION+_} ]]; then
shopt -s lastpipe extglob
eval "${1}[0]="
else
case "${BASH_VERSINFO[*]-${!KSH_VERSION}}" in
.sh.version)
nameref v=$1
v[1]=
if builtin pids; then
function BASHPID.get { .sh.value=$(pids -f '%(pid)d'); }
elif [[ -r /proc/self/stat ]]; then
function BASHPID.get { read -r .sh.value _ </proc/self/stat; }
else
function BASHPID.get { .sh.value=$(exec sh -c 'echo $PPID'); }
fi 2>/dev/null
;;
KSH_VERSION)
nameref "_${1}=$1"
eval "_${1}[2]="
;&
*)
if [[ ! ${BASHPID+_} ]]; then
echo 'BASHPID requires Bash, ksh93, or mksh >= R41' >&2
return 1
fi
esac
fi
}
function main {
typeset -a myShell
doCompat myShell || exit 1 # stripped-down compat function.
typeset x
print -v .sh.version
x=$(print -nv BASHPID; print -nr " $$"); print -r "$x" # comsubs are free for builtins with no redirections
_=$({ print -nv BASHPID; print -r " $$"; } >&2) # but not with a redirect
_=$({ printf '%s ' "$BASHPID" $$; } >&2); echo # nor for expansions with a redirect
_=$(printf '%s ' "$BASHPID" $$ >&2); echo # but if expansions aren't redirected, they occur in the same process.
_=${ { print -nv BASHPID; print -r " $$"; } >&2; } # However, ${ ;} is always subshell-free (obviously).
( printf '%s ' "$BASHPID" $$ ); echo # Basically the same rules apply to ( )
read -r x _ <<<$(</proc/self/stat); print -r "$x $$" # These are free in {{m,}k,z}sh. Only Bash forks for this.
printf '%s ' "$BASHPID" $$ | cat # Sadly, pipes always fork. It isn't possible to precisely mimic "printf -v".
echo
} 2>&1
main "$#"
out:
Version AJM 93v- 2013-02-22
31732 31732
31735 31732
31736 31732
31732 31732
31732 31732
31732 31732
31732 31732
31738 31732
Another neat consequence of all this internal I/O handling is some buffering issues just go away. Here's a funny example of reading lines with tee and head builtins (don't try this in any other shell).
$ ksh -s <<\EOF
integer -a x
builtin head tee
printf %s\\n {1..10} |
while head -n 1 | [[ ${ { x+=("$(tee /dev/fd/{3,4})"); } 3>&1; } ]] 4>&1; do
print -r -- "${x[#]}"
done
EOF
1
0 1
2
0 1 2
3
0 1 2 3
4
0 1 2 3 4
5
0 1 2 3 4 5
6
0 1 2 3 4 5 6
7
0 1 2 3 4 5 6 7
8
0 1 2 3 4 5 6 7 8
9
0 1 2 3 4 5 6 7 8 9
10
0 1 2 3 4 5 6 7 8 9 10
The bash manpage reads:
Each command in a pipeline is executed as a separate process (i.e., in a subshell).
While this sentence is about pipes, it strongly implies a subshell is a separate process.
Wikipedia's disambiguation page also describes a subshell in child-process terms. A child process is certainly itself a process.
The ksh manpage (at a glance) isn't direct about its own definition of a subshell, so it does not imply one way or the other that a subshell is a different process.
Learning the Korn Shell says that they are different processes.
I'd say you're missing something (or the book is wrong or out of date).
The Korn shell does not necessarily use a subshell for command substitution. They are usually handled in the same process. Exceptions include I/O operations
To go a bit farther, I had a command giving a variable value that looked like this, in ksh93, from a VERY old script:
my_variable=(`cat ./my_file`)
In other words, parentheses around the backticked command substitution. "my_file" is a list of 4-digit octal numbers, one to a line.
When this is supplied this way in ksh93t and later, the newlines are preserved, and you can step through the numbers in the variable using a counter. For example, the following code would give a 4 digit octal number from the list discussed above, after which, you would increment the counter:
data_I_want=$(echo "${my_variable[$my_counter]}")
In ksh93, the command for the variable can also be done with this:
my_variable=($(cat ./my_file))
and, finally, to eliminate the "useless use of cat",
my_variable=($(<./my_file))
If the command is structured without the outer parentheses, the newlines are stripped (a POSIX standard), and the first use of the variable includes all of the numbers from the file. Subsequent calls to the variable using the counter return null values.
Putting the command inside parentheses forces the use of a subshell in a new process, and skirts the necessity of resetting the default field separator using IFS="".
Sorry for bumping something so old, but it seemed worthwhile to include this, as I haven't seen this particular behavior discussed elsewhere.
Related
I want to execute a long running command in Bash, and both capture its exit status, and tee its output.
So I do this:
command | tee out.txt
ST=$?
The problem is that the variable ST captures the exit status of tee and not of command. How can I solve this?
Note that command is long running and redirecting the output to a file to view it later is not a good solution for me.
There is an internal Bash variable called $PIPESTATUS; it’s an array that holds the exit status of each command in your last foreground pipeline of commands.
<command> | tee out.txt ; test ${PIPESTATUS[0]} -eq 0
Or another alternative which also works with other shells (like zsh) would be to enable pipefail:
set -o pipefail
...
The first option does not work with zsh due to a little bit different syntax.
Dumb solution: Connecting them through a named pipe (mkfifo). Then the command can be run second.
mkfifo pipe
tee out.txt < pipe &
command > pipe
echo $?
using bash's set -o pipefail is helpful
pipefail: the return value of a pipeline is the status of
the last command to exit with a non-zero status,
or zero if no command exited with a non-zero status
There's an array that gives you the exit status of each command in a pipe.
$ cat x| sed 's///'
cat: x: No such file or directory
$ echo $?
0
$ cat x| sed 's///'
cat: x: No such file or directory
$ echo ${PIPESTATUS[*]}
1 0
$ touch x
$ cat x| sed 's'
sed: 1: "s": substitute pattern can not be delimited by newline or backslash
$ echo ${PIPESTATUS[*]}
0 1
This solution works without using bash specific features or temporary files. Bonus: in the end the exit status is actually an exit status and not some string in a file.
Situation:
someprog | filter
you want the exit status from someprog and the output from filter.
Here is my solution:
((((someprog; echo $? >&3) | filter >&4) 3>&1) | (read xs; exit $xs)) 4>&1
echo $?
See my answer for the same question on unix.stackexchange.com for a detailed explanation and an alternative without subshells and some caveats.
By combining PIPESTATUS[0] and the result of executing the exit command in a subshell, you can directly access the return value of your initial command:
command | tee ; ( exit ${PIPESTATUS[0]} )
Here's an example:
# the "false" shell built-in command returns 1
false | tee ; ( exit ${PIPESTATUS[0]} )
echo "return value: $?"
will give you:
return value: 1
So I wanted to contribute an answer like lesmana's, but I think mine is perhaps a little simpler and slightly more advantageous pure-Bourne-shell solution:
# You want to pipe command1 through command2:
exec 4>&1
exitstatus=`{ { command1; printf $? 1>&3; } | command2 1>&4; } 3>&1`
# $exitstatus now has command1's exit status.
I think this is best explained from the inside out - command1 will execute and print its regular output on stdout (file descriptor 1), then once it's done, printf will execute and print icommand1's exit code on its stdout, but that stdout is redirected to file descriptor 3.
While command1 is running, its stdout is being piped to command2 (printf's output never makes it to command2 because we send it to file descriptor 3 instead of 1, which is what the pipe reads). Then we redirect command2's output to file descriptor 4, so that it also stays out of file descriptor 1 - because we want file descriptor 1 free for a little bit later, because we will bring the printf output on file descriptor 3 back down into file descriptor 1 - because that's what the command substitution (the backticks), will capture and that's what will get placed into the variable.
The final bit of magic is that first exec 4>&1 we did as a separate command - it opens file descriptor 4 as a copy of the external shell's stdout. Command substitution will capture whatever is written on standard out from the perspective of the commands inside it - but since command2's output is going to file descriptor 4 as far as the command substitution is concerned, the command substitution doesn't capture it - however once it gets "out" of the command substitution it is effectively still going to the script's overall file descriptor 1.
(The exec 4>&1 has to be a separate command because many common shells don't like it when you try to write to a file descriptor inside a command substitution, that is opened in the "external" command that is using the substitution. So this is the simplest portable way to do it.)
You can look at it in a less technical and more playful way, as if the outputs of the commands are leapfrogging each other: command1 pipes to command2, then the printf's output jumps over command 2 so that command2 doesn't catch it, and then command 2's output jumps over and out of the command substitution just as printf lands just in time to get captured by the substitution so that it ends up in the variable, and command2's output goes on its merry way being written to the standard output, just as in a normal pipe.
Also, as I understand it, $? will still contain the return code of the second command in the pipe, because variable assignments, command substitutions, and compound commands are all effectively transparent to the return code of the command inside them, so the return status of command2 should get propagated out - this, and not having to define an additional function, is why I think this might be a somewhat better solution than the one proposed by lesmana.
Per the caveats lesmana mentions, it's possible that command1 will at some point end up using file descriptors 3 or 4, so to be more robust, you would do:
exec 4>&1
exitstatus=`{ { command1 3>&-; printf $? 1>&3; } 4>&- | command2 1>&4; } 3>&1`
exec 4>&-
Note that I use compound commands in my example, but subshells (using ( ) instead of { } will also work, though may perhaps be less efficient.)
Commands inherit file descriptors from the process that launches them, so the entire second line will inherit file descriptor four, and the compound command followed by 3>&1 will inherit the file descriptor three. So the 4>&- makes sure that the inner compound command will not inherit file descriptor four, and the 3>&- will not inherit file descriptor three, so command1 gets a 'cleaner', more standard environment. You could also move the inner 4>&- next to the 3>&-, but I figure why not just limit its scope as much as possible.
I'm not sure how often things use file descriptor three and four directly - I think most of the time programs use syscalls that return not-used-at-the-moment file descriptors, but sometimes code writes to file descriptor 3 directly, I guess (I could imagine a program checking a file descriptor to see if it's open, and using it if it is, or behaving differently accordingly if it's not). So the latter is probably best to keep in mind and use for general-purpose cases.
(command | tee out.txt; exit ${PIPESTATUS[0]})
Unlike #cODAR's answer this returns the original exit code of the first command and not only 0 for success and 127 for failure. But as #Chaoran pointed out you can just call ${PIPESTATUS[0]}. It is important however that all is put into brackets.
In Ubuntu and Debian, you can apt-get install moreutils. This contains a utility called mispipe that returns the exit status of the first command in the pipe.
Outside of bash, you can do:
bash -o pipefail -c "command1 | tee output"
This is useful for example in ninja scripts where the shell is expected to be /bin/sh.
The simplest way to do this in plain bash is to use process substitution instead of a pipeline. There are several differences, but they probably don't matter very much for your use case:
When running a pipeline, bash waits until all processes complete.
Sending Ctrl-C to bash makes it kill all the processes of a pipeline, not just the main one.
The pipefail option and the PIPESTATUS variable are irrelevant to process substitution.
Possibly more
With process substitution, bash just starts the process and forgets about it, it's not even visible in jobs.
Mentioned differences aside, consumer < <(producer) and producer | consumer are essentially equivalent.
If you want to flip which one is the "main" process, you just flip the commands and the direction of the substitution to producer > >(consumer). In your case:
command > >(tee out.txt)
Example:
$ { echo "hello world"; false; } > >(tee out.txt)
hello world
$ echo $?
1
$ cat out.txt
hello world
$ echo "hello world" > >(tee out.txt)
hello world
$ echo $?
0
$ cat out.txt
hello world
As I said, there are differences from the pipe expression. The process may never stop running, unless it is sensitive to the pipe closing. In particular, it may keep writing things to your stdout, which may be confusing.
PIPESTATUS[#] must be copied to an array immediately after the pipe command returns.
Any reads of PIPESTATUS[#] will erase the contents.
Copy it to another array if you plan on checking the status of all pipe commands.
"$?" is the same value as the last element of "${PIPESTATUS[#]}",
and reading it seems to destroy "${PIPESTATUS[#]}", but I haven't absolutely verified this.
declare -a PSA
cmd1 | cmd2 | cmd3
PSA=( "${PIPESTATUS[#]}" )
This will not work if the pipe is in a sub-shell. For a solution to that problem,
see bash pipestatus in backticked command?
Base on #brian-s-wilson 's answer; this bash helper function:
pipestatus() {
local S=("${PIPESTATUS[#]}")
if test -n "$*"
then test "$*" = "${S[*]}"
else ! [[ "${S[#]}" =~ [^0\ ] ]]
fi
}
used thus:
1: get_bad_things must succeed, but it should produce no output; but we want to see output that it does produce
get_bad_things | grep '^'
pipeinfo 0 1 || return
2: all pipeline must succeed
thing | something -q | thingy
pipeinfo || return
Pure shell solution:
% rm -f error.flag; echo hello world \
| (cat || echo "First command failed: $?" >> error.flag) \
| (cat || echo "Second command failed: $?" >> error.flag) \
| (cat || echo "Third command failed: $?" >> error.flag) \
; test -s error.flag && (echo Some command failed: ; cat error.flag)
hello world
And now with the second cat replaced by false:
% rm -f error.flag; echo hello world \
| (cat || echo "First command failed: $?" >> error.flag) \
| (false || echo "Second command failed: $?" >> error.flag) \
| (cat || echo "Third command failed: $?" >> error.flag) \
; test -s error.flag && (echo Some command failed: ; cat error.flag)
Some command failed:
Second command failed: 1
First command failed: 141
Please note the first cat fails as well, because it's stdout gets closed on it. The order of the failed commands in the log is correct in this example, but don't rely on it.
This method allows for capturing stdout and stderr for the individual commands so you can then dump that as well into a log file if an error occurs, or just delete it if no error (like the output of dd).
It may sometimes be simpler and clearer to use an external command, rather than digging into the details of bash. pipeline, from the minimal process scripting language execline, exits with the return code of the second command*, just like a sh pipeline does, but unlike sh, it allows reversing the direction of the pipe, so that we can capture the return code of the producer process (the below is all on the sh command line, but with execline installed):
$ # using the full execline grammar with the execlineb parser:
$ execlineb -c 'pipeline { echo "hello world" } tee out.txt'
hello world
$ cat out.txt
hello world
$ # for these simple examples, one can forego the parser and just use "" as a separator
$ # traditional order
$ pipeline echo "hello world" "" tee out.txt
hello world
$ # "write" order (second command writes rather than reads)
$ pipeline -w tee out.txt "" echo "hello world"
hello world
$ # pipeline execs into the second command, so that's the RC we get
$ pipeline -w tee out.txt "" false; echo $?
1
$ pipeline -w tee out.txt "" true; echo $?
0
$ # output and exit status
$ pipeline -w tee out.txt "" sh -c "echo 'hello world'; exit 42"; echo "RC: $?"
hello world
RC: 42
$ cat out.txt
hello world
Using pipeline has the same differences to native bash pipelines as the bash process substitution used in answer #43972501.
* Actually pipeline doesn't exit at all unless there is an error. It executes into the second command, so it's the second command that does the returning.
Why not use stderr? Like so:
(
# Our long-running process that exits abnormally
( for i in {1..100} ; do echo ploop ; sleep 0.5 ; done ; exit 5 )
echo $? 1>&2 # We pass the exit status of our long-running process to stderr (fd 2).
) | tee ploop.out
So ploop.out receives the stdout. stderr receives the exit status of the long running process. This has the benefit of being completely POSIX-compatible.
(Well, with the exception of the range expression in the example long-running process, but that's not really relevant.)
Here's what this looks like:
...
ploop
ploop
ploop
ploop
ploop
ploop
ploop
ploop
ploop
ploop
5
Note that the return code 5 does not get output to the file ploop.out.
I tried this:
qs#BF:~$ echo aaa | read c
qs#BF:~$ echo $c
It gives nothing, which means $c is an empty macro.
But why the following one works:
qs#BF:~$ cat trim.hs | read cc
qs#BF:~$ echo $cc
import qualified Data.Text as T
It correctly gives the first line of trim.hs
There seams to be an exception when echo piped with read.
Am I right? Could you help me to make echo compatible with read? Please.
Neither of these "work"
echo aaa | read c
cat trim.hs | read cc
In bash, commands in a pipeline are all executed in subshells. So, the read command sets the variable c in a subshell, but then that subshell exits and its environment disappears
To demonstrate, let's query the value of $c in the subshell using a grouping construct:
unset c
echo 123 | { read c; echo in subshell: ">$c<"; }
echo in parent: ">$c<"
outputs
in subshell: >123<
in parent: ><
bash does have a setting to allow the last command in a pipeline to run in the current shell:
set +m # job control must be disabled
shopt -s lastpipe # enable the option
unset d
echo 456 | read d
echo ">$d<"
>456<
I think the underlying problem here is subshells that read is run in. These won't (always) propagate values to your invocation.
From the POSIX read standard it outlines how using read within subshells will not be visible to the caller:
If it is called in a subshell or separate utility execution environment, such as one of the following:
(read foo)
nohup read ...
find . -exec read ... \;
it shall not affect the shell variables in the caller's environment.
And noting in these shell tips:
POSIX allows any or all commands in a pipeline to be run in subshells, and which command (if any) runs in the main shell varies greatly between implementations — in particular Bash and ksh differ here. The standard idiom for overcoming this problem is to use a here document:
IFS= read var << EOF
$(foo)
EOF
I am using csh script and I want to see the line numbers when executing csh script in debug mode.
I am doing like
csh -x script_name
What do I have to do apart from this to see line numbers in debug mode?
There is no built-in feature to do this. The only method I can think of is (ab)using the postcmd special alias. This alias will be run after every command:
set _lineno = 0
alias postcmd '# _lineno++ && echo -n $_lineno\ '
echo a
echo b
echo c && echo d
Outputs:
% csh test.csh
1 2 a
3 b
4 c
d
Unfortunately, this will clutter the -x output:
% csh -x test.csh
set _lineno = 0
alias postcmd # _lineno++ && echo -n $_lineno\
# _lineno++
echo -n 1
1 # _lineno++
echo -n 2
2 echo a
a
# _lineno++
echo -n 3
3 echo b
b
# _lineno++
echo -n 4
4 echo c
c
echo d
d
Which we can sort of filter with grep:
% csh -x test.tcsh | & grep -Ev '^(# _lineno|echo -n [[:digit:]]+)
set _lineno = 0
alias postcmd # _lineno++ && echo -n $_lineno\
1 # _lineno++
2 echo a
a
3 echo b
b
4 echo c
c
echo d
d
All of this is less than perfect, and may not even work very well with large scripts...
As you've asked a number of csh related questions for the last few days:
While tcsh can work fine as an interactive shell, tcsh or csh is generally not well suited for scripting tasks. There are many things missing, incomplete, or simply behave in a "strange" way. It's even more ugly than normal shell scripting with POSIX (and that's saying a lot).
Unless you have a good reason not to (such as maintaining existing scripts), you probably should give up on csh for now, and go with either a "real" programming language (Python, Ruby, Perl, whatever), or use /bin/sh, bash, or zsh.
I have an ubuntu machine with default shell set to bash and both ways to the binary in $PATH:
$ which bash
/bin/bash
$ which sh
/bin/sh
$ ll /bin/sh
lrwxrwxrwx 1 root root 4 Mar 6 2013 /bin/sh -> bash*
But when I try to call a script that uses the inline file descriptor (that only bash can handle, but not sh) both calls behave differently:
$ . ./inline-pipe
reached
$ bash ./inline-pipe
reached
$ sh ./inline-pipe
./inline-pipe: line 6: syntax error near unexpected token `<'
./inline-pipe: line 6: `done < <(echo "reached")'
The example-script I am referring to looks like that
#!/bin/sh
while read line; do
if [[ "$line" == "reached" ]]; then echo "reached"; fi
done < <(echo "reached")
the real one is a little bit longer:
#!/bin/sh
declare -A elements
while read line
do
for ele in $(echo $line | grep -o "[a-z]*:[^ ]*")
do
id=$(echo $ele | cut -d ":" -f 1)
elements["$id"]=$(echo $ele | cut -d ":" -f 2)
done
done < <(adb devices -l)
echo ${elements[*]}
When bash is invoked as sh, it (mostly) restricts itself to features found in the POSIX standard. Process substitution is not one of those features, hence the error.
Theoretically, it is a feature of bash: if you call as "sh", it by default switches off all of its features. And the root shell is by default "/bin/sh".
Its primary goal is the security. Secondary is the produce some level of compatibility between some shells of the system, because it enables the system scripts to run in alternate (faster? more secure?) environment.
This is the theory.
Practically goes this so, that there are always people in a development team, who want to reduce and eliminate everything with various arguments (security, simplicity, safety, stability - but these arguments are going somehow always to the direction of the removal, deletion, destroying).
This is because the bash in debian doesn't have network sockets, this is because debian wasn't able in 20 years to normally integrate the best compressors (bz2, xz) - and this is because the root shell is by default so primitive, as of the PDP11 of the eighties.
I believe sh on ubuntu is actually dash which is smaller than bash with fewer features.
I would like to store a command to use at a later time in a variable (not the output of the command, but the command itself).
I have a simple script as follows:
command="ls";
echo "Command: $command"; #Output is: Command: ls
b=`$command`;
echo $b; #Output is: public_html REV test... (command worked successfully)
However, when I try something a bit more complicated, it fails. For example, if I make
command="ls | grep -c '^'";
The output is:
Command: ls | grep -c '^'
ls: cannot access |: No such file or directory
ls: cannot access grep: No such file or directory
ls: cannot access '^': No such file or directory
How could I store such a command (with pipes/multiple commands) in a variable for later use?
Use eval:
x="ls | wc"
eval "$x"
y=$(eval "$x")
echo "$y"
Do not use eval! It has a major risk of introducing arbitrary code execution.
BashFAQ-50 - I'm trying to put a command in a variable, but the complex cases always fail.
Put it in an array and expand all the words with double-quotes "${arr[#]}" to not let the IFS split the words due to Word Splitting.
cmdArgs=()
cmdArgs=('date' '+%H:%M:%S')
and see the contents of the array inside. The declare -p allows you see the contents of the array inside with each command parameter in separate indices. If one such argument contains spaces, quoting inside while adding to the array will prevent it from getting split due to Word-Splitting.
declare -p cmdArgs
declare -a cmdArgs='([0]="date" [1]="+%H:%M:%S")'
and execute the commands as
"${cmdArgs[#]}"
23:15:18
(or) altogether use a bash function to run the command,
cmd() {
date '+%H:%M:%S'
}
and call the function as just
cmd
POSIX sh has no arrays, so the closest you can come is to build up a list of elements in the positional parameters. Here's a POSIX sh way to run a mail program
# POSIX sh
# Usage: sendto subject address [address ...]
sendto() {
subject=$1
shift
first=1
for addr; do
if [ "$first" = 1 ]; then set --; first=0; fi
set -- "$#" --recipient="$addr"
done
if [ "$first" = 1 ]; then
echo "usage: sendto subject address [address ...]"
return 1
fi
MailTool --subject="$subject" "$#"
}
Note that this approach can only handle simple commands with no redirections. It can't handle redirections, pipelines, for/while loops, if statements, etc
Another common use case is when running curl with multiple header fields and payload. You can always define args like below and invoke curl on the expanded array content
curlArgs=('-H' "keyheader: value" '-H' "2ndkeyheader: 2ndvalue")
curl "${curlArgs[#]}"
Another example,
payload='{}'
hostURL='http://google.com'
authToken='someToken'
authHeader='Authorization:Bearer "'"$authToken"'"'
now that variables are defined, use an array to store your command args
curlCMD=(-X POST "$hostURL" --data "$payload" -H "Content-Type:application/json" -H "$authHeader")
and now do a proper quoted expansion
curl "${curlCMD[#]}"
var=$(echo "asdf")
echo $var
# => asdf
Using this method, the command is immediately evaluated and its return value is stored.
stored_date=$(date)
echo $stored_date
# => Thu Jan 15 10:57:16 EST 2015
# (wait a few seconds)
echo $stored_date
# => Thu Jan 15 10:57:16 EST 2015
The same with backtick
stored_date=`date`
echo $stored_date
# => Thu Jan 15 11:02:19 EST 2015
# (wait a few seconds)
echo $stored_date
# => Thu Jan 15 11:02:19 EST 2015
Using eval in the $(...) will not make it evaluated later:
stored_date=$(eval "date")
echo $stored_date
# => Thu Jan 15 11:05:30 EST 2015
# (wait a few seconds)
echo $stored_date
# => Thu Jan 15 11:05:30 EST 2015
Using eval, it is evaluated when eval is used:
stored_date="date" # < storing the command itself
echo $(eval "$stored_date")
# => Thu Jan 15 11:07:05 EST 2015
# (wait a few seconds)
echo $(eval "$stored_date")
# => Thu Jan 15 11:07:16 EST 2015
# ^^ Time changed
In the above example, if you need to run a command with arguments, put them in the string you are storing:
stored_date="date -u"
# ...
For Bash scripts this is rarely relevant, but one last note. Be careful with eval. Eval only strings you control, never strings coming from an untrusted user or built from untrusted user input.
For bash, store your command like this:
command="ls | grep -c '^'"
Run your command like this:
echo $command | bash
Not sure why so many answers make it complicated!
use alias [command] 'string to execute'
example:
alias dir='ls -l'
./dir
[pretty list of files]
I tried various different methods:
printexec() {
printf -- "\033[1;37m$\033[0m"
printf -- " %q" "$#"
printf -- "\n"
eval -- "$#"
eval -- "$*"
"$#"
"$*"
}
Output:
$ printexec echo -e "foo\n" bar
$ echo -e foo\\n bar
foon bar
foon bar
foo
bar
bash: echo -e foo\n bar: command not found
As you can see, only the third one, "$#" gave the correct result.
I faced this problem with the following command:
awk '{printf "%s[%s]\n", $1, $3}' "input.txt"
I need to build this command dynamically:
The target file name input.txt is dynamic and may contain space.
The awk script inside {} braces printf "%s[%s]\n", $1, $3 is dynamic.
Challenge:
Avoid extensive quote escaping logic if there are many " inside the awk script.
Avoid parameter expansion for every $ field variable.
The solutions bellow with eval command and associative arrays do not work. Due to bash variable expansions and quoting.
Solution:
Build bash variable dynamically, avoid bash expansions, use printf template.
# dynamic variables, values change at runtime.
input="input file 1.txt"
awk_script='printf "%s[%s]\n" ,$1 ,$3'
# static command template, preventing double-quote escapes and avoid variable expansions.
awk_command=$(printf "awk '{%s}' \"%s\"\n" "$awk_script" "$input")
echo "awk_command=$awk_command"
awk_command=awk '{printf "%s[%s]\n" ,$1 ,$3}' "input file 1.txt"
Executing variable command:
bash -c "$awk_command"
Alternative that also works
bash << $awk_command
As you don't specify any scripting language, I would recommand tcl, the Tool Command Language for this kind of purpose.
Then in the first line, add the appropriate shebang:
#!/usr/local/bin/tclsh
with appropriate location you can retrieve with which tclsh.
In tcl scripts, you can call operating system commands with exec.
#!/bin/bash
#Note: this script works only when u use Bash. So, don't remove the first line.
TUNECOUNT=$(ifconfig |grep -c -o tune0) #Some command with "Grep".
echo $TUNECOUNT #This will return 0
#if you don't have tune0 interface.
#Or count of installed tune0 interfaces.
First of all, there are functions for this. But if you prefer variables then your task can be done like this:
$ cmd=ls
$ $cmd # works
file file2 test
$ cmd='ls | grep file'
$ $cmd # not works
ls: cannot access '|': No such file or directory
ls: cannot access 'grep': No such file or directory
file
$ bash -c $cmd # works
file file2 test
$ bash -c "$cmd" # also works
file
file2
$ bash <<< $cmd
file
file2
$ bash <<< "$cmd"
file
file2
Or via a temporary file
$ tmp=$(mktemp)
$ echo "$cmd" > "$tmp"
$ chmod +x "$tmp"
$ "$tmp"
file
file2
$ rm "$tmp"
Be careful registering an order with the: X=$(Command)
This one is still executed. Even before being called. To check and confirm this, you can do:
echo test;
X=$(for ((c=0; c<=5; c++)); do
sleep 2;
done);
echo note the 5 seconds elapsed
It is not necessary to store commands in variables even as you need to use it later. Just execute it as per normal. If you store in variables, you would need some kind of eval statement or invoke some unnecessary shell process to "execute your variable".