I'm trying to do the following:
uname>>1.txt | echo #####>>1.txt | echo uname>>1.txt &
to get the following output:
uname
## ## ## ## ##
Linux (or whatever the uname is)
But instead all I get as output is:
uname
However if I try just:
uname>>1.txt | echo uname>>1.txt &
Then I do get the following output:
uname
Linux
Wondering if there is some limitation to this sort of piped redirection?
=======================================================================
I'll be calling this shell command from within a tcl script. Well actually there are a list of commands being executed from within the tcl script, and the outputs need to be formatted in the following way <------->
I wanted to run them in background to decrease the execution time, as the outputs of these commands are not related to each other.
I thought the commands in () would output the formatted output to 1.txt as a background process.
Would you suggest another way of doing this?
There are a number of problems here.
In general it's a bad idea to combine output redirection and pipes. Once redirected, there's nothing left to pipe.
Piping to echo doesn't make a bit of sense.
Use parentheses to put a suite of commands in the background.
You shouldn't be putting this in the background.
In general commands run from left to right, not right to left.
What you want is
(echo uname > 1.txt; echo ------ >>1.txt; uname >>1.txt)
Update (per comments and changes to the question)
You are continuing to invoke what is essentially undefined behavior with this command:
uname>>1.txt | echo uname>>1.txt &
The pipe from uname is invalid because there's nothing to pipe once you have redirected output. The pipe to echo is invalid because doesn't read from standard input. Which of the uname or echo commands prints it's output first to the file 1.txt is up for grabs here. This is apparently what you want:
bash -c 'echo uname >> 1.txt; echo ------ >> 1.txt; uname >> 1.txt'
Note the -c option to bash. This tells bash that the argument following -c is a string that contains shell commands.
Related
Recently in my work, I am facing an interesting problem regarding tee and process substitution.
Let's start with examples:
I have three little scripts:
$ head *.sh
File one.sh
#!/bin/bash
echo "one starts"
if [ -p /dev/stdin ]; then
echo "$(cat /dev/stdin) from one"
else
echo "no stdin"
fi
File two.sh
#!/bin/bash
echo "two starts"
if [ -p /dev/stdin ]; then
echo "$(cat /dev/stdin) from two"
else
echo "no stdin"
fi
File three.sh
#!/bin/bash
echo "three starts"
if [ -p /dev/stdin ]; then
sed 's/^/stdin for three: /' /dev/stdin
else
echo "no stdin"
fi
All three scripts read from standard input and print something to standard output.
The one.sh and two.sh are quite similar, but the three.sh is a bit different. It just adds some prefix to show what it reads from the standard input.
Now I am going to execute two commands:
1: echo "hello" | tee >(./one.sh) >(./two.sh) | ./three.sh
2: echo "hello" | tee >(./one.sh) >(./two.sh) >(./three.sh) >/dev/null
First in Bash and then in Z shell (zsh).
Bash (GNU bash, version 5.0.17(1))
$ echo "hello" | tee >(./one.sh) >(./two.sh) |./three.sh
three starts
stdin for three: hello
stdin for three: one starts
stdin for three: two starts
stdin for three: hello from two
stdin for three: hello from one
Why are the outputs of one.sh and two.sh mixed with the origin "hello" and passed to three.sh? I expected to see the output of one and two in standard output and only the "hello" is going to pass to three.sh.
Now the other command:
$ echo "hello" | tee >(./one.sh) >(./two.sh) >(./three.sh) >/dev/null
one starts
two starts
three starts
stdin for three: hello
hello from two
hello from one
<---!!!note here I don't have prompt unless I press Enter or Ctrl-c)
I redirect all standard output to /dev/null. Why do I see all output from all process substitution this time? Does it seem this behavior conflict with the one above?
Why don't I have the prompt after having executed the command?
Why does the command start in order one->two->three, but outputs come in 3->2->1? Even if I added sleep 3 in three.sh, the output is always 3-2-1. I know it should have something to do with standard input blocking, but I'd learn the exact reason.
Zsh (zsh 5.8 (x86_64-pc-linux-gnu))
Both commands,
echo "hello" | tee >(./one.sh) >(./two.sh) >(./three.sh) >/dev/null
echo "hello" | tee >(./one.sh) >(./two.sh) |./three.sh
Give the expected result:
one starts
three starts
two starts
hello from two
hello from one
stdin for three: hello
It works as expected. But the order of the output is random, it seems that Z shell does something non-blocking here, and the order of the output is dependent on how long each script has been running. What exactly leads to the result?
echo "hello"|tee >(./one.sh) >(./two.sh) |./three.sh
There are two possible order of operations for the tee part of the pipeline
First
Redirect standard output to a pipe that's connected to ./three.sh's standard input.
Set up the pipes and subprocesses for the command substitutions. They inherit the same redirected standard output pipe used by tee.
Execute tee.
Second
Set up the pipes and subprocesses for the the command substitutions. They share the same default standard output - to the terminal.
Redirect tee's standard output to a pipe that's connected to ./three.sh's standard input. This redirection doesn't affect the pipes set up in step 1.
Execute tee.
bash uses the first set of operations, zsh uses the second. In both cases, the order you see output from your shell scripts in is controlled by your OS's process scheduler and might as well be random. In the case where you redirect tee's standard output to /dev/null, they both seem to follow the second scenario and set up the subprocesses before the parent tee's redirection. This inconsistency on bash's part does seem unusual and a potential source of subtle bugs.
I can't replicate the missing prompt issue, but that's with bash 4.4.20 - I don't have 5 installed on this computer.
Im wondering why awk print different output when run in background
My script:
#!/bin/bash
echo "Name of shell is $SHELL"
relase=`uname -r`
echo "Release is: $relase"
if [ $SHELL != "/bin/bash" ] || [ $relase != "3.13.0-32-generic" ] ; then
echo "Warning, different configuration"
fi
if [ $# -eq 0 ] ; then
echo "Insert name of shell"
read sname
else
sname=$1
fi
awk -v sname="$sname" 'BEGIN {FS=":"} {if ($7 == sname) print $1 }' </etc/passwd &
When i run awk without ampersand, output is:
petr#PetrLinux-VirtualBox:~/Documents$ ./script1 /bin/bash
Name of shell is /bin/bash
Release is: 3.13.0-32-generic
root
petr
but when i run awk with ampersand - in background, output is folowing:
petr#PetrLinux-VirtualBox:~/Documents$ ./script1 /bin/bash
Name of shell is /bin/bash
Release is: 3.13.0-32-generic
petr#PetrLinux-VirtualBox:~/Documents$ root
petr
First record (root) is not printed on single line. Please tell me why ańd if there is way how to print on single line while running on background. Thanks.
What you see is a mix of two outputs. The first output is of your shell, printing the command prompt (petr#PetrLinux-VirtualBox:~/Documents$). The second output is root from your script.
As your shell script runs in the background, you now have two processes writing to your terminal window: the bash (printing the prompt), and your script, printing the awk-output. This then just mixes up.
The only way to prevent that is to redirect the output of the script to a file or other device, instead of your console. For example:
$ ./script1 /bin/bash &> output.txt &
The output is the same. It just appears to be different because two processes write on the same channel (your terminal) and mix their output. One process is the awk script and the other is your shell which prints a new prompt.
There is no way to determine the precise point in which the output will switch from one process to the other. It can be different on different systems (with the same software), it can also depend on the load of the computer and lots of other things.
The only decent solution is to redirect the output into a different stream, e. g. a file using > outfile.
For example, I have a very simple script, ping.sh:
#!/bin/bash
/usr/bin/xterm -e ping localhost
Right now, the output of the ping only shows up in the new xterm. I would like the output to show in both the original terminal (stdout of ping.sh) as well as in the new xterm. Is there a way to do this?
PS: I'm struggling with a title for this.
Seems like a weird thing to do, but this might work:
#!/bin/bash
f=$(mktemp)
touch "$f"
tail -f "$f" &
/usr/bin/xterm -e "sh -c 'ping localhost 2>&1 | tee -a $f'"
Alternatively, it's possible to get the file name of the terminal connected to standard input using the command tty, then use tee in the new terminal to copy the output to the old terminal.
/usr/bin/xterm -e "ping localhost | tee $(tty)"
Of course, this only works if the script is not called with redirected stdin.
In case the script is called with redirected stdin, solutions in shell - How to get the real name of the controlling terminal? - Unix & Linux Stack Exchange can be used. readlink /proc/self/fd/1, or ps (require some output parsing)
I'm trying some staff that is working perfectly when I write it in the regular shell, but when I include it in a bash script file, it doesn't.
First example:
m=`date +%m`
m_1=$((m-1))
echo $m_1
This gives me the value of the last month (actual minus one), but doesn't work if its executed from a script.
Second example:
m=6
m=$m"t"
echo m
This returns "6t" in the shell (concatenates $m with "t"), but just gives me "t" when executing from a script.
I assume all these may be answered easily by an experienced Linux user, but I'm just learning as I go.
Thanks in advance.
Re-check your syntax.
Your first code snippet works either from command line, from bash and from sh since your syntax is valid sh. In my opinion you probably have typos in your script file:
~$ m=`date +%m`; m_1=$((m-1)); echo $m_1
4
~$ cat > foo.sh
m=`date +%m`; m_1=$((m-1)); echo $m_1
^C
~$ bash foo.sh
4
~$ sh foo.sh
4
The same can apply to the other snippet with corrections:
~$ m=6; m=$m"t"; echo $m
6t
~$ cat > foo.sh
m=6; m=$m"t"; echo $m
^C
~$ bash foo.sh
6t
~$ sh foo.sh
6t
Make sure the first line of your script is
#!/bin/bash
rather than
#!/bin/sh
Bash will only enable its extended features if explicitly run as bash. If run as sh, it will operate in POSIX compatibility mode.
First of all, it works fine for me in a script, and on the terminal.
Second of all, your last line, echo m will just output "m". I think you meant "$m"..
When using a *nix shell (usually bash), I often spawn a sub-shell with which I can take care of a small task (usually in another directory), then exit out of to resume the session of the parent shell.
Once in a while, I'll lose track of whether I'm running a nested shell, or in my top-level shell, and I'll accidentally spawn an additional sub-shell or exit out of the top-level shell by mistake.
Is there a simple way to determine whether I'm running in a nested shell? Or am I going about my problem (by spawning sub-shells) in a completely wrong way?
The $SHLVL variable tracks your shell nesting level:
$ echo $SHLVL
1
$ bash
$ echo $SHLVL
2
$ exit
$ echo $SHLVL
1
As an alternative to spawning sub-shells you could push and pop directories from the stack and stay in the same shell:
[root#localhost /old/dir]# pushd /new/dir
/new/dir /old/dir
[root#localhost /new/dir]# popd
/old/dir
[root#localhost /old/dir]#
Here is a simplified version of part of my prompt:
PS1='$(((SHLVL>1))&&echo $SHLVL)\$ '
If I'm not in a nested shell, it doesn't add anything extra, but it shows the depth if I'm in any level of nesting.
Look at $0: if it starts with a minus -, you're in the login shell.
pstree -s $$ is quite useful to see your depth.
The environment variable $SHLVL contains the shell "depth".
echo $SHLVL
The shell depth can also be determined using pstree (version 23 and above):
pstree -s $$ | grep sh- -o | wc -l
I've found the second way to be more robust than the first whose value was reset when using sudo or became unreliable with env -i.
None of them can correctly deal with su.
The information can be made available in your prompt:
PS1='\u#\h/${SHLVL} \w \$ '
PS1='\u#\h/$(pstree -s $$ | grep sh- -o | tail +2 | wc -l) \w \$ '
The | tail +2 is there to remove one line from the grep output. Since we are using a pipeline inside a "$(...)" command substitution, the shell needs to invoke a sub-shell, so pstree report it and grep detects one more sh- level.
In debian-based distributions, pstree is part of the package psmisc. It might not be installed by default on non-desktop distributions.
As #John Kugelman says, echo $SHLVL will tell you the bash shell depth.
And as #Dennis Williamson shows, you can edit your prompt via the PS1 variable to get it to print this value.
I prefer that it always prints the shell depth value, so here's what I've done: edit your "~/.bashrc" file:
gedit ~/.bashrc
and add the following line to the end:
export PS1='\$SHLVL'":$SHLVL\n$PS1"
Now you will always see a printout of your current bash level just above your prompt. Ex: here you can see I am at a bash level (depth) of 2, as indicated by the $SHLVL:2:
$SHLVL:2
7510-gabriels ~ $
Now, watch the prompt as I go down into some bash levels via the bash command, then come back up via exit. Here you see my commands and prompt (response), starting at level 2 and going down to 5, then coming back up to level 2:
$SHLVL:2
7510-gabriels ~ $ bash
$SHLVL:3
7510-gabriels ~ $ bash
$SHLVL:4
7510-gabriels ~ $ bash
$SHLVL:5
7510-gabriels ~ $ exit
exit
$SHLVL:4
7510-gabriels ~ $ exit
exit
$SHLVL:3
7510-gabriels ~ $ exit
exit
$SHLVL:2
7510-gabriels ~ $
Bonus: always show in your terminal your current git branch you are on too!
Make your prompt also show you your git branch you are working on by using the following in your "~/.bashrc" file instead:
git_show_branch() {
__gsb_BRANCH=$(git symbolic-ref -q --short HEAD 2>/dev/null)
if [ -n "$__gsb_BRANCH" ]; then
echo "$__gsb_BRANCH"
fi
}
export PS1="\e[7m\$(git_show_branch)\e[m\n\h \w $ "
export PS1='\$SHLVL'":$SHLVL $PS1"
Source: I have no idea where git_show_branch() originally comes from, but I got it from Jason McMullan on 5 Apr. 2018. I then added the $SHLVL part shown above just last week.
Sample output:
$SHLVL:2 master
7510-gabriels ~/GS/dev/temp $
And here's a screenshot showing it in all its glory. Notice the git branch name, master, highlighted in white!
Update to the Bonus section
I've improved it again and put my ~/.bashrc file on github here. Here's a sample output of the new terminal prompt. Notice how it shows the shell level as 1, and it shows the branch name of the currently-checked-out branch (master in this case) whenever I'm inside a local git repo!:
Cross-referenced:
Output of git branch in tree like fashion
ptree $$ will also show you how many levels deep you are
If you running inside sub-shell following code will yield 2:
ps | fgrep bash | wc -l
Otherwise, it will yield 1.
EDIT Ok, it's not so robust approach as was pointed out in comments :)
Another thing to try is
ps -ef | awk '{print $2, " ", $8;}' | fgrep $PPID
will yield 'bash' if you in sub-shell.