How translate this:
echo "test" | tee -a test1 test2
To a pure UNIX descriptor redirection solution (oneliner better and no PIPES).
Is it possible?
If you want a byte written to one file descriptor (pipe, socket etc.) to show up as readable data on more than one file descriptor which are not dup()s of each other (but e.g. they correspond to two different regular files), then it's not possible on a generic Unix system. Even if the two file descriptors are dup()s, after reading the byte from one of them it would make the byte disappear from the other one, so it can't be read twice.
If you want to do it in Bash without using a |, then it's not possible.
If you want to do it in Zsh without using a |, then just follow chepner's comment: do setopt multios, and then echo test >>test1 >>test2. In the background Zsh will create a helper process to do the copying equivalent to what tee -a does.
Related
I am trying to create multiple descriptors to files named 1, 2, 3, etc. in bash.
For example, exec 9>abc/1 works just fine, but when I try to create descriptors in a for loop, like this: exec $[$i+8]>abc/$i, it doesn't work. I tried many different ways, but it seems that exec just does not accept variables. Is there any way to do what I want to?
EDIT: If not, maybe there is a way to use flock without descriptors?
Yes, exec doesn't accept variables for file descriptor numbers. As pointed out in comments, you can use
eval "exec $((i + 8))>"'"abc/$i"'
which, if $i is 1, is equivalent to
exec 9>"abc/$i"
Those complex quotes ensure that eval-ed and then exec-ed command is safe even if file name is changed to something different than abc/1.
But there is a warning:
Redirections using file descriptors greater than 9 should be used with care, as they may conflict with file descriptors the shell uses internally.
So if your task doesn't require consecutive file descriptor numbers, you can use automatically allocated descriptors:
Each redirection that may be preceded by a file descriptor number may instead be preceded by a word of the form {varname}. In this case, for each redirection operator except >&- and <&-, the shell will allocate a file descriptor greater than 10 and assign it to varname.
So,
exec {fd}>"abc/$i"
echo "$fd"
will open file descriptor 10 (or greater) for writing to abc/1 and print that file descriptor number (e.g. 10).
Edit: This question was originally bash specific. I'd still rather have a bash solution, but if there's a good way to do this in another shell then that would be useful to know as well!
Okay, top level description of the problem. I would like to be able to add a hook to bash such that, when a user enters, for example $cat foo | sort -n | less, this is intercepted and translated into wrapper 'cat foo | sort -n | less'. I've seen ways to run commands before and after each command (using DEBUG traps or PROMPT_COMMAND or similar), but nothing about how to intercept each command and allow it to be handled by another process. Is there a way to do this?
For an explanation of why I'd like to do this, in case people have other suggestions of ways to approach it:
Tools like script let you log everything you do in a terminal to a log (as, to an extent, does bash history). However, they don't do it very well - script mixes input with output into one big string and gets confused with applications such as vi which take over the screen, history only gives you the raw commands being typed in, and neither of them work well if you have commands being entered into multiple terminals at the same time. What I would like to do is capture much richer information - as an example, the command, the time it executed, the time it completed, the exit status, the first few lines of stdin and stdout. I'd also prefer to send this to a listening daemon somewhere which could happily multiplex multiple terminals. The easy way to do this is to pass the command to another program which can exec a shell to handle the command as a subprocess whilst getting handles to stdin, stdout, exit status etc. One could write a shell to do this, but you'd lose much of the functionality already in bash, which would be annoying.
The motivation for this comes from trying to make sense of exploratory data analysis like procedures after the fact. With richer information like this, it would be possible to generate decent reporting on what happened, squashing multiple invocations of one command into one where the first few gave non-zero exits, asking where files came from by searching for everything that touched the file, etc etc.
Run this bash script:
#!/bin/bash
while read -e line
do
wrapper "$line"
done
In its simplest form, wrapper could consist of eval "$LINE". You mentioned wanting to have timings, so maybe instead have time eval "$line". You wanted to capture exit status, so this should be followed by the line save=$?. And, you wanted to capture the first few lines of stdout, so some redirecting is in order. And so on.
MORE: Jo So suggests that handling for multiple-line bash commands be included. In its simplest form, if eval returns with "syntax error: unexpected end of file", then you want to prompt for another line of input before proceeding. Better yet, to check for proper bash commands, run bash -n <<<"$line" before you do the eval. If bash -n reports the end-of-line error, then prompt for more input to add to `$line'. And so on.
Binfmt_misc comes to mind. The Linux kernel has a capability to allow arbitrary executable file formats to be recognized and passed to user application.
You could use this capability to register your wrapper but instead of handling arbitrary executable, it should handle all executable.
Does running something like below cause the textfile lines to be directed to the STDIN of program.sh?
cat textfile | program.sh
Yes; and the rest of this answer comes to satisfy SO's requirement of minimum 30 characters per answer (excluding links).
http://en.wikipedia.org/wiki/Pipeline_(Unix)
Yes. You're writing the stdout from cat to the stdin of program.sh. Because cat isn't doing much except reading the file, you can also write it as:
program.sh < textfile
...which does the same thing.
From a technical standpoint, stdin is accessed through file descriptor 0, while stdout is file descriptor 1 and stderr is file descriptor 2. With this information, you can make more complicated redirections, such as redirecting stderr to the same place (or a different place!) than stdout. For a cheat sheet about redirections, see Peteris Krumins's Bash Redirections Cheat Sheet.
Yes.
You are running the command sort on a text file. The output goes to program.sh
I know the concept sounds a little abusive (?), but still - how can I create a pipe in bash which:
has no capacity
and therefore requires no memory copy, and
requires the write to be blocking
I am guessing a lot here. But possibly you are thinking about coprocesses and do not know what that term means.
bash supports coprocesses:
http://www.gnu.org/software/bash/manual/html_node/Coprocesses.html
The format for a coprocess is:
coproc [NAME] command [redirections]
This creates a coprocess named NAME.
If NAME is not supplied, the default name is COPROC.
NAME must not be supplied if command is a simple command (see Simple Commands);
otherwise, it is interpreted as the first word of the simple command.
When the coproc is executed, the shell creates an array variable (see Arrays) named NAME in the context of the executing shell. The standard output of command is connected via a pipe to a file descriptor in the executing shell, and that file descriptor is assigned to NAME[0].
The standard input of command is connected via a pipe to a file descriptor in the executing shell, and that file descriptor is assigned to NAME[1].
This pipe is established before any redirections specified by the command (see Redirections).
The file descriptors can be utilized as arguments to shell commands and redirections using standard word expansions.
I just want to know the difference between:
sort < output
and
sort output
in Linux. How does it work exactly?
This has been discussed on unix.stackexchange here: Performance difference between stdin and command line argument
In sort < file the shell performs redirection. It opens the file and passes the stdin file descriptor to the sort command which reads it.
In sort file, the sort command opens the file and then reads it.
sort < output is telling the shell to use the contents of the file output and dump it to standard in for the command sort.
sort output is telling the command sort to use the file output on disk as it's source.
Many unix commands will accept either standard in or a file as input. The acceptance of standard in allows easier chaining of commands, often for things like ps aux | grep "my process" | sort. (List all processes, filter by "my process", sort lines).
With sort < input the shell will run the sort command, and attach its input to the file 'input'.
With sort input the shell will run the sort command, and give it as parameter the string input. The sort command will then open the file to read it- content.
Effectively there is no difference.
sort < output uses a feature of the shell called file redirection (see e.g. here)
The shell opens tile file output and attaches that open file as stdin to the sort program.
sort output gives the output filename as an command line argument to sort.
sort, as many utilities that takes a filename as an argument, will try to read input from stdin if you do not give it a filename as an argument, such as in the first case here. In both cases, sort will read the content of the output file, sort it, and write the result to stdout.