The formal definition of pipe states that the STDOUT of the left file will be immediately piped to the STDIN of the right file.I have two files, hello.txt and human.txt. cat hello.txt returns Hello and cat human.txt returns I am human.Now if I do cat hello.txt | cat human.txt, shouldn't that return Hello I am human?Instead I'm seeing command not found.I am new to shell scripting.Can someone explain?
Background: A pipe arranges for the output of the command on the left (that is, contents written to FD 1, stdout) to be delivered as input to the command on the right (on FD 0, stdin). It does this by connecting the processes with a "pipe", or FIFO, and executing them at the same time; attempts to read from the FIFO will wait until the other process has written something, and attempts to write to the FIFO will wait until the other process is ready to read.
cat hello.txt | cat human.txt
...feeds the content of hello.txt into the stdin of cat human.txt, but cat human.txt isn't reading from its stdin; instead, it's been directed by its command line arguments to read only from human.txt.
Thus, that content on the stdin of cat human.txt is ignored and never read, and cat hello.txt receives a SIGPIPE when cat human.txt exits, and thereafter exits as well.
cat hello.txt | cat - human.txt
...by contrast will have the second cat read first from stdin (you could also use /dev/stdin in place of - on many operating systems, including Linux), then from a file.
You don't need to pipe them rather you can read from multiple file like below which will in-turn concatenate the content of both file content
cat hello.txt human.txt
| generally used when you want to fed output of first command to the second command in pipe. In this case specifically your second command is reading from a file and thus don't need to be piped. If you want to you can do like
echo "Hello" | cat - human.txt
First thing the command will not give an error it will print I m human i.e the contents of human.txt
Yeah you are right about pipe definition , but on the right side of pipe there should be some command.
If the command is for receiving the input and providing the output than it will give you output,otherwise the command will do its own behaviour
But here there is a command i.e cat human.txt on the right side but it will print its own contents and does no operation on the received input .
And also this error comes when when you write like
cat hello.txt | human.txt
bash will give you this error :
human.txt: command not found
Related
I have a text file a.txt
hello world
I use following commands:
cmd1:
$ more a.txt
output:
hello world
cmd2:
$ echo 'a.txt'|more
output:
a.txt
I thought cmd2 should equal to echo 'a.txt'|xargs -i more {}，but it's not.
I want to know why cmd2 worked like that and how to write code which work differently in pipeline mode.
Redirection with | or < controls what the stdin stream contains; it has no impact on a program's command line argument list.
Thus, more <a.txt (efficiently) or cat a.txt | more (inefficiently) both attach a file handle from which one can read the contents of a.txt to the stdin file handle of a new process before replacing that process with more. Similarly, echo a.txt | more makes a.txt itself the literal text that more reads from its stdin stream, which is the default place it's documented to get the input to display from, if not given any more specific filename(s) on its command line.
Generally, if you have a list of filenames and want to convert them to command-line arguments, this is what xargs is for (though using it without a great deal of care can introduce bugs, potentially-security-impacting ones).
Consider the following, which (using NUL rather than newline delimiters to separate filenames) is a safe use of xargs to take a list of filenames being piped into it, and transform that into an argument list to cat, used to concatenate all those files together and generate a single stream of input to more:
printf '%s\0' a.txt b.txt |
xargs -0 cat -- |
more
I am running some scripts with commands having cat pipelined with grep like:
cat file.txt | grep "pattern"
Most of the times there are no problems. But sometimes I get below error:
cat: write error: Broken pipe
So how do I find out when the command is causing this problem and why?
The reason is because the pipe is closed by grep when it still has some data to be read from cat. The signal SIGPIPE is caught by cat and it exits.
What usually happens in a pipeline is the shell runs cat in one process and grep in another. The stdout of cat is connected to the write-end of the pipe and stdin of grep to the read end. What happened was grep hit a pattern search that did not exist and exited immediately causing the read end of the pipe to be closed, which cat does not like since it has some more data to be write out to the pipe. Since the write actions happens to an other which has been closed other end, SIGPIPE is caught by the cat on which it immediately exits.
For such a trivial case, you could remove the pipeline usage altogether and run it as grep "pattern" file.txt when the file's contents are made available over the stdin of grep on which it could read from.
You can use only grep without pipe like this :
grep "pattern" file.txt
I think it's better to resolve this problem
I have a program which takes input and standard input until eof is given (CTRL-D on linux). I want to run this program with a lot of default input, then continue entering things until I manually hit CTRL-D to stop it. Is there any way to remove the EOF that a bash pipeline puts in?
IE: cat somedata.dat | <insert answer here> | ./myprogram such that myprogram never receives EOF on stdin.
Bash doesn't actually add an "end-of-file" character; there isn't such a thing. Rather, the problem is that ./myprogram reaches the end of its standard input (which is hooked up to the pipe), so the next time it tries to read a character, it gets end-of-file instead. There's no way to have it suddenly switch over to "stealing" the standard input from the terminal, because it's not hooked up to that input at all.
Instead, to feed more input to ./myprogram than just what's in somedata.dat, you can ask cat to start reading (and forwarding) its own standard input after it's finished reading somedata.dat:
cat somedata.dat - | ./myprogram
or
cat somedata.dat /dev/stdin | ./myprogram
Edited to add (per further question in the comments): If you have a more complicated pipeline feeding into ./myprogram, rather than just a file, then you can run your main command and then cat, piping the whole thing to ./myprogram:
{
reallyConfusingTransform < somedata.dat
cat
} | ./myprogram
or in one line:
{ reallyConfusingTransform < somedata.dat ; cat ; } | ./myprogram
(Note that I've also eliminated a "useless use of cat" (UUOC), but if you really prefer to use cat that way, you can still write cat somedata.dat | reallyConfusingTransform instead of reallyConfusingTransform < somedata.dat.)
As the image. All commands are similar.
I know how to use that, but I don't really know the detail.
Would anyone know that? Thank you very much.
# does `cat` read fd and print?
$ cat file
# does `cat` read from stdin and print?
$ cat < file
$ cat - < file
# with heredoc or herestring, what methods `cat` command use to read from heredoc?stdin?
$ cat << EOF
heredoc> test
heredoc> EOF
test
$ cat <<< "test"
$ cat - << EOF
heredoc> test
heredoc> EOF
test
$ cat - <<< "test"
# and I dont why these commands works?
$ cat <(echo "test")
$ cat - <<(echo "test")
# why this command doesn't work?
$ cat - <(echo "test")
Some reading material, all from the very useful Bash manual:
Redirection (<filename) -- causes standard input to be redirected to the file filename
Here documents (<<WORD) -- causes standard input to be redirected to the script source from the next line, up to but not including the line WORD
Here strings (<<<"string") -- causes standard input to be redirected to the string string (as though the string were written to a temporary file and then standard input redirected to that file)
Process substitution (<(command)) -- starts a process executing command and inserts a name onto the command line which acts like a filename, such that reading from that "file" produces the output of the command
The use of - to indicate the the source file is standard input is common to many commands and recommended by Posix. Many commands read from standard input if no file is specified. Some, like cat, implement both ways of indicating that the intention is to read from standard input.
Note that - and <(command) are both filename arguments, while <filename, <<WORD and <<<"string" are redirections. So while they superficially look similar, they are quite different under the hood. What they have in common is that they have to do with input; some of them (but not here-docs/strings) have analogues that have to do with output, using > instead of <.
In Unix, I've made a FIFO and I tried to read it with tail:
mkfifo fifo.file
tail -f fifo.file
Then I try to write messages into it from another process so I do as below:
cat > fifo.file
Then I type messages such as:
abc
def
Before I type Ctrl-D, nothing is printed at the first process (tail -f fifo.file).
Then I type Ctrl-D, the two lines above are printed.
Now If I do cat > fifo.file again and I type one line such as qwe and type Enter at the end of line, this string will be printed immediately at the first process.
I'm wondering why I get two different behaviors with the same command.
Is it possible to make it the second behavior without the first, meaning that when I cat the first time, I can see messages printed once I type Enter, instead of Ctrl-D?
This is just how tail works. Basically it outputs the specified file contents only when EOF occurs which Ctrl-D effectively sends to the terminal. And the -f switch just makes tail not exit and continue reading when that happens.
Meaning no matter the switches tail still needs EOF to output anything at all.
Just to test this you can use simple cat instead of tail:
term_1$ mkfifo fifo.file
term_1$ cat < fifo.file
...
term_2$ cat > fifo.file