As the image. All commands are similar.
I know how to use that, but I don't really know the detail.
Would anyone know that? Thank you very much.
# does `cat` read fd and print?
$ cat file
# does `cat` read from stdin and print?
$ cat < file
$ cat - < file
# with heredoc or herestring, what methods `cat` command use to read from heredoc?stdin?
$ cat << EOF
heredoc> test
heredoc> EOF
test
$ cat <<< "test"
$ cat - << EOF
heredoc> test
heredoc> EOF
test
$ cat - <<< "test"
# and I dont why these commands works?
$ cat <(echo "test")
$ cat - <<(echo "test")
# why this command doesn't work?
$ cat - <(echo "test")
Some reading material, all from the very useful Bash manual:
Redirection (<filename) -- causes standard input to be redirected to the file filename
Here documents (<<WORD) -- causes standard input to be redirected to the script source from the next line, up to but not including the line WORD
Here strings (<<<"string") -- causes standard input to be redirected to the string string (as though the string were written to a temporary file and then standard input redirected to that file)
Process substitution (<(command)) -- starts a process executing command and inserts a name onto the command line which acts like a filename, such that reading from that "file" produces the output of the command
The use of - to indicate the the source file is standard input is common to many commands and recommended by Posix. Many commands read from standard input if no file is specified. Some, like cat, implement both ways of indicating that the intention is to read from standard input.
Note that - and <(command) are both filename arguments, while <filename, <<WORD and <<<"string" are redirections. So while they superficially look similar, they are quite different under the hood. What they have in common is that they have to do with input; some of them (but not here-docs/strings) have analogues that have to do with output, using > instead of <.
Related
I'm using cat to create a new file via a shell script. It looks something like:
./script.sh > output.txt
How can I access output.txt as a variable in my script. I've tried $1 but that doesn't work.
The script looks something like:
#!/bin/sh
cat << EOF
echo "stuff"
EOF
Since there doesn't apear to be an os-agnostic way to do this, is there a way I pass the output into the script as an argument and then save the cat results to a file inside the script?
So the command would look like: ./script.sh output.txt and I can access the output as $1. Is something like this possible?
The Literal Question: Determining Where Your Stdout Was Redirected To
When a user runs:
./yourscript >outfile
...they're telling their shell to open outfile for write, and connect it to the stdout of your script, before starting your script. Consequently, all the operations on the filename are already finished when your script is started, so the name isn't passed to the script directly.
On Linux (only), you can access the location to which your stdout was redirected before your script was started through procfs:
output_dest=$(readlink -f /dev/fd/1)
echo "My output is being written to $output_dest"
This is literally interrogating where your first file descriptor (which is stdout) is open to. Note that the results won't always be useful -- if your program is being piped into something else, for instance, it might be something like [pipe: 12345].
If you care about portability or robustness, you should generally write your software in such a way that it doesn't need to know or care where its stdout is being directed.
The Best Practice: Redirecting Your Script's Stdout Yourself
Better practice, if you need an output filename that your script can access, is to accept that as an explicit argument:
#!/bin/sh
# ^^ note that that makes this a POSIX sh script, not a bash script
outfile=$1
exec >"$outfile" # all commands below here have their output written to outfile
cat >>EOF
This is written to $outfile
EOF
...and then directing the user to pass the filename as an argument:
./yourscript outfile
#!/bin/sh
outfile=$1
cat << EOF > "$outfile"
echo "stuff"
EOF
With
./script.sh output.txt
You write to the file output.txt
Setting a default value, in case the user doesn't pass an argument, is left for a different question.
I am trying to cat a file.txt and loop it twice through the whole content and copy it to a new file file_new.txt. The bash command I am using is as follows:
for i in {1..3}; do cat file.txt > file_new.txt; done
The above command is just giving me the same file contents as file.txt. Hence file_new.txt is also of the same size (1 GB).
Basically, if file.txt is a 1GB file, then I want file_new.txt to be a 2GB file, double the contents of file.txt. Please, can someone help here? Thank you.
Simply apply the redirection to the for loop as a whole:
for i in {1..3}; do cat file.txt; done > file_new.txt
The advantage of this over using >> (aside from not having to open and close the file multiple times) is that you needn't ensure that a preexisting output file is truncated first.
Note that the generalization of this approach is to use a group command ({ ...; ...; }) to apply redirections to multiple commands; e.g.:
$ { echo hi; echo there; } > out.txt; cat out.txt
hi
there
Given that whole files are being output, the cost of invoking cat for each repetition will probably not matter that much, but here's a robust way to invoke cat only once:[1]
# Create an array of repetitions of filename 'file' as needed.
files=(); for ((i=0; i<3; ++i)); do files[i]='file'; done
# Pass all repetitions *at once* as arguments to `cat`.
cat "${files[#]}" > file_new.txt
[1] Note that, hypothetically, you could run into your platform's command-line length limit, as reported by getconf ARG_MAX - given that on Linux that limit is 2,097,152 bytes (2MB) that's not likely, though.
You could use the append operator, >>, instead of >. Then adjust your loop count as needed to get the output size desired.
You should adjust your code so it is as follows:
for i in {1..3}; do cat file.txt >> file_new.txt; done
The >> operator appends data to a file rather than writing over it (>)
if file.txt is a 1GB file,
cat file.txt > file_new.txt
cat file.txt >> file_new.txt
The > operator will create file_new.txt(1GB),
The >> operator will append file_new.txt(2GB).
for i in {1..3}; do cat file.txt >> file_new.txt; done
This command will make file_new.txt(3GB),because for i in {1..3} will run three times.
As others have mentioned, you can use >> to append. But, you could also just invoke cat once and have it read the file 3 times. For instance:
n=3; cat $( yes file.txt | sed ${n}q ) > file_new.txt
Note that this solution exhibits a common anti-pattern and fails to properly quote the arguments, which will cause issues if the filename contains whitespace. See mklement's solution for a more robust solution.
I want to save a command to a file (for example I want to save the string "cat /etc/passwd" to a file) but I can't use the echo command.
How can I create and save string to a file directly without using echo command?
You can redirect cat to a file, type the text, and press Control-D when you're done, like this:
cat > file.txt
some text
some more text
^D
By ^D I mean to press Control-D at the end. The line must be empty.
It will not be part of the file, it is just to terminate the input.
Are you avoiding ECHO for security purposes (e.g. you're using a shared terminal and you don't want to leave trace in the shell history of what you've written inside your files) or you're just curious for an alternative method?
Simple alternative to echo:
As someone said, redirecting cat is probably the simpler way to go.
I'd suggest you to manually type your end-of-file, like this:
cat <<EOF > outputfile
> type here
> your
> text
> and finish it with
> EOF
Here's the string you're asking for, as an example:
cat <<EOF > myscript.sh
cat /etc/passwd
EOF
You probably don't want everyone to know you've peeked into that file, but if that's your purpose please notice that wrapping it inside an executable file won't make it more private, as that lines will be logged anyway...
Security - Avoiding history logs etc..
In modern shell, just try adding a space at the beginning of every command and use freely whatever you want.
BTW, my best hint is to avoid using that terminal at all, if you can. If you got two shells (another machine or even just another secure user in the same machine), I'd recommend you using netcat. See here: http://www.thegeekstuff.com/2012/04/nc-command-examples/?utm_source=feedburner
{ { command ls $(dirname $(which cat)) |
grep ^ca't$'; ls /etc/passwd; } |
tr \\n ' '; printf '\n'; } > output-file
But it's probably a lot simpler to just do : printf 'cat /etc/passwd\n'
To be clear, this is a tongue-in-cheek solution. The initial command is an extraordinarily convoluted way to get what you want, and this is intended to be a humorous answer. Perhaps instructive to understand.
I am not sure I understood you correctly but
cat /etc/passwd > target.file
use the > operator to write it to file without echoing
If you need to use it, inside a program :
cat <<EOF >file.txt
some text
some more text
EOF
I would imagine that you are probably trying to print the content of a string to a file, hence you mentioned echo.
You are avoiding this:
echo "cat /etc/passwd" > target.file
You can use a here string combined with cat.
cat > target.file <<< "cat /etc/passwd"
Now the file target.file will contain a string cat /etc/passwd.
$ cat target.file
cat /etc/passwd
$
To create string:
var1=your command
to save a file or variable in a file without echo use:
cat $FILE/VAR1 > /new/file/path
I am looking for a shell parameter to represent the redirection operand.
e.g: data/temp.txt in this command:
cat file.txt > data/temp.txt
Is there any such bash special Parameters which will allow me to open the file that I am redirecting to in interactive use, after the command will exit ?
$ cat file.txt > data/temp.txt
$ vim data/temp.txt
I don't think so. But since the redirection operand usually is the last, you can use Bash's history expansion:
echo hello > testfile
cat !$ # cat testfile
Or for interactive session, alt+. or esc+_ can be used to insert the last word of the previous line, as a comment pointed out.
Bash documentation for history expansion.
Another possibility here is bash's process substitution -- if you're most interested in opening an editor on the output of cat, but you're not as interested in writing the output to a file, you can use
vim <(cat file.txt)
This writes stdout to a named pipe and opens the named pipe (probably named something like /dev/fd/NN) in a text editor. You make your edits, and you can then save your edits using <esc>:w data/temp.txt if you wish.
I am learning the cat command of linux, and I found this command :
$ echo 'Text through stdin' | cat - file.txt
What does "-" mean here? If I don't type it , then 'Text through stdin' will not be shown.
it is common to write stdin as dash (-).
even man cat mentions that:
With no FILE, or when FILE is -, read standard input.
and the manpage even has an example illustrating the use of dash and ordinary filenames (which is quite close to your original question, but includes the answer):
cat f - g
Output f's contents, then standard input, then g's contents.
- tells cat to read from stdin. This is quite common, a lot of apps read from stdin if you pass - to them.
Some apps use - as stdout.
Here is an example of downloading blender and instead of writing it to a file we write it directly to stdout and pipe it to tar, which expands it on the fly during download.
wget -c https://download.blender.org/source/blender-2.90.1.tar.xz -O - | tar -xzv
Here the -O - tells wget to write directly to stdout
$ echo 'Text through stdin' | cat - file.txt
- tells cat to read from standard input, in this case, from the pipe, i.e, what echo 'Text through stdin' outputs.