I need to:
Accept STDIN in my script from a pipe
save it to a temp file so that I don't modify the original source
perform operations on the temp file to generate some output
output to STDOUT
Here is my script:
#!/bin/bash
temp=$(cat)
sed 's/the/THE/g' <temp
echo "$temp"
Right now, I am just trying to get it to be able to replace all occurences of "the" with "THE".
Here is the sample text:
the quick brown fox jumped over the lazy
brown dog the quick
brown fox jumped
over
Here is my command line:
cat test.txt | ./hwscript >hwscriptout
"test.txt" contains the sample text, "hwscript" is the script, "hwscriptout" is the output
However, when I look at the output file, nothing has changed (all of occurences of "the" remain uncapitalized). When I do the sed command on the command line instead of the script, it works though. I also tried to use $(sed) instead of sed but when I did that, the command returned an error:
"./hwscript: line 5: s/the/THE/g: no such file or directory"
I have tried to search for a solution but could not find one.
Help is appreciated, thank you.
save it to a temp file so that I don't modify the original source
Anything received via stdin is just a stream of data, disconnected from wherever it originated from: whatever you do with that stream has no effect whatsoever on its origin.
Thus, there is no need to involve a temporary file - simply modify stdin input as needed.
#!/bin/bash
sed 's/the/THE/g' # without a filename operand or pipe input, this will read from stdin
# Without an output redirection, the output will go to stdout.
As you can tell, in this simple case you may as well use the sed command directly, without creating a script.
Use this:
temp=$(sed 's/the/THE/' <<<"$temp")
or
temp=$(printf "%s" "$temp" | sed 's/the/THE/')
You were telling sed to process a file named temp, not the contents of the variable $temp. You also weren't saving the result anywhere, so echo "$temp" simply prints the old value
Here is a way to do it as you described it
#!/bin/sh
# Read the input and append to tmp file
while read LINE; do
echo ${LINE} >> yourtmpfile
done
# Edit the file in place
sed -i '' 's/the/THE/g' yourtmpfile
#Output the result
cat yourtmpfile
rm yourtmpfile
And here is a simpler way without a tmp file
#!/bin/sh
# Read the input and output the line after sed
while read LINE; do
echo ${LINE} | sed 's/the/THE/g'
done
Related
I have a file that contains a list of file names to be opened later.
After I load lines (file names) to variables, for a reason unknown to me I cannot open it as a file later.
Here is a simplified example of what i'm trying to do:
Main file's contents:
first_file.txt
second_file.txt
Bash commands:
read line < $main_file # file with the list, received as an argument
echo $line # to check that correct filename has been read
cat $line # attempt to dump "first_file.txt" contents <- FAILS
cat first_file.txt # read "first_file.txt" contents manually
Execution esult:
first_file.txt
: No such file or directory
*** this is 1st file's contents ***
*** ....
So, cat first_file.txt works, $line contains "first_file.txt", but cat $line fails...
I obviously misunderstand something here, suggestions are welcomed!
Edit:
As requested, here is cat -v $main_file's output:
first_file.txt^M
second_file.txt^M
third_file.txt^M
^M
^M
The ^M characters are carriage returns (a.k.a. \r) and are often part of a Windows line ending. They don't show up when you echo them, but they are messing up your ability to open a file with the text having it at the end.
The best solution is to remove them from your "main file." You could use the dos2unix tool if you have it, or you could use GNU sed like sed -i -e 's/\s+$//g' $main_file to edit it in place and remove the extra white space (which includes ^M) from the end of each line.
I want to get a fifo's content and print it in a file, and I have this code:
path=$1 #path file get from script's input
if [ -p "$path" ];then #check if path is pipe
content = 'cat "$path"'
echo "$content" > output
exit 33
fi
My problem is that when I execute the cat "$path" line the script is stopped and the terminal displays the underscore.
I don't know how to solve this problem
P.S the fifo isn't empty and output is the file where I want to print fifo's content
If the FIFO is not empty, and there are no longer any file descriptors writing to that FIFO, you'll get EOF in the cat command. From man 7 pipe:
If all file descriptors referring to the write end of a pipe have been
closed, then an attempt to read(2) from the pipe will see end- of-file
(read(2) will return 0).
Source: man7.org/linux/man-pages/man7/pipe.7.html
Your assignment statement is incorrect.
Whitespace around = is not permitted.
You're confusing single quotes with backquotes. However, you should use $(...) for command substitution anyway.
The correct assignment is
content=$(cat "$path")
or more efficiently in bash,
content=$(< "$path")
I'm trying to do the following :
cat > somefile "some text" <ctrl+d>; clear; some other program
but without having to press
<"ctrl + d">
so that line will create the file and then run some other program. I tried echo "some text" > somefile; but there are too many special chars for that. Anyone know a way around this
I think what you may be looking for is something along these lines:
pax> cat >tempfile ; tput clear ; someprog
Enter your data here for tempfile
<ctrl-d>
**screen clears and someprog runs**
The end-file CTRL-D isn't part of the command you enter, it's part of the input stream for the cat command.
If you don't want to use the input stream, you're either going to have to work out the echo variant (learn to embrace the wonderful world of shell escapes - you can get around most of them by just using single quotes instead of double ones), or simply create your input file in advance and use something like:
cp sourcefile tempfile ; tput clear ; someprog
If you wish to write some text in somefile in multiple lines or with special characters, you should try this. EOF is treated as a special string here that will terminate cat automatically when found but it could be anything else.
cat > somefile << EOF
some text
EOF
some other program
I had a bash file which prompted the user for some parameters and used defaults if nothing was given. The script then went on to perform some other commands with the parameters.
This worked great - no problems until most recent addition.
In an attempt to read the NAMES parameter from a txt file, I've added a while-loop to take in the names in the file, but I would still like the remaining parameters prompted for.
But once I added the while loop, the output shows the printed prompt in get_ans() and never pauses for a read, thus all the defaults are selected.
I would like to read the first parameter from a file, then all subsequent files from prompting the user.
What did I break by adding the while-loop?
cat list.txt |
while read line
do
get_ans "Name" "$line"
read NAME < $tmp_file
get_ans "Name" "$line"
read NAME < $tmp_file
done
function get_ans
{
if [ -f $tmp_file ]; then
rm $tmp_file
PROMPT=$1
DEFAULT=$2
echo -n "$PROMPT [$DEFAULT]: "
read ans
if [ -z "$ans" ]; then
ans="$DEFAULT"
fi
echo "$ans" > $tmp_file
}
(NOTE: Code is not copy&paste so please excuse typos. Actual code has function defined before the main())
You pipe data into your the while loops STDIN. So the read in get_ans is also taking data from that STDIN stream.
You can pipe data into while on a different file descriptor to avoid the issue and stop bothering with temp files:
while read -u 9 line; do
NAME=$(get_ans Name "$line")
done 9< list.txt
get_ans() {
local PROMPT=$1 DEFAULT=$2 ans
read -p "$PROMPT [$DEFAULT]: " ans
echo "${ans:-$DEFAULT}"
}
To read directly from the terminal, not from stdin (assuming you're on a *NIX machine, not a Windows machine):
while read foo</some/file; do
read bar</dev/tty
echo "got <$bar>"
done
When you pipe one command into another on the command line, like:
$ foo | bar
The shell is going to set it up so that bar's standard input comes from foo's standard output. Anything that foo sends to stdout will go directly to bar's stdin.
In your case, this means that the only thing that your script can read from is the standard output of the cat command, which will contain the contents of your file.
Instead of using a pipe on the command line, make the filename be the first parameter of your script. Then open and read from the file inside your code and read from the user as normal.
I am using bash with linux to accomplish adding content to the top of a file.
Thus far i know that i am able to get this done by using a temporary file. so
i am doing it this way:
tac lines.bar > lines.foo
echo "a" >> lines.foo
tac lines.foo > lines.bar
But is there a better way of doing this without having to write a second file?
echo a | cat - file1 > file2
same as shellter's
and sed in one line.
sed -i -e '1 i<whatever>' file1
this will insert to file1 inplace.
the sed example i referred to
tac is very 'expensive' solution, especially as you need to use it 2x. While you still need to use a tmp file, this will take less time:
edit per notes from KeithThompson, now using '.$$' filename and condtional /bin/mv.
{
echo "a"
cat file1
} > file1.$$ && /bin/mv file1.$$ file1
I hope this helps
Using a named pipe and in place replacement with sed, you could add the output of a command at the top of a file without explicitly needing a temporary file:
mkfifo output
your_command >> output &
sed -i -e '1x' -e '1routput' -e '1d' -e '2{H;x}' file
rm output
What this does is buffering the output of your_command in a named pipe (fifo), and inserts in place this output using the r command of sed. For that, you need to start your_command in the background to avoid blocking on output in the fifo.
Note that the r command output the file at the end of the cycle, so we need to buffer the 1st line of file in the hold space, outputting it with the 2nd line.
I write without explicitly needing a temporary file as sed might use one for itself.