I had a bash file which prompted the user for some parameters and used defaults if nothing was given. The script then went on to perform some other commands with the parameters.
This worked great - no problems until most recent addition.
In an attempt to read the NAMES parameter from a txt file, I've added a while-loop to take in the names in the file, but I would still like the remaining parameters prompted for.
But once I added the while loop, the output shows the printed prompt in get_ans() and never pauses for a read, thus all the defaults are selected.
I would like to read the first parameter from a file, then all subsequent files from prompting the user.
What did I break by adding the while-loop?
cat list.txt |
while read line
do
get_ans "Name" "$line"
read NAME < $tmp_file
get_ans "Name" "$line"
read NAME < $tmp_file
done
function get_ans
{
if [ -f $tmp_file ]; then
rm $tmp_file
PROMPT=$1
DEFAULT=$2
echo -n "$PROMPT [$DEFAULT]: "
read ans
if [ -z "$ans" ]; then
ans="$DEFAULT"
fi
echo "$ans" > $tmp_file
}
(NOTE: Code is not copy&paste so please excuse typos. Actual code has function defined before the main())
You pipe data into your the while loops STDIN. So the read in get_ans is also taking data from that STDIN stream.
You can pipe data into while on a different file descriptor to avoid the issue and stop bothering with temp files:
while read -u 9 line; do
NAME=$(get_ans Name "$line")
done 9< list.txt
get_ans() {
local PROMPT=$1 DEFAULT=$2 ans
read -p "$PROMPT [$DEFAULT]: " ans
echo "${ans:-$DEFAULT}"
}
To read directly from the terminal, not from stdin (assuming you're on a *NIX machine, not a Windows machine):
while read foo</some/file; do
read bar</dev/tty
echo "got <$bar>"
done
When you pipe one command into another on the command line, like:
$ foo | bar
The shell is going to set it up so that bar's standard input comes from foo's standard output. Anything that foo sends to stdout will go directly to bar's stdin.
In your case, this means that the only thing that your script can read from is the standard output of the cat command, which will contain the contents of your file.
Instead of using a pipe on the command line, make the filename be the first parameter of your script. Then open and read from the file inside your code and read from the user as normal.
Related
What "... | tee file" does is take stdin (standard input) and divert it to two places: stdout (standard output) and to a path/file named "file". In effect it does this, as far as I can judge:
#!/bin/bash
var=(cat) # same as var=(cat /dev/stdin)
echo -e "$var"
for file in "$#"
do
echo -e "$var" > "${file}"
done
exit 0
So I use the above code to create tee1 to see if I could emulate what tee does. But my real intent is to write a modified version that appends to existing file(s) rather than redo them from scratch. I call this one tee2:
#!/bin/bash
var=(cat) # same as var=(cat /dev/stdin)
echo -e "$var"
for file in "$#"
do
echo -e "$var" >> "${file}"
done
exit 0
It makes sense to me, but not to bash. Now an alternative approach is to do something like this:
echo -e "$var"
for file in "$#"
do
echo -e "$var"| tee tmpfile
cat tmpfile >> "${file}"
done
rm tmpfile
exit 0
It also makes sense to me to do this:
#!/bin/bash
cp -rfp /dev/stdin tmpfile
cat tmpfile
for file in "$#"
do
cat tmpfile >> "${file}"
done
exit 0
Or this:
#!/bin/bash
cat /dev/stdin
for file in "$#"
do
cat /dev/stdin >> "${file}"
done
exit 0
Some online searches suggest that printf be used in place of echo -e for more consistency across platforms. Other suggest that cat be used in place of read, though since stdin is a device, it should be able to be used in place of catm as in:
> tmpfile
IFS=\n
while read line
do
echo $line >> tmpfile
echo $line
done < /dev/stdin
unset IFS
Then the for loop follows. But I can't get that to work. How can I do it with bash?
But my real intent is to write a modified version that appends to existing file(s) rather than redo them from scratch.
The tee utility is specified to support an -a option, meaning "Append the output to the files." [spec]
(And I'm not aware of any implementations of tee that deviate from the spec in this regard.)
Edited to add: If your question is really "what's wrong with all the different things I tried", then, that's probably too broad for a single Stack Overflow question. But here's a short list:
var=(cat) means "Set the array variable var to contain a single element, namely, the string cat."
Note that this does not, in any way, involve the program cat.
You probably meant var=$(cat), which means "Run the command cat, capturing its standard output. Discard any null bytes, and discard any trailing sequence of newlines. Save the result in the regular variable var."
Note that even this version is not useful for faithfully implementing tee, since tee does not discard null bytes and trailing newlines. Also, tee forwards input as it becomes available, whereas var=$(cat) has to wait until input has completed. (This is a problem if standard input is coming from the terminal — in which case the user would expect to see their input echoed back — or from a program that might be trying to communicate with the user — in which case you'd get a deadlock.)
echo -e "$var" makes a point of processing escape sequences like \t. (That's what the -e means.) This is not what you want. In addition, it appends an extra newline, which isn't what you want if you've managed to set $var correctly. (If you haven't managed to set $var correctly, then this might help compensate for that, but it won't really fix the problem.)
To faithfully print the contents of var, you should write printf %s "$var".
I don't understand why you switched to the | tee tmpfile approach. It doesn't improve anything so far as I can tell, and it introduces the bug that now if you're copying to n files, then you will also write n copies to standard output. (You fixed that bug in later versions, though.)
The versions where you write directly to a file, instead of saving to a variable first, are a massive improvement in terms of faithfully copying the contents of standard input. But they still have the problem of waiting until input is complete.
The version where you cat /dev/stdin multiple times (once for each destination) won't work, because there's no "rewinding" of standard input. Once something is consumed, it's gone. (This makes sense when you consider that standard input is frequently passed around from program to program — your cat-s, for example, are inheriting it from your Bash script, and your Bash script may be inheriting it from the terminal. If some sort of automatic rewinding were to happen, how would it decide how far back to go?) (Note: if standard input is coming from a regular file, then it's possible to explicitly seek backward along it, and thereby "unconsume" already-consumed input. But that doesn't happen automatically, and anyway that's not possible when standard input is coming from a terminal, from a pipe, etc.)
I want to get a fifo's content and print it in a file, and I have this code:
path=$1 #path file get from script's input
if [ -p "$path" ];then #check if path is pipe
content = 'cat "$path"'
echo "$content" > output
exit 33
fi
My problem is that when I execute the cat "$path" line the script is stopped and the terminal displays the underscore.
I don't know how to solve this problem
P.S the fifo isn't empty and output is the file where I want to print fifo's content
If the FIFO is not empty, and there are no longer any file descriptors writing to that FIFO, you'll get EOF in the cat command. From man 7 pipe:
If all file descriptors referring to the write end of a pipe have been
closed, then an attempt to read(2) from the pipe will see end- of-file
(read(2) will return 0).
Source: man7.org/linux/man-pages/man7/pipe.7.html
Your assignment statement is incorrect.
Whitespace around = is not permitted.
You're confusing single quotes with backquotes. However, you should use $(...) for command substitution anyway.
The correct assignment is
content=$(cat "$path")
or more efficiently in bash,
content=$(< "$path")
I have the following functions:
execIn ()
{
local STORE_INvar="${1}" ; shift
printf -v "${STORE_INvar}" '%s' "$( eval "$#" ; printf %s x ; )"
printf -v "${STORE_INvar}" '%s' "${!STORE_INvar%x}"
}
and
getFifo ()
{
local FIFOfile
FIFOfile="/tmp/diamondLang-FIFO-$$-${RANDOM}"
while [ -e "${FIFOfile}" ]
do
FIFOfile="/tmp/diamondLang-FIFO-$$-${RANDOM}"
done
mkfifo "${FIFOfile}"
echo "${FIFOfile}"
}
I want to store the output of the end of a pipeline into a variable as given to a function at the end of the pipeline, however, the only way I have found to do this that will work in early versions of Bash is to use mkfifo to make a temp fifo file. I was hoping to use file descriptors to avoid having to create temporary files. So, This works, but is not ideal:
Set Up: (before I can do this I need to have assigned a FIFO file to a var that can be used by the rest of the process)
$ FIFOfile="$( getFifo )"
The Pipeline I want to persist:
$ printf '\n\n123\n456\n524\n789\n\n\n' | grep 2 # for e.g.
The action: (I can now add) >${FIFOfile} &
$ printf '\n\n123\n456\n524\n789\n\n\n' | grep 2 >${FIFOfile} &
N.B. The need to background it with & - Problem 1: I get [1] <PID_NO> output to the screen.
The actual persist:
$ execIn SOME_VAR cat - <${FIFOfile}
Problem 2: I get more noise to the screen
[1]+ Done printf '\n\n123\n456\n524\n789\n\n\n' | grep 2 > ${FIFOfile}
Problem 3: I loose the blanks at the start of the stream rather than at the end as I have experienced before.
So, am I doing this the right way? I am sure that there must be a way to avoid the need of a FIFO file that needs cleanup afterwards using file descriptors, but I cannot seem to do this as I cannot assign either side of the problem to a file descriptor that is not attached to a file or a FIFO file.
I can try and resolve the problems with what I have, although to make this work properly I guess I need to pre-establish a pool of FIFO files that can be pulled in to use or else I have a pre-req of establishing this file before the command. So, for many reasons this is far from ideal. If anyone can advise me of a better way you would make my day/week/month/life :)
Thanks in advance...
Process substitution was available in bash from the ancient days. You absolutely do not have a version so ancient as to be unable to use it. Thus, there's no need to use a FIFO at all:
readToVar() { IFS= read -r -d '' "$1"; }
readToVar targetVar < <(printf '\n\n123\n456\n524\n789\n\n\n')
You'll observe that:
printf '%q\n' "$targetVar"
...correctly preserves the leading newlines as well as the trailing ones.
By contrast, in a use case where you can't afford to lose stdin:
readToVar() { IFS= read -r -d '' "$1" <"$2"; }
readToVar targetVar <(printf '\n\n123\n456\n524\n789\n\n\n')
If you really want to pipe to this command, are willing to require a very modern bash, and don't mind being incompatible with job control:
set +m # disable job control
shopt -s lastpipe # in a pipeline, parent shell becomes right-hand side
readToVar() { IFS= read -r -d '' "$1"; }
printf '\n\n123\n456\n524\n789\n\n\n' | grep 2 | readToVar targetVar
The issues you claim to run into with using a FIFO do not actually exist. Put this in a script, and run it:
#!/bin/bash
trap 'rm -rf "$tempdir"' 0 # cleanup on exit
tempdir=$(mktemp -d -t fifodir.XXXXXX)
mkfifo "$tempdir/fifo"
printf '\n\n123\n456\n524\n789\n\n\n' >"$tempdir/fifo" &
IFS= read -r -d '' content <"$tempdir/fifo"
printf '%q\n' "$content" # print content to console
You'll notice that, when run in a script, there is no "noise" printed to the screen, because all that status is explicitly tied to job control, which is disabled by default in scripts.
You'll also notice that both leading and tailing newlines are correctly represented.
One idea, tell me I am crazy, might be to use the !! notation to grab the line just executed, e.g. if there is a command that can terminate a pipeline and stop it actually executing, whilst still as far as the shell is concerned, consider it as a successful execution, I am thinking something like the true command, I could then use !! to grab that line and call my existing function to execute it with process substitution or something. I could then wrap this into an alias, something like: alias streamTo=' | true ; LAST_EXEC="!!" ; myNewCommandVariation <<<' which I think could be used something like: $ cmd1 | cmd2 | myNewCommandVariation THE_VAR_NAME_TO_SET and the <<< from the alias would pass the var name to the command as an arg or stdin, either way, the command would be not at the end of a pipeline. How mad is this idea?
Not a full answer but rather a first point: is there some good reason not using mktemp for creating a new file with a random name? As far as I can see, your function called getFifo() doesn't perform much more.
mktemp -u
will give to you a free new name without creating anything; then you can use mkfifo with this name.
I am foxed by the following situation.
I have a file list.txt that I want to run through line by line, in a loop, in bash. A typical line in list.txt has spaces in. The problem is that the loop contains a "read" command. I want to write this loop in bash rather than something like perl. I can't do it :-(
Here's how I would usually write a loop to read from a file line by line:
while read p; do
echo $p
echo "Hit enter for the next one."
read x
done < list.txt
This doesn't work though, because of course "read x" will be reading from list.txt rather than the keyboard.
And this doesn't work either:
for i in `cat list.txt`; do
echo $i
echo "Hit enter for the next one."
read x
done
because the lines in list.txt have spaces in.
I have two proposed solutions, both of which stink:
1) I could edit list.txt, and globally replace all spaces with "THERE_SHOULD_BE_A_SPACE_HERE" . I could then use something like sed, within my loop, to replace THERE_SHOULD_BE_A_SPACE_HERE with a space and I'd be all set. I don't like this for the stupid reason that it will fail if any of the lines in list.txt contain the phrase THERE_SHOULD_BE_A_SPACE_HERE (so malicious users can mess me up).
2) I could use the while loop with stdin and then in each loop I could actually launch e.g. a new terminal, which would be unaffected by the goings-on involving stdin in the original shell. I tried this and I did get it to work, but it was ugly: I want to wrap all this up in a shell script and I don't want that shell script to be randomly opening new windows. What would be nice, and what might somehow be the answer to this question, would be if I could figure out how to somehow invoke a new shell in the command and feed commands to it without feeding stdin to it, but I can't get it to work. For example this doesn't work and I don't really know why:
while read p; do
bash -c "echo $p; echo ""Press enter for the next one.""; read x;";
done < list.txt
This attempt seems to fail because "read x", despite being in a different shell somehow, is still seemingly reading from list.txt. But I feel like I might be close with this one -- who knows.
Help!
You must open as a different file descriptor
while read p <&3; do
echo "$p"
echo 'Hit enter for the next one'
read x
done 3< list.txt
Update: Just ignore the lengthy discussion in the comments below. It has nothing to do with the question or this answer.
I would probably count lines in a file and iterate each of those using eg. sed. It is also possible to read infinitely from stdin by changing while condition to: while true; and exit reading with ctrl+c.
line=0 lines=$(sed -n '$=' in.file)
while [ $line -lt $lines ]
do
let line++
sed -n "${line}p" in.file
echo "Hit enter for the next ${line} of ${lines}."
read -s x
done
AWK is also great tool for this. Simple way to iterate through input would be like:
awk '{ print $0; printf "%s", "Hit enter for the next"; getline < "-" }' file
As an alternative, you can read from stderr, which by default is connected to the tty as well. The following then also includes a test for that assumption:
(
tty -s <& 2|| exit 1
while read -r line; do
echo "$line"
echo 'Hit enter'
read x <& 2
done < file
)
I wrote the following program in my linux bashrc
open()
{
echo enter file name
read fname
locate $fname> /home/vvajendla/Desktop/backup/loc;
cat loc
exec < /home/vvajendla/Desktop/backup/loc;
value=0
while read line
do
value=`expr $value + 1`;
echo $value
echo $line
if [ $value -le 6 ]
then
gedit $line;
else
echo too many files to open
fi
done
}
The above function searches all the directories for the file-string match and opens them using GEDIT if they are less than or equal to 6.
whenever i run this function in the terminal,it gets closed.
Can you please tell me what i can do to keep it open?
The exec causes the standard input of the calling shell to be permanently redirected from the file. Once the file closes, the shell runs out of input, and exits. I assume you import this function with source; running it standalone should work.
The usual way to write this sort of function would be to make it accept an argument, so you would invoke it like "open fnord" instead of run "open" and enter "fnord" at the prompt.
open () {
local fname
fname=$1 # notice this arrangement instead of read
local value
value=0
locate "$fname" | # notice double quotes
tee /dev/stderr | # as a superior alternative to using a temporary file
while read line
do
value=`expr $value + 1`
if [ $value -le 6 ]
then
gedit "$line" # notice double quotes
else
echo too many files to open >&2 # notice redirection to stderr
fi
done
}
The diagnostic is misleading; this code will still open the first six files, then bail with an error message at the seventh. Is that what you intend? Or should it count the number of outputs, and refuse to run if there are more than six?
If you don't care for the other improvements, the minimal fix is to remove the exec and read the while loop's input from your temporary file. (You should take care to properly clean up; if you can avoid a temporary file, that's basically always a better solution.)
while read line; do
....
done <tempfile
I would be tempted to add line numbers with nl to get rid of the unattractive expr, but this might break file names with a space at the beginning. (On the other hand, locate always produces a full path name, right?)
As an alternative, and assuming gedit can read multiple file name arguments, try this:
locate "$fname" | head -n 6 | xargs gedit
This fails to produce a warning if there are more than six files, but I would actually consider that a feature.