I would like to do the follow
while read input
do echo "$input"
sleep 1
done < input.txt | program $1 $2
But run many copies of the program in the background, I was thinking something with a for loop and an &, but that doesn't work well, anyone have know how?
Like so
for (( i=1; i<=3; i++ ))
do
while read input
do echo "$input"
sleep 1
done < input.txt | program $1 $2 &
done
Or would it be better to have a different bash script call this bash script using &?
Related
I have to run as an exercise at my university a bash script to reverse lookup all their DNS entries for a B class network block they own.
This is the fastest I have got but takes forever. Any help optimising this code?
#!/bin/bash
network="a.b"
CMD=/usr/bin/dig
for i in $(seq 1 254); do
for y in $(seq 1 254); do
answer=`$CMD -x $network.$i.$y +short`;
echo $network.$i.$y ' resolves to ' $answer >> hosts_a_b.txt;
done
done
Using GNU xargs to run 64 processes at a time might look like:
#!/usr/bin/env bash
lookupArgs() {
for arg; do
# echo entire line together to ensure atomicity
echo "$arg resolves to $(dig -x "$arg" +short)"
done
}
export -f lookupArgs
network="a.b"
for (( x=1; x<=254; x++ )); do
for (( y=1; y<=254; y++ )); do
printf '%s.%s.%s\0' "$network" "$x" "$y"
done
done | xargs -0 -P64 bash -c 'lookupArgs "$#"' _ >hosts_a_b.txt
Note that this doesn't guarantee order of output (and relies on the lookupArgs function doing one write() syscall per result) -- but output is sortable so you should be able to reorder. Otherwise, one could get ordered output (and ensure atomicity of results) by switching to GNU parallel -- a large perl script, vs GNU xargs' small, simple, relatively low-feature implementation.
Please explain to me why the very last echo statement is blank? I expect that XCODE is incremented in the while loop to a value of 1:
#!/bin/bash
OUTPUT="name1 ip ip status" # normally output of another command with multi line output
if [ -z "$OUTPUT" ]
then
echo "Status WARN: No messages from SMcli"
exit $STATE_WARNING
else
echo "$OUTPUT"|while read NAME IP1 IP2 STATUS
do
if [ "$STATUS" != "Optimal" ]
then
echo "CRIT: $NAME - $STATUS"
echo $((++XCODE))
else
echo "OK: $NAME - $STATUS"
fi
done
fi
echo $XCODE
I've tried using the following statement instead of the ++XCODE method
XCODE=`expr $XCODE + 1`
and it too won't print outside of the while statement. I think I'm missing something about variable scope here, but the ol' man page isn't showing it to me.
Because you're piping into the while loop, a sub-shell is created to run the while loop.
Now this child process has its own copy of the environment and can't pass any
variables back to its parent (as in any unix process).
Therefore you'll need to restructure so that you're not piping into the loop.
Alternatively you could run in a function, for example, and echo the value you
want returned from the sub-process.
http://tldp.org/LDP/abs/html/subshells.html#SUBSHELL
The problem is that processes put together with a pipe are executed in subshells (and therefore have their own environment). Whatever happens within the while does not affect anything outside of the pipe.
Your specific example can be solved by rewriting the pipe to
while ... do ... done <<< "$OUTPUT"
or perhaps
while ... do ... done < <(echo "$OUTPUT")
This should work as well (because echo and while are in same subshell):
#!/bin/bash
cat /tmp/randomFile | (while read line
do
LINE="$LINE $line"
done && echo $LINE )
One more option:
#!/bin/bash
cat /some/file | while read line
do
var="abc"
echo $var | xsel -i -p # redirect stdin to the X primary selection
done
var=$(xsel -o -p) # redirect back to stdout
echo $var
EDIT:
Here, xsel is a requirement (install it).
Alternatively, you can use xclip:
xclip -i -selection clipboard
instead of
xsel -i -p
I got around this when I was making my own little du:
ls -l | sed '/total/d ; s/ */\t/g' | cut -f 5 |
( SUM=0; while read SIZE; do SUM=$(($SUM+$SIZE)); done; echo "$(($SUM/1024/1024/1024))GB" )
The point is that I make a subshell with ( ) containing my SUM variable and the while, but I pipe into the whole ( ) instead of into the while itself, which avoids the gotcha.
#!/bin/bash
OUTPUT="name1 ip ip status"
+export XCODE=0;
if [ -z "$OUTPUT" ]
----
echo "CRIT: $NAME - $STATUS"
- echo $((++XCODE))
+ export XCODE=$(( $XCODE + 1 ))
else
echo $XCODE
see if those changes help
Another option is to output the results into a file from the subshell and then read it in the parent shell. something like
#!/bin/bash
EXPORTFILE=/tmp/exportfile${RANDOM}
cat /tmp/randomFile | while read line
do
LINE="$LINE $line"
echo $LINE > $EXPORTFILE
done
LINE=$(cat $EXPORTFILE)
This is my first bash script using some resources I found online. I think there is a better way to write this perhaps using some other form of conditionals (if then vs control operators).
It is a script that basically checks if a host is up or down (check if it is ping-able). You dump all the ip addresses you want into a file, then run the script calling on the file. The text file looks like this:
8.8.8.8
4.8.8.8
4.4.4.4
127.0.0.1
The actual script looks like this. Is the 2>&1 necessary because it worked without. I had to play around with the brackets a lot.
#!/bin/bash
while read line
do
A=$(ping -c 1 $line)
((echo $A | grep "64 bytes") > /dev/null 2>&1 && (echo "UP - "$line)) || echo "DOWN - "$line
done < $1
Thank you!
You can do it entirely without brackets:
while read -r address; do
ping -c 1 $address >/dev/null 2>&1 && echo "UP - $address" || echo "DOWN - $address"
done < file
The >/dev/null 2>&1 redirects both STDOUT and STDERR to /dev/null meaning that whatever ping outputs won't be printed to your terminal.
You can then use the && and || operators to echo a message in case of success (ping exits with 0) or failure (ping exits with >0)
You could use if..then..else if you prefer:
while read -r address; do
if ping -c 1 $address > /dev/null 2>&1; then
echo "UP - $address"
else
echo "DOWN - $address"
fi
done < file
arco444 has the right answer. Some other notes
with the form A && B || C, if A succeeds then B executes; if B fails, C will execute. That will not occur for if A; then B; else C; fi
proper indentation is extremely helpful to identify errors (none here, but in general)
Use better variable names: $ip is more meaningful than $line
variables go inside the quotes
parentheses introduce subshells, which will reduce performance. Use only when necessary
I'm trying to get this script to work for checking the output of a program (test21) against known correct result text files. I've tried playing around with a couple ways of writing it, but I can't get it to work. Any ideas, thanks!
#!/bin/bash
arr=$(find resultfile*)
x=0
for i in $(find resultfile*);do
arr[$x]=$i
x=$(($x + 1))
done
for i in ${arr[#]}; do
echo $i
done
x=0
for i in $(find testfile*);
do
if diff ${arr[$x]} <(./test21< testfile1.txt 2>&1); then
echo Everything is correct
else
echo Everything is wrong
fi
x=$(($x+1))
done
It says for this,I get:
Segmentation fault: 11 ./test21 < "testfile1.txt" 2>&1
My program test21 takes in an input text file and parses it, which could result in the segmentation faults. But if I use ./test21 testfile1.txt my program runs perfectly fine. I hardcoded in testfile1.txt, but it should be
if diff ${arr[$x]} <(./test21< $i 2>&1); then
However, it's reading it as $i, even if I do "$i".
I get:
Segmentation fault: 11 ./test21 < "testfile1.txt" 2>&1
My program test21 takes in an input text file and parses it, which
could result in the segmentation faults. But if I use ./test21 testfile1.txt my program runs perfectly fine.
Then just use that; instead of
./test21< testfile1.txt 2>&1
write
./test21 testfile1.txt 2>&1
Lets say that you want to run a program myprogram which will use input.txt file as input.
You want to compare output of your program with some known correct result which is written in correct.txt file. We will save output of your program in actual.txt file.
#!/bin/bash
./myprogram < input.txt > actual.txt
cmp -s actual.txt correct.txt
if [ $? -eq 1 ]; then
echo "WRONG"
else
echo "CORRECT"
fi
Note that actual.txt will capture only output that came from stdout. If out want to redirect both stdout and stderr you then want to use this command:
./myprogram < input.txt &> actual.txt
If you want to ignore stderr and throw it in "black hole" you will use:
./myprogram < input.txt 2> /dev/null > actual.txt
I have a simple BASH command that looks like
for i in `seq 2`; do echo $i; done; > out.dat
When this runs the output of seq 2 is output to the terminal and nothing is output to the data file (out.dat)
I am expecting standard out to be redirected to out.dat like it does simply running the command seq 2 > out.dat
Remove your semicolon.
for i in `seq 2`; do echo "$i"; done > out.dat
SUGGESTIONS
Also as suggested by Fredrik Pihl, try not to use external binaries when they are not needed, or at least when practically not:
for i in {1..2}; do echo "$i"; done > out.dat
for (( i = 1; i <= 2; ++i )); do echo "$i"; done > out.dat
for i in 1 2; do echo "$i"; done > out.dat
Also, be careful of outputs in words that may cause pathname expansion.
for a in $(echo '*'); do echo "$a"; done
Would show your files instead of just a literal *.
$() is also recommended as a clearer syntax for command substitution in Bash and POSIX shells than backticks (`), and it supports nesting.
The cleaner solutions as well for reading output to variables are
while read var; do
...
done < <(do something)
And
read ... < <(do something) ## Could be done on a loop or with readarray.
for a in "${array[#]}"; do
:
done
Using printf can also be an easier alternative with respect to the intended function:
printf '%s\n' {1..2} > out.dat
Another possibility, for the sake of completeness: You can move the output inside the loop, using >> to append to the file, if it exists.
for i in `seq 2`; do echo $i >> out.dat; done;
Which one is better certainly depends on the use case. Writing the file in one go is certainly better than appending to it a thousand times. Also, if the loop contains multiple echo statements, all of which shall go to the file, doing done > out.dat is probably more readable and easier to maintain. The advantage of this solution, of course, is that it gives more flexibility.
Try:
(for i in `seq 2`; do echo $i; done;) > out.dat