#!/bin/bash
cat input.txt | while read ips
do
cmd="$(snmpwalk -v2c -c abc#123 $ips sysUpTimeInstance)"
echo "$ips ---> $cmd"
echo "$ips $cmd" >> out_uptime.txt
done
How can i add threading to this bash script, i have around 80000 input and it takes lot of time?
Simple method. Assuming the order of the output is unimportant, and that snmpwalk's output is of no interest if it should fail, put a && at the end of each of the commands to background, except the last command which should have a & at the end:
#!/bin/bash
while read ips
do
cmd="$(nice snmpwalk -v2c -c abc#123 $ips sysUpTimeInstance)" &&
echo "$ips ---> $cmd" &&
echo "$ips $cmd" >> out_uptime.txt &
done < input.txt
Less simple. If snmpwalk can fail, and that output is also needed, lose the && and surround the code with curly braces,{}, followed by &. To redirect the appended output to include standard error use &>>:
#!/bin/bash
while read ips
do {
cmd="$(nice snmpwalk -v2c -c abc#123 $ips sysUpTimeInstance)"
echo "$ips ---> $cmd"
echo "$ips $cmd" &>> out_uptime.txt
} &
done < input.txt
The braces can contain more complex if ... then ... else ... fi statements, all of which would be backgrounded.
For those who don't have a complex snmpwalk command to test, here's a similar loop, which prints one through five but sleeps for random durations between echo commands:
for f in {1..5}; do
RANDOM=$f &&
sleep $((RANDOM/6000)) &&
echo $f &
done 2> /dev/null | cat
Output will be the same every time, (remove the RANDOM=$f && for varying output), and requires three seconds to run:
2
4
1
3
5
Compare that to code without the &&s and &:
for f in {1..5}; do
RANDOM=$f
sleep $((RANDOM/6000))
echo $f
done 2> /dev/null | cat
When run, the code requires seven seconds to run, with this output:
1
2
3
4
5
You can send tasks to the background by &. If you intend to wait for all of them to finish you can use the wait command:
process_to_background &
echo Processing ...
wait
echo Done
You can get the pid of the given task started in the background if you want to wait for one (or few) specific tasks.
important_process_to_background &
important_pid=$!
while i in {1..10}; do
less_important_process_to_background $i &
done
wait $important_pid
echo Important task finished
wait
echo All tasks finished
On note though: the background processes can mess up the output as they will run asynchronously. You might want to use a named pipe to collect the output from them.
Related
I have a container, it starts with shell script.sh such as:
FROM bash:4.4
COPY script.sh /
COPY process.sh /
CMD ["bash", "/script.sh"]
Here is script.sh:
#!/bin/sh
sh process.sh &
for i in {1..10000}
do
sleep 1
echo "Looping ... number $i"
done
It starts another process by running process.sh script.
Here is the process.sh script:
#!/bin/sh
for i in {1..10}
do
sleep 1
echo "I am from child process ... number $i"
done
Now I want to see all the stdout message. If I go to the directory like /var/lib/docker/containers/container_sha:
I see something like below:
I am from child process ... number {1..10}
Looping ... number 1
Looping ... number 2
Looping ... number 3
Looping ... number 4
Looping ... number 5
Looping ... number 6
.....
It is obvious that, I see only the script.sh output but not process.sh
Why is that? And how can i get all the logs?
Note: docker logs containerName does the same.
{1..10} is bash syntax, and does not expand to anything in sh. So the loop runs once, with the word {1..10} (literally).
You can run process.sh with bash instead of sh
Or if you want/need sh, you could either:
Use a counter:
while c=$((c+1)); [ "$c" -le 10 ]; do
Use a program like seq (not POSIX):
for i in $(seq 10); do
Iterate arguments passed from bash like:
sh process.sh {1..10} &
and in process.sh:
for i do
I am trying to write a small shell script, make it go to sleep for some amount of time like 20 seconds and then run it. Now if i open another terminal and try to run the same script, it shouldn't run as the process is running else where. How do I do it?
I know i should write something, make it go to sleep captures its pid and write a condition that if this pis is running somewhere then don't let it run anywhere. but how do i do it? Please give a code.
echo "this is a process"
sleep 60
testfilepid = `ps ax | grep test1.sh | grep -v grep | tr -s " " | cut -f1 -d " "| tail -1`
echo $testfilepid
if [[ $tesfilepid = " " ]]
sh test1.sh
else
echo "this process is already running"
fi
This is what I tried. when i execute this in 2 windows, both the windows give me the output this is a process.
You could use pgrep to check that your script/process is running, and negate the output, this is a very basic example that could give you an idea:
if ! pgrep -f sleep >/dev/null; then echo "will sleep" && sleep 3; fi
Notice the !, pgrep -f sleep will search for a process matching against full argument lists. (you could customize this to your needs). so if nothing matches your pattern then your script will be called.
I am trying to run two scripts in the background. However I would like to have one script run first, wait for it to finish and run the next script recursively. Will this code snippet do as such:
for i in "${studyinstanceuids[#]}"
do
#let count="$count+1"
echo "$i" | ./cmd2&
sleep 5
if job1 is alive then sleep 5
echo "$i" | ./sendExamToRepo.sh&
wait
fi
for i in "${studyinstanceuids[#]}"; do
( echo "$i" | ./cmd2; echo "$1" | ./sendExamToRepo.sh )&
done
wait
Is there any way I could run several echo statements one after the other with a delay?
For example:
The first statement will be:
echo Hello1
after 1/2 second, run the Second echo statement:
echo Hello2
Likewise, is it possible to run several statements one after the other with a time delay without printing all echoes at once?
Perhaps you would like to use sleep <number of seconds>
Like sleep 60 to wait for a minute.
eg. run from commandline
$ echo 'hello1'; sleep 2; echo 'hello2'
or in a bash script file (myscript.sh)
#!/bin/bash
echo 'hello1'
sleep 2
echo 'hello2 after 2 seconds'
sleep 2
echo 'hello3 after 2 seconds'
echo Hello1
usleep 500000 # sleep 500,000 microseconds
echo Hello2
The usleep(1) command is part of the initscripts package on Fedora.
for i in `echo "hello1 hello2 hello3"`; do echo $i; sleep 2; done
I have an interesting situation I am trying to script. I have a program that outputs 26,000 lines after 10 seconds when it starts successfully. Otherwise I have to kill it and start it again. I tried doing something like this:
test $(./long_program | wc -l) -eq 26000 && echo "Started successfully"
but that only works if the program finishes running. Is there a clever way to watch the output stream of a command and make decisions accordingly? I'm at a loss, not quite sure even how to start searching for this. Thanks!
What about
./long_program > mylogfile &
pid=$!
sleep 10
# then test on mylogfile length and kill $pid if needed
count=0
until [ $count -eq 26000 ]; do
killall ./longrun
#start in background
./longrun >output.$$ &
sleep 10
count=$(wc -l output.$$ |awk '{print $1}')
done
echo "done"
#disown so it continues after current login quits
disown -h