execute command in bash script until output exceeds certain value - linux

Question

I use a command which parses video files for certain frames and returning their timecode, when found. At the moment, I have to execute the command, wait, until the values printed to stdout reach the desired position and then abort the execution using Ctrl+C.
As I have to watch the process and to abort the execution in the right moment to get the information I need, I thought, I could automate this to some degree by creating a bash script.
I am not certain, if it can be done in bash, as I don't exactly know, how to abort the execution in connection with the values it writes to stdout.
The output of the command looks like
0.040000
5.040000
10.040000
15.040000
18.060000
(...)
I tried
until [[ "$timecode" -gt 30 ]]; do
timecode=$(mycommand)
sleep 0.1
done
echo "Result: $timecode"
or
while [[ "$timecode" -le 30 ]]; do
timecode=$(mycommand)
sleep 0.1
done
echo "Result: $timecode"
which both seem to result in the command being executed until it finishes and afterwards the rest of the loop is being processed. But I want to evaluate the output while the command executes and break execution depending on the output.
Additional information
The command has no capability to be stopped at a certain point in the stream. It parses the whole file and gives the results unless signalled to stop. This was my first shot.
The execution time of the command is very long as the files I parse are ~2GB. As I don't need all frames of the file but only a few around a given timecode, I never let it execute until it finished.
The output of the command varies from file to file, so I can't look for an exact value. If I knew the exact value, I probably wouldn't have to look for it.
The destination time code - in the example it is specified by "-gt 30" - is different for every file I will have to parse, so I will have to put this into a command line parameter once the script works. I would also have to make sure to get back more than the last value of the execution but about the last 5 values. For these two I already have Ideas.
I'm totally stuck on that one and have not even an idea what to google for.
Thank you for your input!
Manuel
With the answers of PSkocik and Kyle Burton, I was able to integrate the suggested solution into my script. It doesn't work and I don't see, why.
Here the complete script including the external command providing the output:
#!/usr/bin/env bash
set -eu -o pipefail
parser () {
local max="$1"
local max_int
max_int="${max%.*}"
while read tc;
do
local tc_int
tc_int="${tc%.*}"
echo $tc
if (( "$tc_int" >= "$max_int" )); then
echo "Over 30: $tc";
exec 0>&-
return 0
fi
done
}
ffprobe "$1" -hide_banner -select_streams v -show_entries frame=key_frame,best_effort_timestamp_time -of csv=nk=1:p=0:s="|" -v quiet | sed -ne "s/^1|//p" | parser 30
I don't get any output from the "echo $tc" but the ffprobe is running - I can see it in top. It runs until I stop the script using Ctrl+C.
Thank you Kyle for your big efforts in this. I'd never come to such a conclusion. I changed the commandline of ffprobe to your suggestion
ffprobe "$1" -hide_banner -select_streams v -show_entries frame=key_frame,best_effort_timestamp_time -of csv=nk=1:p=0:s="|" -v quiet | cut -f2 -d\| | parser 30
and now, I'm getting results while ffprobe runs. But... the way you changed the command returns all frames, ffprobe finds and not only the Keyframes. The original output of the ffprobe command looks like
1|0.000000
0|0.040000
0|0.080000
0|0.120000
0|0.160000
0|0.200000
(...)
The 0 at the beginning of the line means: this is no keyframe.
The 1 at the beginning of the line means: this is a keyframe.
The script is intended to provide only the keyframes around a certain timecode of the video file. The way you changed the command, it now provides all frames of the video file what makes the resulting output useless. It has to be filtered for all lines starting with zero to be dropped.
As I don't exactly understand, why this doesn't work with sed, I can only try to find a solution by try and error, facilitating different tools to filter the output. But if the filtering itself causes the problem, we might have hit a wall here.

Answer 1

If you have process a that's outputting stuff to stdout and process b that reads the outputted stuff via a pipe:
a | b
all b has to usually do to kill a when a certain item is outputted
is to close its standard input.
A sample b:
b()
{
while read w;
do case $w in some_pattern)exec 0>&-;; esac;
echo $w
done
}
This closing of stdin (filedescriptor 0) will cause the producer process to be killed by SIGPIPE the moment it tries to make its next write.

Answer 2

I think PSkocik's approach makes sense. I think all you need to do is run your mycommand and pipe it into your while loop. If you put PSkocik's code in a file wait-for-max.sh then you should be able to run it as:
mycommand | bash wait-for-max.sh
After working with M. Uster in comments above, we've come up with the following solution:
#!/usr/bin/env bash
set -eu -o pipefail
# echo "bash cutter.sh rn33.mp4"
# From: https://stackoverflow.com/questions/45304233/execute-command-in-bash-script-until-output-exceeds-certain-value
# test -f stack_overflow_q45304233.tar || curl -k -O https://84.19.186.119/stack_overflow_q45304233.tar
# test -f stack_overflow_q45304233.tar || curl -k -O https://84.19.186.119/stack_overflow_q45304233.tar
# test -f rn33.mp4 || curl -k -O https://84.19.186.119/rn33.mp4
function parser () {
local max="$1"
local max_int
# NB: this removes everything after the decimal point
max_int="${max%.*}"
# I added a line number so I could match up the ouptut from this function
# with the output captured by the 'tee' command
local lnum="0"
while read -r tc;
do
lnum="$(( 1 + lnum ))"
# if a blank line is read, just ignore it and continue
if [ -z "$tc" ]; then
continue
fi
local tc_int
# NB: this removes everything after the decimal point
tc_int="${tc%.*}"
echo "Read[$lnum]: $tc"
if (( "$tc_int" >= "$max_int" )); then
echo "Over 30: $tc";
# This closes stdin on this process, which will cause an EOF on the
# process writing to us across the pipe
exec 0>&-
return 0
fi
done
}
# echo "bash version: $BASH_VERSION"
# echo "ffprobe version: $(ffprobe -version | head -n1)"
# echo "sed version: $(sed --version | head -n1)"
# NB: by adding in the 'tee ffprobe.out' into the pipeline I was able to see
# that it was producing lines like:
#
# 0|28.520000
# 1|28.560000
#
#
# changing the sed to look for any single digit and a pipe fixed the script
# another option is to use cut, see below, which is probalby more robust.
# ffprobe "$1" \
# -hide_banner \
# -select_streams v \
# -show_entries frame=key_frame,best_effort_timestamp_time \
# -of csv=nk=1:p=0:s="|" \
# -v quiet 2>&1 | \
# tee ffprobe.out |
# sed -ne "s/^[0-9]|//p" | \
# parser 30
ffprobe "$1" \
-hide_banner \
-select_streams v \
-show_entries frame=key_frame,best_effort_timestamp_time \
-of csv=nk=1:p=0:s="|" \
-v quiet 2>&1 | \
cut -f2 -d\| | \
parser 30

Answer 3

The answer to my question has finally been found by the help of PSkocik and intense support of Kyle Burton. Thanks to both of you!
I didn't know, that it is possible to pipe the output of commands executed in a script to a function that belongs to the script. This was the first piece of information necessary.
And I didn't know, how to evaluate the piped information inside the function properly and how to signal from inside the function, that the execution of the command generating the values should be terminated.
Additionally, Kyle found, that the filtering I did by piping the original output to sed and the resulting data to the function inside the script prohibited the script to function as designed. I'm still uncertain, why - but it definitively does.
The original command generating the output is now being piped as it is to the internal function of the script. The filtering is being done inside the function to avoid the problem with sed. Now everything works as expected and I can continue completing the script.
This is the working code of the soultion:
#!/usr/bin/env bash
set -eu -o pipefail
function parser () {
local max="$1"
local max_int
max_int="${max%.*}"
while read tc;
do
#If line is empty, continue
if [ -z "$tc" ]; then
continue
fi
#If first char is 0 (=non-Index Frame), continue
local iskey="${tc:0:1}";
if [ $iskey == "0" ]; then
continue
fi
#Return timecode if intended maximum has been reached
local val="${tc:2:10}"
local tc_int
tc_int="${val%.*}"
if (( "$tc_int" >= "$max_int" )); then
echo "First index frame at/after given Timecode: $tc";
exec 0>&-
return 0
fi
done
}
ffprobe "$1" -hide_banner -select_streams v -show_entries frame=key_frame,best_effort_timestamp_time -of csv=nk=1:p=0:s="|" -v quiet | parser "$2"
Usage:
./script.sh "Name of Movie.avi" 30
where 30 represents the timecode at which the next found index frame is being searched and returned.