ffmpeg generate m3u8 from mp4 (Resume option) - multithreading

I have a mp4 file or other file (non mp4 format) and i need generate ts files and m3u8 playlist.
I am using this command and works fine:
ffmpeg -i foo.mp4 -codec copy -vbsf h264_mp4toannexb -map 0 -f segment
-segment_list out.m3u8 -segment_time 10 out%03d.ts
Now I need to generate many ts simultaneous so i need a "resume option".
Please see the example below:
One thread (first 20 seconds (0-20))
ffmpeg -i foo.mp4 -codec copy -vbsf h264_mp4toannexb -map 0 -f segment
-segment_list out.m3u8 -segment_time 10 out%03d.ts
Seconds thread (20 seconds to 40 seconds)
ffmpeg -i foo.mp4 ......
Third thread (40 seconds to 60 seconds)
ffmpeg -i foo.mp4 ......
I have lots of core processor to do this jobs.
In resume i need generate .ts files and m3u8 fastest way possible
I need help or advises to resolve my problem.
Prove of concept i build a litle script that use -ss and -t option:
<?php
//generate all the commands using -ss and -t <seconds>
$startTime = new DateTime("00:00:00");
for ($i=1; $i < 20; $i++) {
$data = $startTime->format('H:i:s');
$exec = 'ffmpeg -i "<FILE>" -ss '.$data.' -t 10 -c:v copy -bsf h264_mp4toannexb -flags -global_header -map 0 -f segment -segment_time 10 -segment_start_number '.$i.' -segment_list '.sprintf("%04d", $i).'_test.m3u8 -segment_format mpegts '.$i.'stream%05d.ts';
shell_exec($exec);
$startTime->modify('+10 seconds');
echo "\n";
}
//cycle all m3u8 and creates a master hls playlist
$files = glob('*.{m3u8}', GLOB_BRACE);
sort($files);
$ret = "#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:11
";
foreach($files as $file) {
$ret .= shell_exec('sed -n -e 6,7p '. $file);
}
$ret .= "#EXT-X-ENDLIST";
file_put_contents('final.m3u8', $ret);
?>
Thanks

Related

ffmpeg messes up variables [duplicate]

This question already has answers here:
Bash script stops execution of ffmpeg in while loop - why?
(3 answers)
Execute "ffmpeg" command in a loop [duplicate]
(3 answers)
Closed 7 days ago.
I am trying to split audio files by their chapters. I have downloaded this as audio with yt-dlp with its chapters on. I have tried this very simple script to do the job:
#!/bin/sh
ffmpeg -loglevel 0 -i "$1" -f ffmetadata meta # take the metadata and output it to the file meta
cat meta | grep "END" | awk -F"=" '{print $2}' | awk -F"007000000" '{print $1}' > ends #
cat meta | grep "title=" | awk -F"=" '{print $2}' | cut -c4- > titles
from="0"
count=1
while IFS= read -r to; do
title=$(head -$count titles | tail -1)
ffmpeg -loglevel 0 -i "$1" -ss $from -to $to -c copy "$title".webm
echo $from $to
count=$(( $count+1 ))
from=$to
done < ends
You see that I echo out $from and $to because I noticed they are just wrong. Why is this? When I comment out the ffmpeg command in the while loop, the variables $from and $to turn out to be correct, but when it is uncommented they just become some stupid numbers.
Commented output:
0 465
465 770
770 890
890 1208
1208 1554
1554 1793
1793 2249
2249 2681
2681 2952
2952 3493
3493 3797
3797 3998
3998 4246
4246 4585
4585 5235
5235 5375
5375 5796
5796 6368
6368 6696
6696 6961
Uncommented output:
0 465
465 70
70 890
890 08
08 1554
1554 3
3 2249
2249
2952
2952 3493
3493
3998
3998 4246
4246 5235
5235 796
796 6368
6368
I tried lots of other stuff thinking that they might be the problem but they didn't change anything. One I remember is I tried havin $from and $to in the form of %H:%M:%S which, again, gave the same result.
Thanks in advance.
Here is an untested refactoring; hopefully it can at least help steer you in another direction.
Avoid temporary files.
Avoid reading the second input file repeatedly inside the loop.
Refactor the complex Awk scripts into a single script.
To be on the safe side, add a redirection from /dev/null to prevent ffmpeg from eating the input data.
#!/bin/sh
from=0
ffmpeg -loglevel 0 -i "$1" -f ffmetadata - |
awk -F '=' '/END/ { s=$2; sub(/007000000.*/, "", s); end[++i] = s }
/title=/ { t=$2; sub(/^([^-]-){3}/, "", t); title[++j] = t }
END { for(n=1; n<=i; n++) print end[n]; print title[n] }' |
while IFS="" read -r end; do
IFS="" read -r title
ffmpeg -loglevel 0 -i "$1" -ss "$from" -to "$end" -c copy "$title".webm </dev/null
from="$end"
done
The Awk script reads all the data into memory, and then prints one "end" marker followed by the corresponding title on the next line; I can't be sure what your ffmpeg -f ffmetadata command outputs, so I just blindly refactored what your scripts seemed to be doing. If the output is somewhat structured you can probably read one record at a time.

ffmpeg add music to video playing after video

I'm using ffmpeg to create slideshow of images.
I have to add music in the backgroup which will be played to the size of the slideshow (slicing audio if audio length is greater than video or repeating audio if audio length is shorter than video)
This is what my script is
ffmpeg -y \
-loop 1 -i in1.png \
-loop 1 -i in2.png \
-loop 1 -i in3.png \
-loop 1 -i in4.png \
-loop 1 -i in5.png \
-filter_complex \
"[0:v]trim=duration=6,fade=t=in:st=0:d=1,fade=t=out:st=5:d=1,setsar=1:1[v0]; \
[1:v]trim=duration=6,fade=t=in:st=0:d=1,fade=t=out:st=5:d=1,setsar=1:1[v1]; \
[2:v]trim=duration=6,fade=t=in:st=0:d=1,fade=t=out:st=5:d=1,setsar=1:1[v2]; \
[3:v]trim=duration=6,fade=t=in:st=0:d=1,fade=t=out:st=5:d=1,setsar=1:1[v3]; \
[4:v]trim=duration=6,fade=t=in:st=0:d=1,fade=t=out:st=5:d=1,setsar=1:1[v4]; \
[v0][v1][v2][v3][v4]concat=n=5:v=1:a=0,setsar=1:1[v]" -i music.mp3 -shortest -map "[v]" -aspect 16:9 -r 24 shortSlideshow1234.mp4;
This generates output, but slideshow is silent and there is no music in the video.
You need to also map the audio:
ffmpeg -y \
-loop 1 -framerate 24 -i in1.png \
-loop 1 -framerate 24 -i in2.png \
-loop 1 -framerate 24 -i in3.png \
-loop 1 -framerate 24 -i in4.png \
-loop 1 -framerate 24 -i in5.png \
-i music.mp3 \
-filter_complex \
"[0:v]trim=duration=6,fade=t=in:st=0:d=1,fade=t=out:st=5:d=1,setsar=1[v0]; \
[1:v]trim=duration=6,fade=t=in:st=0:d=1,fade=t=out:st=5:d=1,setsar=1[v1]; \
[2:v]trim=duration=6,fade=t=in:st=0:d=1,fade=t=out:st=5:d=1,setsar=1[v2]; \
[3:v]trim=duration=6,fade=t=in:st=0:d=1,fade=t=out:st=5:d=1,setsar=1[v3]; \
[4:v]trim=duration=6,fade=t=in:st=0:d=1,fade=t=out:st=5:d=1,setsar=1[v4]; \
[v0][v1][v2][v3][v4]concat=n=5:v=1:a=0[v]" -map "[v]" -map 5:a shortSlideshow1234.mp4

ffmpeg not working in script - moov atom not found

I made a simple script that divides a flv file into multiple parts, converts them all to .mp4 individually and then merge all of them to form a final mp4 file. I did this to save time and convert large files in parallel.
However, I am stuck because the command that normally runs on command line for ffmpeg, doesn't run via script.
I am kind of stuck here and will like to have some assistance.
#!/bin/bash
#sleep 5
filenametmp=$1;
filename=`echo "$filenametmp" | awk '{split($0,a,"."); print a[1]}'`
echo $filename
output="$filename-output"
filenamewithoutpath=`echo "$output" | awk '{split($0,a,"/"); print a[4]}'`
echo $output $filenamewithoutpath
/usr/bin/ffmpeg -i $filenametmp -c copy -map 0 -segment_time $2 -f segment $output%01d.flv
#sleep 10
#echo "/bin/ls -lrt /root/storage/ | /bin/grep $filenamewithoutpath | /usr/bin/wc -l"
filecounttmp=`/bin/ls -lrt /opt/storage/ | /bin/grep $filenamewithoutpath | /usr/bin/wc -l`
filecount=`expr $filecounttmp - 1`
echo $filecount
for i in `seq 0 $filecount`
do
suffix=`expr 0000 + $i`
filenametoconvert="$output$suffix.flv"
convertedfilename="$output$suffix.mp4"
echo $filenametoconvert
/usr/bin/ffmpeg -i $filenametoconvert -c:v libx264 -crf 23 -preset medium -vsync 1 -r 25 -c:a aac -strict -2 -b:a 64k -ar 44100 -ac 1 $convertedfilename > /dev/null 2>&1 &
done
sleep 5
concatstring=""
for j in `seq 0 $filecount`
do
suffix=`expr 0000 + $j`
convertedfilenamemp4="$output$suffix.mp4"
#concatstring=`concat:$concatstring|$convertedfilenamemp4`
echo "file" $convertedfilenamemp4 >> $filename.txt
#ffmpeg -i concat:"$concatstring" -codec copy $filename.mp4
#ffmpeg -f concat -i $filename.txt -c copy $filename.mp4
done
echo $concatstring
ffmpeg -f concat -i $filename.txt -c copy $filename.mp4
rm $output*
rm $filename.txt
I run any flv file like this :
./ff.sh /opt/storage/tttttssssssssss_573f5b1cd473202daf2bf694.flv 20
I get this error message :
moov atom not found
I am on Ubuntu 14.04 LTS version, standard installation of ffmpeg.

Bash Shell syntax error done < $vid'

i have a error
test: line 16: syntax error near unexpected token `done'
test: line 16: ` done < $vid'
my code test
read -p "entrer Liens Video ou le nom d'un fichier contenant des liens:" vid
#Download Video
if [[ -e $vid ]] ; then
while read line
do
python vid-xml-decoder/ultimate.py $line
find . -name "*.flv" | while read line
do
name="$(basename "${line}" .flv)"
#Variable de déplacement de fichier
repExport="./export/"
#mux des fichier
ffmpeg -i "${name}.flv" -vcodec copy -acodec copy mkvtemp.mkv
mkvmerge -v -o "${repExport}${name}.mkv" --default-track 0 --language 0:fre "${name}.ass" mkvtemp.mkv && \
done < $vid
else
python vid-xml-decoder/ultimate.py $vid
# rest of video processing steps
find . -name "*.flv" | while read line
do
name="$(basename "${line}" .flv)"
#Variable de déplacement de fichier
repExport="./export/"
#mux des fichier
ffmpeg -i "${name}.flv" -vcodec copy -acodec copy mkvtemp.mkv
mkvmerge -v -o "${repExport}${name}.mkv" --default-track 0 --language 0:fre "${name}.ass" mkvtemp.mkv && \
fi
in my script I want to offer two possible entering the links manually or with the text file in the same code
exemple
www.website.me/348744jnm
www.website.me/43545ljkjjk
www.website.me/554erer5cf
I have 3 links in my text file and the script will download then mux switch to another mux download links etc
Note: The script must accept space and special character
if you want more information please do not hesitate :)
tx for your help
The error source:
mkvmerge -v -o ..... nguage 0:fre "${name}.ass" mkvtemp.mkv && \
# HERE -------------------------------------------------> ^^
# probably missing one line before "done"
done < $vid
Comments:
easier to debug and maintenace if you start using functions. Break your code to more maintenable parts... You can do anything with functinos as with external commands, like: funcname | while read .. or redirect its output like funcname > somehere and so on..
double quote variables

Improve avconv load and speed?

When I convert video through avconv it's take above 95% percentage, is there any way to reduce the converting time?
Try using -threads auto or push it to the number of cores / threads your CPU has.
Here is my full script
#!/bin/sh
infile=$1
tmpfile="$1-tmp.mp4"
outfile="$1-new.mp4"
options="-vcodec libx264 -b 512k -flags +loop+mv4 -cmp 256 \
-partitions +parti4x4+parti8x8+partp4x4+partp8x8+partb8x8 \
-me_method hex -subq 7 -trellis 1 -refs 5 -bf 3 \
-flags2 +bpyramid+wpred+mixed_refs+dct8x8 -coder 1 -me_range 16 \
-g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -qmin 10\
-qmax 51 -qdiff 4"
# Half size
options=$options" -vf scale=iw*0.5:-1"
# Copy audio
options=$options" -codec:a copy"
# Shut up
options=$options" -loglevel info "
# echo "Options : $options"
avconv -y -i "$infile" -threads auto $options "$outfile"
# avconv -y -i "$infile" -an -pass 1 -threads auto $options "$tmpfile"
# avconv -y -i "$infile" -acodec aac -strict experimental -ar 44100 -ab 96k -pass 2 -threads auto $options "$tmpfile"
# qt-faststart "$tmpfile" "$outfile"

Resources