Concat multiple (self-generated) videos using ffmpeg on raspbian linux - linux

I am a very talented sleep talker, so I decided to write a solution that records the things I talk at night to make funny videos with subtitles of it. The project is nearly done, but I got a big problem with concating videos I generated before.
The video parts are generated from single png frames using this command:
ffmpeg -y -framerate 15 -i "${images_file_path}" -c:v libx264 -r 30 -pix_fmt yuv420p "${video_file_path}"
Then the sound is added using this command (got this from #9049970 and #11779490):
ffmpeg -y -i "${video_file_path}" -i "${mp3_file_path}" -map 0:v -map 1:a -vcodec copy -acodec copy -shortest "${final_video_file_path}"
All this is causing no problems so far, but I think it may be relevant to know how the videos are generated. I can watch all this and get valid video and sound - the full source code of this first part can be found here.
Now I added a feature that is able to generate "full videos" containing a title and a various number of previously generated "video parts" using this command:
ffmpeg -f concat -i "${video_list_path}" -filter_complex "${filter_string} concat=n=${input_file_counter}:v=1:a=1 [v] [a]" -map "[v]" -map "[a]" "${full_video_path}"
But something is wrong with it and I get this error:
Invalid file index 1 in filtergraph description [0:v:0] [1:v:0] [2:v:0] [2:a:0] [3:v:0] [4:v:0] [4:a:0] [5:v:0] [6:v:0] [6:a:0] [7:v:0] concat=n=8:v=1:a=1 [v] [a].
The full output is:
ffmpeg version N-77213-g7c1c453 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.9.2 (Raspbian 4.9.2-10)
configuration: --enable-shared --enable-gpl --prefix=/usr --enable-nonfree --enable-libmp3lame --enable-libfaac --enable-libx264 --enable-version3 --disable-mmx
libavutil 55. 10.100 / 55. 10.100
libavcodec 57. 17.100 / 57. 17.100
libavformat 57. 20.100 / 57. 20.100
libavdevice 57. 0.100 / 57. 0.100
libavfilter 6. 20.100 / 6. 20.100
libswscale 4. 0.100 / 4. 0.100
libswresample 2. 0.101 / 2. 0.101
libpostproc 54. 0.100 / 54. 0.100
[mov,mp4,m4a,3gp,3g2,mj2 # 0xc2e900] Auto-inserting h264_mp4toannexb bitstream filter
Input #0, concat, from '/usr/sleeptalk/records-rendered/3enguzpuu2gw0ogk8wkko/videos.txt':
Duration: N/A, start: 0.000000, bitrate: 61 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 58 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 2 kb/s
Metadata:
handler_name : SoundHandler
Invalid file index 1 in filtergraph description [0:v:0] [1:v:0] [2:v:0] [2:a:0] [3:v:0] [4:v:0] [4:a:0] [5:v:0] [6:v:0] [6:a:0] [7:v:0] concat=n=8:v=1:a=1 [v] [a].
I also wrote a test case so you can reproduce this on your local machine. Download the files from my dropbox. Also, the full script that renders the "final move" can be found here.
Would be great to get an Idea, got struggle to fix this the last two days.

You're using both the concat demuxer as well as the concat filter. Skip the latter, because
a) it's unnecessary and
b) I don't believe the demuxer is inducting all input files as separate inputs so the indices beyond 0 don't make sense. Also, the concat filter needs equal number of streams per input file, and their input assignment has to be pair-wise i.e. [0:v:0] [0:a:0] [1:v:0] [1:a:0] [2:v:0] [2:a:0]....
Instead, use
ffmpeg -f concat -i "${video_list_textfile}" -c copy "${full_video_path}"
where ${video_list_textfile} is a text file of the form
file 'file1.mp4'
file 'file2.mp4'
file 'file3.mp4'
...

Related

How do you use FFMPEG to transcode h264_qsv from Apple PRORES Quicktime?

I am trying to transcode an Apple Prores 444 to H.264 using qsv without success.
If I use this command line:
ffmpeg -i 10minute_Pipeline_Test.mov -c:v h264_qsv -c:a aac -pix_fmt qsv chris.mp4
I get:
ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
configuration: --prefix=/root/ffmpeg_build --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --bindir=/root/bin --extra-libs=-ldl --enable-gpl --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-nonfree --enable-libmfx
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Guessed Channel Layout for Input Stream #0.2 : mono
Guessed Channel Layout for Input Stream #0.3 : mono
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '10minute_Pipeline_Test.mov':
Metadata:
major_brand : qt
minor_version : 537134592
compatible_brands: qt
creation_time : 2020-12-19T12:43:38.000000Z
com.apple.quicktime.author:
com.apple.quicktime.comment:
com.apple.quicktime.copyright:
com.apple.quicktime.description:
com.apple.quicktime.director:
com.apple.quicktime.genre:
com.apple.quicktime.information:
com.apple.quicktime.keywords:
com.apple.quicktime.producer:
com.apple.quicktime.displayname:
timecode : 12:43:37;28
Duration: 00:10:06.72, start: 0.000000, bitrate: 167429 kb/s
Stream #0:0(eng): Data: none (tmcd / 0x64636D74)
Metadata:
creation_time : 1970-01-04T00:49:14.000000Z
timecode : 12:43:37;28
Stream #0:1(eng): Video: prores (Standard) (apcn / 0x6E637061), yuv422p10le(tv, GBR, progressive), 1280x720, 164985 kb/s, SAR 1:1 DAR 16:9, 59.94 fps, 59.94 tbr, 60k tbn, 60k tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:04.000000Z
Stream #0:2(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Stream #0:3(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Metadata:
creation_time : 2003-10-05T11:26:56.000000Z
File 'chris.mp4' already exists. Overwrite ? [y/N] y
Stream mapping:
Stream #0:1 -> #0:0 (prores (native) -> h264 (h264_qsv))
Stream #0:2 -> #0:1 (pcm_s24le (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_qsv # 0x56265b81a800] Selected ratecontrol mode is unsupported
[h264_qsv # 0x56265b81a800] Low power mode is unsupported
[h264_qsv # 0x56265b81a800] Current frame rate is unsupported
[h264_qsv # 0x56265b81a800] Current picture structure is unsupported
[h264_qsv # 0x56265b81a800] Current resolution is unsupported
[h264_qsv # 0x56265b81a800] Current pixel format is unsupported
[h264_qsv # 0x56265b81a800] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!
user#NUC:~$ ffmpeg -i 10minute_Pipeline_Test.mov -c:v h264_qsv -c:a aac -pix_fmt qsv chris.mp4
ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
configuration: --prefix=/root/ffmpeg_build --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --bindir=/root/bin --extra-libs=-ldl --enable-gpl --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-nonfree --enable-libmfx
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Guessed Channel Layout for Input Stream #0.2 : mono
Guessed Channel Layout for Input Stream #0.3 : mono
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '10minute_Pipeline_Test.mov':
Metadata:
major_brand : qt
minor_version : 537134592
compatible_brands: qt
creation_time : 2020-12-19T12:43:38.000000Z
com.apple.quicktime.author:
com.apple.quicktime.comment:
com.apple.quicktime.copyright:
com.apple.quicktime.description:
com.apple.quicktime.director:
com.apple.quicktime.genre:
com.apple.quicktime.information:
com.apple.quicktime.keywords:
com.apple.quicktime.producer:
com.apple.quicktime.displayname:
timecode : 12:43:37;28
Duration: 00:10:06.72, start: 0.000000, bitrate: 167429 kb/s
Stream #0:0(eng): Data: none (tmcd / 0x64636D74)
Metadata:
creation_time : 1970-01-04T00:49:14.000000Z
timecode : 12:43:37;28
Stream #0:1(eng): Video: prores (Standard) (apcn / 0x6E637061), yuv422p10le(tv, GBR, progressive), 1280x720, 164985 kb/s, SAR 1:1 DAR 16:9, 59.94 fps, 59.94 tbr, 60k tbn, 60k tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:04.000000Z
Stream #0:2(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Stream #0:3(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Metadata:
creation_time : 2003-10-05T11:26:56.000000Z
File 'chris.mp4' already exists. Overwrite ? [y/N] y
Stream mapping:
Stream #0:1 -> #0:0 (prores (native) -> h264 (h264_qsv))
Stream #0:2 -> #0:1 (pcm_s24le (native) -> aac (native))
Press [q] to stop, [?] for help
Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:1
Conversion failed!
If I use:
ffmpeg -i 10minute_Pipeline_Test.mov -c:v h264_qsv -c:a aac chris.mp4
I get:
ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
configuration: --prefix=/root/ffmpeg_build --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --bindir=/root/bin --extra-libs=-ldl --enable-gpl --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-nonfree --enable-libmfx
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Guessed Channel Layout for Input Stream #0.2 : mono
Guessed Channel Layout for Input Stream #0.3 : mono
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '10minute_Pipeline_Test.mov':
Metadata:
major_brand : qt
minor_version : 537134592
compatible_brands: qt
creation_time : 2020-12-19T12:43:38.000000Z
com.apple.quicktime.author:
com.apple.quicktime.comment:
com.apple.quicktime.copyright:
com.apple.quicktime.description:
com.apple.quicktime.director:
com.apple.quicktime.genre:
com.apple.quicktime.information:
com.apple.quicktime.keywords:
com.apple.quicktime.producer:
com.apple.quicktime.displayname:
timecode : 12:43:37;28
Duration: 00:10:06.72, start: 0.000000, bitrate: 167429 kb/s
Stream #0:0(eng): Data: none (tmcd / 0x64636D74)
Metadata:
creation_time : 1970-01-04T00:49:14.000000Z
timecode : 12:43:37;28
Stream #0:1(eng): Video: prores (Standard) (apcn / 0x6E637061), yuv422p10le(tv, GBR, progressive), 1280x720, 164985 kb/s, SAR 1:1 DAR 16:9, 59.94 fps, 59.94 tbr, 60k tbn, 60k tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:04.000000Z
Stream #0:2(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Stream #0:3(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Metadata:
creation_time : 2003-10-05T11:26:56.000000Z
File 'chris.mp4' already exists. Overwrite ? [y/N] y
Stream mapping:
Stream #0:1 -> #0:0 (prores (native) -> h264 (h264_qsv))
Stream #0:2 -> #0:1 (pcm_s24le (native) -> aac (native))
Press [q] to stop, [?] for help
Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:1
Conversion failed!
user#NUC:~$ ffmpeg -i 10minute_Pipeline_Test.mov -c:v h264_qsv -c:a aac chris.mp4
ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
configuration: --prefix=/root/ffmpeg_build --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --bindir=/root/bin --extra-libs=-ldl --enable-gpl --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-nonfree --enable-libmfx
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Guessed Channel Layout for Input Stream #0.2 : mono
Guessed Channel Layout for Input Stream #0.3 : mono
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '10minute_Pipeline_Test.mov':
Metadata:
major_brand : qt
minor_version : 537134592
compatible_brands: qt
creation_time : 2020-12-19T12:43:38.000000Z
com.apple.quicktime.author:
com.apple.quicktime.comment:
com.apple.quicktime.copyright:
com.apple.quicktime.description:
com.apple.quicktime.director:
com.apple.quicktime.genre:
com.apple.quicktime.information:
com.apple.quicktime.keywords:
com.apple.quicktime.producer:
com.apple.quicktime.displayname:
timecode : 12:43:37;28
Duration: 00:10:06.72, start: 0.000000, bitrate: 167429 kb/s
Stream #0:0(eng): Data: none (tmcd / 0x64636D74)
Metadata:
creation_time : 1970-01-04T00:49:14.000000Z
timecode : 12:43:37;28
Stream #0:1(eng): Video: prores (Standard) (apcn / 0x6E637061), yuv422p10le(tv, GBR, progressive), 1280x720, 164985 kb/s, SAR 1:1 DAR 16:9, 59.94 fps, 59.94 tbr, 60k tbn, 60k tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:04.000000Z
Stream #0:2(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Stream #0:3(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Metadata:
creation_time : 2003-10-05T11:26:56.000000Z
File 'chris.mp4' already exists. Overwrite ? [y/N] y
Stream mapping:
Stream #0:1 -> #0:0 (prores (native) -> h264 (h264_qsv))
Stream #0:2 -> #0:1 (pcm_s24le (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_qsv # 0x55b3bb6e8800] Selected ratecontrol mode is unsupported
[h264_qsv # 0x55b3bb6e8800] Low power mode is unsupported
[h264_qsv # 0x55b3bb6e8800] Current frame rate is unsupported
[h264_qsv # 0x55b3bb6e8800] Current picture structure is unsupported
[h264_qsv # 0x55b3bb6e8800] Current resolution is unsupported
[h264_qsv # 0x55b3bb6e8800] Current pixel format is unsupported
[h264_qsv # 0x55b3bb6e8800] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!
I cannot get ANYTHING to work. I can transcode other h264 files without issue. I cannot seem to transcode this prores file.
Here is a link to the source file if anyone can help I would REALLY appreciate it...
https://www.dropbox.com/s/ejrfzad20yzaifm/10minute_Pipeline_Test.mov?dl=1
I use H264_QSV daily, and I find you have to declare the QSV device as being available.
Try this:
ffmpeg -err_detect ignore_err -hide_banner -loglevel verbose -init_hw_device qsv:qsv,child_device_type=qsv ^ -hwaccel qsv -hwaccel_output_format qsv -i "input.mov" -q:v 30 -preset slow -c:a aac output.mp4
There are many more options that can be added to improve efficiency, change the quality (the -q:v setting), etc.
I've found that QSV speeds things up so much that you can use a -preset of slow or very slow to get more compression for a given quality setting without significantly increasing the time it takes to convert the file.
I may not have done the copy as well as I should have.
This is a more complete copy of how I use ffmpeg.
ffmpeg -err_detect ignore_err -hide_banner -loglevel verbose -stats -benchmark -init_hw_device qsv:qsv,child_device_type=qsv ^
-hwaccel qsv -hwaccel_output_format qsv ^
-i "input file" ^
-c:a aac -q:a 1.9 -strict normal -sws_flags lanczos ^
-vf "vpp_qsv=cw=704:ch=480:cx=11:cy=0:w=640:h=480" ^
-async_depth 128 -q:v 28 -c:v h264_qsv -preset veryslow (a bunch of optimization options on how I want the compression to be done go in here, which can be discussed separately) -movflags +faststart "output file.mp4"
This is on Windows so the carat "^" is the command line continuation character.
-err_detect suppresses some of the more useless messages. -hide_banner suppresses things that I normally don't need to see at all.
-loglevel is usually set to "info" or "quiet", but if you want to know exactly which codecs are being used, set it to "verbose" as it is here.
This is the simple answer to the original question, "Am I using the QSV codec?".
-strict normal is optional, but I found some applications didn't do well with some of the newer optimizations. It does not appear to increase file size to any significant extent, and I don't run into problems running videos on old equipment.
I put the audio processing first as it seems to work better that way.
I let the codec choose the bit rate by setting the quality, as with the video (see below).
I have also included an example of the vpp_qsv video processing filter, as I find it speeds up many operations. It can, of course, be left out if you don't need it. I put it before the compression codec: ffmpeg will process them in the proper order, but I find it's easier to keep track of what's going on if I put the commands in about the same order as they will eventually be processed. When I put the commands in this order and "verbose" is on, ffmpeg reports that the output of the vpp_qsv filter remains in video memory as the input to the h264_qsv codec. This speeds things up in my tests: or, at least, it reduces the CPU load so other programs can run at the same time.
-async_depth is optional, increases the number of frames that are read before compression is done; I find this also usually makes things go a bit faster. -q:v is the compression quality setting: I've found 28 to 30 gives me good results for watching videos on a reasonably large TV, but you will have to make tests for yourself to see what setting is right for you. Doing this is much, much better than guessing what bitrate you need, the codec can do better optimizations, and so on. You will, in most cases, get variable bit rate compression, and sometimes variable frame rates. This improves compression for parts of the video that don't have much going on, while still providing higher bit rates when needed. You may be surprised at how low a bit rate can be produced this way and still have a good quality video.
I put -movflags +faststart in ALL of my MP4 videos. This moves a copy of the MOOV atom from the end of the video to the beginning. This does at least two things. First, for many players, the video will start playing faster as the information the player needs about the video is read immediately. Second, if an MP4 file ever gets truncated and the MOOV atom is missing, you will not be able to play the file at all. There are programs that pretend to be able to recover the missing information, but I have yet to see one actually work. But if the MOOV atom is also included at the beginning of the video, you will at least be able to start processing the video, and should at least get to the point where the file is damaged. It's cheap insurance, and only takes a moment or two. (This won't work if your output is a live stream, the video has to be "finished" before the atom is created.)
-stats and -benchmark are optional, I like to see how fast processing is going and be able to compare it to other times I process videos to see if any changes I make to the options are helping or not.
If there is an interest in the various vpp_qsv filter options, or in what other compression settings I use, or what settings will allow videos to work with Roku Media Player, let me know which topic I should post that in.

How can I mux a MKV and MKA file and get it to play in a browser?

I'm using ffmpeg to merge .mkv and .mka files into .mp4 files. My current command looks like this:
ffmpeg -i video.mkv -i audio.mka output_path.mp4
The audio and video files are pre-signed urls from Amazon S3. Even on a server with sufficient resources, this process is going very slowly. I've researched situations where you can tell ffmpeg to skip re-encoding each frame, but I think that in my situation it actually does need to re-encode each frame.
I've downloaded 2 sample files to my macbook pro and have installed ffmpeg locally via homebrew. When I run the command
ffmpeg -i video.mkv -i audio.mka -c copy output.mp4
I get the following output:
ffmpeg version 3.3.2 Copyright (c) 2000-2017 the FFmpeg developers
built with Apple LLVM version 8.1.0 (clang-802.0.42)
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.3.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vda
libavutil 55. 58.100 / 55. 58.100
libavcodec 57. 89.100 / 57. 89.100
libavformat 57. 71.100 / 57. 71.100
libavdevice 57. 6.100 / 57. 6.100
libavfilter 6. 82.100 / 6. 82.100
libavresample 3. 5. 0 / 3. 5. 0
libswscale 4. 6.100 / 4. 6.100
libswresample 2. 7.100 / 2. 7.100
libpostproc 54. 5.100 / 54. 5.100
Input #0, matroska,webm, from '319_audio_1498590673766.mka':
Metadata:
encoder : GStreamer matroskamux version 1.8.1.1
creation_time : 2017-06-27T19:10:58.000000Z
Duration: 00:00:03.53, start: 2.831000, bitrate: 50 kb/s
Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
Metadata:
title : Audio
Input #1, matroska,webm, from '319_video_1498590673766.mkv':
Metadata:
encoder : GStreamer matroskamux version 1.8.1.1
creation_time : 2017-06-27T19:10:58.000000Z
Duration: 00:00:03.97, start: 2.851000, bitrate: 224 kb/s
Stream #1:0(eng): Video: vp8, yuv420p(progressive), 640x480, SAR 1:1 DAR 4:3, 30 tbr, 1k tbn, 1k tbc (default)
Metadata:
title : Video
[mp4 # 0x7fa4f0806800] Could not find tag for codec vp8 in stream #0, codec not currently supported in container
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Stream mapping:
Stream #1:0 -> #0:0 (copy)
Stream #0:0 -> #0:1 (copy)
Last message repeated 1 times
So it appears that the specific encodings I'm working with are vp8 videos and opus audio files, which I believe are incompatible with the .mp4 output container. I would appreciate answers that cover ways of optimally merging vp8 and opus into .mp4 output or answers that point me in the direction of output media formats that are both compatible with vp8 & opus and are playable on web and mobile devices so that I can bypass the re-encoding step altogether.
EDIT:
Just wanted to provide a benchmark after following LordNeckbeard's advice:
4 min 41 second video transcoded locally on my mac
LordNeckbeard’s approach : 15 mins 55 seconds (955 seconds)
Current approach : 18 mins 49 seconds (1129 seconds)
18% speed increase
You can use ffmpeg to mux and/or re-encode MKV and MKA into web browser compatible formats such as Webm or MP4.
Webm mux: If the input formats are VP8/VP9 video with Vorbis or Opus audio
You can just mux into Webm if your inputs are VP8 or VP9 video and Vorbis or Opus audio, such as the inputs in your question. This should be fast because it will not re-encode:
ffmpeg -i video.mkv -i audio.mka -c copy output.webm
Default stream selection behavior is to select one stream per stream type, so with -map you can tell it which streams to choose to prevent mistakes. For example, if both inputs contain multiple streams, but you only want to first video stream from video.mkv and the first audio stream from audio.mka:
ffmpeg -i video.mkv -i audio.mka -map 0:v:0 -map 1:a:0 -c copy -movflags +faststart output.webm
MP4 mux: If the input formats are H.264/H.265 video and AAC audio
ffmpeg -i video.mkv -i audio.mka -c copy -movflags +faststart output.mp4
-movflags +faststart was added because you mentioned web playback. This will allow the video to begin playback before it is completely downloaded by the client.
Webm Re-encode: If the input formats are not compatible with Webm
You'll need to re-encode:
ffmpeg -i video.mkv -i audio.mka -c:v libvpx-vp9 -crf 33 -b:v 0 -c:a libopus output.webm
VP9 is really slow. If you want VP8 instead use -c:v libvpx. For more info see FFmpeg Wiki: VP8 and FFmpeg Wiki: VP9.
If you don't have libopus support use libvorbis instead.
MP4 Re-encode: If the input formats are not compatible with MP4
ffmpeg -i video.mkv -i audio.mka -c:v libx264 -crf 23 -preset medium -c:a aac -movflags +faststart output.mp4
For video, control quality with -crf and encoding speed with -preset. See FFmpeg Wiki: H.264 and FFmpeg Wiki: AAC for more info.
If your target devices are limited in the H.264 profiles they support you can add -profile:v main or -profile:v baseline.
ffprobe for scripting
You can make a script to automate this. ffprobe can be used to determine the formats:
$ ffprobe -loglevel error -select_streams v:0 -show_entries stream=codec_name -of csv=p=0 video.mkv
h264
$ ffprobe -loglevel error -select_streams a:0 -show_entries stream=codec_name -of csv=p=0 audio.mka
aac
The ffprobe outputs can be used as variables in an if/then statement.

Merging video and audio stream, where audio drifts

I want to record audio and video with my raspberry pi b+ 2.
I tried to accomplish this with one ffmpeg command but this is to slow. and i could not get it working correctly
I have a raspberry pi camera module and a Cirrus audio card. On the raspberry i have compiled a new kernel with support for the audio card. I also compiled ffmpeg on the raspberr with alsa support
~$ ffmpeg
ffmpeg version N-71470-g2db24cf Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.6 (Debian 4.6.3-14+rpi1)
configuration: --arch=armel --target-os=linux --enable-gpl --extra-libs=-lasound --enable-nonfree
libavutil 54. 22.101 / 54. 22.101
libavcodec 56. 34.100 / 56. 34.100
libavformat 56. 30.100 / 56. 30.100
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 14.100 / 5. 14.100
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...
Now i try to record an audio stream and a video stream 'at the same time'
I do this my running a shell script
raspivid -t 60000 -vs -w 1280 -h 720 -b 5000000 -fps 25 -o video.h264 &
arecord -Dhw:sndrpiwsp -r 44100 -c 2 -d 60 -f S32_LE audio.aac
i also tried with -r 22050 and -f S16_LE
when running this it sometimes gives an (i think)
overrun!!! (at least 1038.725 ms long)
at the end of the script i have two files. a video and a audio file.
now i want to merge those two together by using ffmpeg
ffmpeg -i video.h264 -i audio.aac -c:v copy -c:a aac -strict experimental output.mp4
this gives the output:
ffmpeg version N-71470-g2db24cf Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.6 (Debian 4.6.3-14+rpi1)
configuration: --arch=armel --target-os=linux --enable-gpl --extra-libs=-lasound --enable-nonfree
libavutil 54. 22.101 / 54. 22.101
libavcodec 56. 34.100 / 56. 34.100
libavformat 56. 30.100 / 56. 30.100
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 14.100 / 5. 14.100
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Input #0, h264, from 'video_1min_3.h264':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: h264 (High), yuv420p, 1280x720, 25 fps, 25 tbr, 1200k tbn, 50 tbc
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, wav, from 'audio_1min_3.aac':
Duration: 00:01:00.00, bitrate: 705 kb/s
Stream #1:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 22050 Hz, 2 channels, s16, 705 kb/s
[mp4 # 0x3230f20] Codec for stream 0 does not use global headers but container format requires global headers
Output #0, mp4, to 'output_1min_3.mp4':
Metadata:
encoder : Lavf56.30.100
Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1280x720, q=2-31, 25 fps, 25 tbr, 1200k tbn, 1200k tbc
Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 22050 Hz, stereo, fltp, 128 kb/s
Metadata:
encoder : Lavc56.34.100 aac
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #1:0 -> #0:1 (pcm_s16le (native) -> aac (native))
Press [q] to stop, [?] for help
frame= 1822 fps=310 q=-1.0 Lsize= 33269kB time=00:01:12.84 bitrate=3741.7kbits/s
video:32300kB audio:941kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.086073%
so finally i have a file output.mp4 that is a movie with audio that is in sync at the beginning but drifts away to a difference of about 4 seconds. where the audio is ahead of the video.
I hope you can help me trying to solve this issue so the audio does not drift away anymore.
Thanks in advance
( i tried to be as clear as possible )
We can try to use the -async and -vsync options to correct the audio and video time shift.
for example, i have used the below option to reduce the time lag of 2 sec seen in the audio.
./ffmpeg -async 1 -i "weatherinput.mov" -strict -2 -vcodec libx264 -movflags +faststart -vprofile high -preset slow -b:v 500k -maxrate 500k -bufsize 1000k -threads 0 -b:a 128k -pix_fmt yuv420p "weatheroutput.mp4"
Also we can use vsync options if required apart from the ioffset.
The link below can also referred for other combination of using th async, vsync and i offset to avoid the drift.

FFMpeg Concatenation Filters: Stream specifier ':0' in filtergraph matches no streams

I am developing an application that relies heavily on FFMpeg to perform various transformations on audio files. I am currently testing my FFMpeg configuration on the command line.
I am trying to concatenate multiple audio files which are in different formats (Primarily MP3, MP2 & WAV). I have been using the official TRAC documentation (https://trac.ffmpeg.org/wiki/How%20to%20concatenate%20(join%2C%20merge)%20media%20files#differentcodec) to help me with this and have created the following command:
ffmpeg -i OHIn.wav -i OHOut.wav -filter_complex '[0:0] [1:0] concat=n=2:a=1 [a]' -map '[a]' output.wav
However, when I run this on Mac OS X using version 2.0.1 of FFMpeg, I get the following error message:
Stream specifier ':0' in filtergraph description [0:0] [1:0] concat=n=2:a=1 [a] matches no streams.
Here is my full output from the terminal:
~/ffmpeg -i OHIn.wav -i OHOut.wav -filter_complex '[0:0] [1:0] concat=n=2:a=1 [a]' -map '[a]' output.wav
ffmpeg version 2.0.1 Copyright (c) 2000-2013 the FFmpeg developers
built on Aug 15 2013 10:56:46 with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
configuration: --prefix=/Volumes/Ramdisk/sw --enable-gpl --enable-pthreads --enable-version3 --enable-libspeex --enable-libvpx --disable-decoder=libvpx --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-avfilter --enable-libopencore_amrwb --enable-libopencore_amrnb --enable-filters --enable-libgsm --arch=x86_64 --enable-runtime-cpudetect
libavutil 52. 38.100 / 52. 38.100
libavcodec 55. 18.102 / 55. 18.102
libavformat 55. 12.100 / 55. 12.100
libavdevice 55. 3.100 / 55. 3.100
libavfilter 3. 79.101 / 3. 79.101
libswscale 2. 3.100 / 2. 3.100
libswresample 0. 17.102 / 0. 17.102
libpostproc 52. 3.100 / 52. 3.100
Guessed Channel Layout for Input Stream #0.0 : stereo
Input #0, wav, from 'OHIn.wav':
Duration: 00:00:06.71, bitrate: 1411 kb/s
Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, stereo, s16, 1411 kb/s
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, wav, from 'OHOut.wav':
Duration: 00:00:07.19, bitrate: 1411 kb/s
Stream #1:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, stereo, s16, 1411 kb/s
Stream specifier ':0' in filtergraph description [0:0] [1:0] concat=n=2:a=1 [a] matches no streams.
I do not understand why this does not work. FFMpeg shows that the streams 0:0 and 1:0 exist in the source files. The only other similar problems online have surrounded the use of the single quote in Windows, however testing of this confirm it does not apply to my Mac command line.
Any help would be much appreciated.
You need to tell the concat filter the number of output video streams. The default is v=1 for video and a=0 for audio, but you have no video streams. It's best to not rely on the defaults. Manually list the number of input segments (n), output video streams (v), and output audio streams (a).
ffmpeg -i input0.wav -i input1.wav -filter_complex "[0:a][1:a]concat=n=2:v=0:a=1[a]" -map "[a]" output.wav
Notice that I added v=0.
See the concat filter documentation for more info.
In addition to upvoting Lord Neckbeard's response, which, solved my problem btw: I wanted to provide a working example of a Bash Shell script, showing how I concatenate three mp3 files (an intro, middle and outro, each having the same bitrate of 160 kbps, sample rate of 44.1 Khz) into one result mp3. The reason why my filter graph reads:
[0:a] [1:a] [2:a]
instead of something like:
[0:0] [1:0] [2:0]
is because some mp3s had artwork, which, ffmpeg sees as two streams for each input mp3 file, one audio (for the music itself) and one video (for the image artwork file).
The :a portion lets ffmpeg know that you want it to use only the audio stream(s) that it reads for that input file and to pass that along to the concat filter. So any video filters get ignored. The benefit of doing this is that you don't need to know the position of the video stream (so that you don't accidentally pass it) as specified by running a command like:
ffprobe control-intro-recording.mp3.
Anyways, I digress, here's the shell script:
#!/bin/sh
ffmpeg -i ./source-files/control-intro-recording.mp3 \
-i ./source-files/control-middle-1-hour-recording-with-artwork-160-kbps.mp3 \
-i ./source-files/control-outro-recording.mp3 \
-filter_complex '[0:a] [1:a] [2:a] concat=n=3:v=0:a=1 [a]' \
-map '[a]' ./output-files/control-output-with-artwork-160-kbps-improved.mp3
I ran into this Stream specifier ':0' in filtergraph description [0:0] [1:0]... error trying to combine two video files. #LordNeckbeard's answer helped me diagnose the issue. I mention it as a separate answer in case a future querent like myself encounters this situation with video files.
It turned out that one of my videos didn't have an audio track. Adding an audio track with
ffmpeg -f lavfi -i aevalsrc=0 -i title-slide.mp4 -shortest -c:v copy \
-c:a mp3 -strict experimental title.mp4
got me going.

ffmpeg segments only the first part of my audio file

I'm implementing a http live streaming server to send audio file to iOS devices.
No problem with Apple's tools, mediafilesegmenter, my files are valid and it works fine.
I'm trying now to segment the same file using ffmpeg. I've downloaded the last stable version which is the 0.10.2 for now.
Here is how I try to segment my mp3 file:
./ffmpeg -re -i input.mp3 -f segment -segment_time 10 -segment_list outputList.m3u8 -acodec libmp3lame -map 0 output%03d.mp3
It starts the mapping like expected but finish with only one .mp3 file.
Did I miss something in the process?
Thanks in advance.
edit
Ok here is my latest command line:
ffmpeg -i input.mp3 -c:a libmp3lame -b:a 128k -map 0:0 -f segment -segment_time 10 -segment_list outputlist.m3u8 -segment_format mp3 'output%03d.mp3'
It still gives me only one file but the file is the hole song, not only one part.
Here is the output of ffmpeg:
ffmpeg version 0.10.2 Copyright (c) 2000-2012 the FFmpeg developers
built on Apr 20 2012 07:08:29 with gcc 4.5.2
configuration: --enable-gpl --enable-version3 --enable-nonfree --enable-postproc --enable-libmp3lame
libavutil 51. 35.100 / 51. 35.100
libavcodec 53. 61.100 / 53. 61.100
libavformat 53. 32.100 /
53. 32.100
libavdevice 53. 4.100 / 53. 4.100
libavfilter 2. 61.100 / 2. 61.100
libswscale 2. 1.100 / 2. 1.100
libswresample 0. 6.100 / 0. 6.100
libpostproc 52. 0.100 / 52. 0.100
[mp3 # 0x8e4f120] max_analyze_duration 5000000 reached at 5015510
Input #0, mp3, from 'BeachHouse-Myth.mp3':
Metadata:
title : Myth
artist : Beach House
track : /
album : Bloom
disc : /
genre : Alternative
TSRC : USSUB1296501
Duration: 00:04:18.69, start: 0.000000, bitrate: 320 kb/s
Stream #0:0: Audio: mp3, 44100 Hz, stereo, s16, 320 kb/s Output #0, segment, to 'stream%03d.mp3': Metadata:
title : Myth
artist : Beach House
track : /
album : Bloom
disc : /
genre : Alternative
TSRC : USSUB1296501
encoder : Lavf53.32.100
Stream #0:0: Audio: mp3, 44100 Hz, stereo, s16, 128 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mp3 -> libmp3lame)
Press [q] to stop, [?] for help
Truncating packet of size 1024 to 105ate= 0.0kbits/s
Truncating packet of size 1024 to 1
size= 0kB time=00:04:18.71 bitrate= 0.0kbits/s video:0kB audio:4042kB global headers:0kB muxing overhead -100.000000%
Audio only might be a bug. I contacted the FFMPEG player bug list, and a bug is filed: http://ffmpeg.org/trac/ffmpeg/ticket/1290

Resources