Merging video and audio stream, where audio drifts - audio

I want to record audio and video with my raspberry pi b+ 2.
I tried to accomplish this with one ffmpeg command but this is to slow. and i could not get it working correctly
I have a raspberry pi camera module and a Cirrus audio card. On the raspberry i have compiled a new kernel with support for the audio card. I also compiled ffmpeg on the raspberr with alsa support
~$ ffmpeg
ffmpeg version N-71470-g2db24cf Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.6 (Debian 4.6.3-14+rpi1)
configuration: --arch=armel --target-os=linux --enable-gpl --extra-libs=-lasound --enable-nonfree
libavutil 54. 22.101 / 54. 22.101
libavcodec 56. 34.100 / 56. 34.100
libavformat 56. 30.100 / 56. 30.100
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 14.100 / 5. 14.100
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...
Now i try to record an audio stream and a video stream 'at the same time'
I do this my running a shell script
raspivid -t 60000 -vs -w 1280 -h 720 -b 5000000 -fps 25 -o video.h264 &
arecord -Dhw:sndrpiwsp -r 44100 -c 2 -d 60 -f S32_LE audio.aac
i also tried with -r 22050 and -f S16_LE
when running this it sometimes gives an (i think)
overrun!!! (at least 1038.725 ms long)
at the end of the script i have two files. a video and a audio file.
now i want to merge those two together by using ffmpeg
ffmpeg -i video.h264 -i audio.aac -c:v copy -c:a aac -strict experimental output.mp4
this gives the output:
ffmpeg version N-71470-g2db24cf Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.6 (Debian 4.6.3-14+rpi1)
configuration: --arch=armel --target-os=linux --enable-gpl --extra-libs=-lasound --enable-nonfree
libavutil 54. 22.101 / 54. 22.101
libavcodec 56. 34.100 / 56. 34.100
libavformat 56. 30.100 / 56. 30.100
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 14.100 / 5. 14.100
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Input #0, h264, from 'video_1min_3.h264':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: h264 (High), yuv420p, 1280x720, 25 fps, 25 tbr, 1200k tbn, 50 tbc
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, wav, from 'audio_1min_3.aac':
Duration: 00:01:00.00, bitrate: 705 kb/s
Stream #1:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 22050 Hz, 2 channels, s16, 705 kb/s
[mp4 # 0x3230f20] Codec for stream 0 does not use global headers but container format requires global headers
Output #0, mp4, to 'output_1min_3.mp4':
Metadata:
encoder : Lavf56.30.100
Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1280x720, q=2-31, 25 fps, 25 tbr, 1200k tbn, 1200k tbc
Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 22050 Hz, stereo, fltp, 128 kb/s
Metadata:
encoder : Lavc56.34.100 aac
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #1:0 -> #0:1 (pcm_s16le (native) -> aac (native))
Press [q] to stop, [?] for help
frame= 1822 fps=310 q=-1.0 Lsize= 33269kB time=00:01:12.84 bitrate=3741.7kbits/s
video:32300kB audio:941kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.086073%
so finally i have a file output.mp4 that is a movie with audio that is in sync at the beginning but drifts away to a difference of about 4 seconds. where the audio is ahead of the video.
I hope you can help me trying to solve this issue so the audio does not drift away anymore.
Thanks in advance
( i tried to be as clear as possible )

We can try to use the -async and -vsync options to correct the audio and video time shift.
for example, i have used the below option to reduce the time lag of 2 sec seen in the audio.
./ffmpeg -async 1 -i "weatherinput.mov" -strict -2 -vcodec libx264 -movflags +faststart -vprofile high -preset slow -b:v 500k -maxrate 500k -bufsize 1000k -threads 0 -b:a 128k -pix_fmt yuv420p "weatheroutput.mp4"
Also we can use vsync options if required apart from the ioffset.
The link below can also referred for other combination of using th async, vsync and i offset to avoid the drift.

Related

How do you use FFMPEG to transcode h264_qsv from Apple PRORES Quicktime?

I am trying to transcode an Apple Prores 444 to H.264 using qsv without success.
If I use this command line:
ffmpeg -i 10minute_Pipeline_Test.mov -c:v h264_qsv -c:a aac -pix_fmt qsv chris.mp4
I get:
ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
configuration: --prefix=/root/ffmpeg_build --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --bindir=/root/bin --extra-libs=-ldl --enable-gpl --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-nonfree --enable-libmfx
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Guessed Channel Layout for Input Stream #0.2 : mono
Guessed Channel Layout for Input Stream #0.3 : mono
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '10minute_Pipeline_Test.mov':
Metadata:
major_brand : qt
minor_version : 537134592
compatible_brands: qt
creation_time : 2020-12-19T12:43:38.000000Z
com.apple.quicktime.author:
com.apple.quicktime.comment:
com.apple.quicktime.copyright:
com.apple.quicktime.description:
com.apple.quicktime.director:
com.apple.quicktime.genre:
com.apple.quicktime.information:
com.apple.quicktime.keywords:
com.apple.quicktime.producer:
com.apple.quicktime.displayname:
timecode : 12:43:37;28
Duration: 00:10:06.72, start: 0.000000, bitrate: 167429 kb/s
Stream #0:0(eng): Data: none (tmcd / 0x64636D74)
Metadata:
creation_time : 1970-01-04T00:49:14.000000Z
timecode : 12:43:37;28
Stream #0:1(eng): Video: prores (Standard) (apcn / 0x6E637061), yuv422p10le(tv, GBR, progressive), 1280x720, 164985 kb/s, SAR 1:1 DAR 16:9, 59.94 fps, 59.94 tbr, 60k tbn, 60k tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:04.000000Z
Stream #0:2(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Stream #0:3(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Metadata:
creation_time : 2003-10-05T11:26:56.000000Z
File 'chris.mp4' already exists. Overwrite ? [y/N] y
Stream mapping:
Stream #0:1 -> #0:0 (prores (native) -> h264 (h264_qsv))
Stream #0:2 -> #0:1 (pcm_s24le (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_qsv # 0x56265b81a800] Selected ratecontrol mode is unsupported
[h264_qsv # 0x56265b81a800] Low power mode is unsupported
[h264_qsv # 0x56265b81a800] Current frame rate is unsupported
[h264_qsv # 0x56265b81a800] Current picture structure is unsupported
[h264_qsv # 0x56265b81a800] Current resolution is unsupported
[h264_qsv # 0x56265b81a800] Current pixel format is unsupported
[h264_qsv # 0x56265b81a800] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!
user#NUC:~$ ffmpeg -i 10minute_Pipeline_Test.mov -c:v h264_qsv -c:a aac -pix_fmt qsv chris.mp4
ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
configuration: --prefix=/root/ffmpeg_build --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --bindir=/root/bin --extra-libs=-ldl --enable-gpl --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-nonfree --enable-libmfx
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Guessed Channel Layout for Input Stream #0.2 : mono
Guessed Channel Layout for Input Stream #0.3 : mono
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '10minute_Pipeline_Test.mov':
Metadata:
major_brand : qt
minor_version : 537134592
compatible_brands: qt
creation_time : 2020-12-19T12:43:38.000000Z
com.apple.quicktime.author:
com.apple.quicktime.comment:
com.apple.quicktime.copyright:
com.apple.quicktime.description:
com.apple.quicktime.director:
com.apple.quicktime.genre:
com.apple.quicktime.information:
com.apple.quicktime.keywords:
com.apple.quicktime.producer:
com.apple.quicktime.displayname:
timecode : 12:43:37;28
Duration: 00:10:06.72, start: 0.000000, bitrate: 167429 kb/s
Stream #0:0(eng): Data: none (tmcd / 0x64636D74)
Metadata:
creation_time : 1970-01-04T00:49:14.000000Z
timecode : 12:43:37;28
Stream #0:1(eng): Video: prores (Standard) (apcn / 0x6E637061), yuv422p10le(tv, GBR, progressive), 1280x720, 164985 kb/s, SAR 1:1 DAR 16:9, 59.94 fps, 59.94 tbr, 60k tbn, 60k tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:04.000000Z
Stream #0:2(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Stream #0:3(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Metadata:
creation_time : 2003-10-05T11:26:56.000000Z
File 'chris.mp4' already exists. Overwrite ? [y/N] y
Stream mapping:
Stream #0:1 -> #0:0 (prores (native) -> h264 (h264_qsv))
Stream #0:2 -> #0:1 (pcm_s24le (native) -> aac (native))
Press [q] to stop, [?] for help
Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:1
Conversion failed!
If I use:
ffmpeg -i 10minute_Pipeline_Test.mov -c:v h264_qsv -c:a aac chris.mp4
I get:
ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
configuration: --prefix=/root/ffmpeg_build --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --bindir=/root/bin --extra-libs=-ldl --enable-gpl --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-nonfree --enable-libmfx
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Guessed Channel Layout for Input Stream #0.2 : mono
Guessed Channel Layout for Input Stream #0.3 : mono
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '10minute_Pipeline_Test.mov':
Metadata:
major_brand : qt
minor_version : 537134592
compatible_brands: qt
creation_time : 2020-12-19T12:43:38.000000Z
com.apple.quicktime.author:
com.apple.quicktime.comment:
com.apple.quicktime.copyright:
com.apple.quicktime.description:
com.apple.quicktime.director:
com.apple.quicktime.genre:
com.apple.quicktime.information:
com.apple.quicktime.keywords:
com.apple.quicktime.producer:
com.apple.quicktime.displayname:
timecode : 12:43:37;28
Duration: 00:10:06.72, start: 0.000000, bitrate: 167429 kb/s
Stream #0:0(eng): Data: none (tmcd / 0x64636D74)
Metadata:
creation_time : 1970-01-04T00:49:14.000000Z
timecode : 12:43:37;28
Stream #0:1(eng): Video: prores (Standard) (apcn / 0x6E637061), yuv422p10le(tv, GBR, progressive), 1280x720, 164985 kb/s, SAR 1:1 DAR 16:9, 59.94 fps, 59.94 tbr, 60k tbn, 60k tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:04.000000Z
Stream #0:2(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Stream #0:3(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Metadata:
creation_time : 2003-10-05T11:26:56.000000Z
File 'chris.mp4' already exists. Overwrite ? [y/N] y
Stream mapping:
Stream #0:1 -> #0:0 (prores (native) -> h264 (h264_qsv))
Stream #0:2 -> #0:1 (pcm_s24le (native) -> aac (native))
Press [q] to stop, [?] for help
Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:1
Conversion failed!
user#NUC:~$ ffmpeg -i 10minute_Pipeline_Test.mov -c:v h264_qsv -c:a aac chris.mp4
ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
configuration: --prefix=/root/ffmpeg_build --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --bindir=/root/bin --extra-libs=-ldl --enable-gpl --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-nonfree --enable-libmfx
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Guessed Channel Layout for Input Stream #0.2 : mono
Guessed Channel Layout for Input Stream #0.3 : mono
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '10minute_Pipeline_Test.mov':
Metadata:
major_brand : qt
minor_version : 537134592
compatible_brands: qt
creation_time : 2020-12-19T12:43:38.000000Z
com.apple.quicktime.author:
com.apple.quicktime.comment:
com.apple.quicktime.copyright:
com.apple.quicktime.description:
com.apple.quicktime.director:
com.apple.quicktime.genre:
com.apple.quicktime.information:
com.apple.quicktime.keywords:
com.apple.quicktime.producer:
com.apple.quicktime.displayname:
timecode : 12:43:37;28
Duration: 00:10:06.72, start: 0.000000, bitrate: 167429 kb/s
Stream #0:0(eng): Data: none (tmcd / 0x64636D74)
Metadata:
creation_time : 1970-01-04T00:49:14.000000Z
timecode : 12:43:37;28
Stream #0:1(eng): Video: prores (Standard) (apcn / 0x6E637061), yuv422p10le(tv, GBR, progressive), 1280x720, 164985 kb/s, SAR 1:1 DAR 16:9, 59.94 fps, 59.94 tbr, 60k tbn, 60k tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:04.000000Z
Stream #0:2(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Stream #0:3(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32 (24 bit), 1152 kb/s (default)
Metadata:
creation_time : 2003-10-05T11:26:56.000000Z
File 'chris.mp4' already exists. Overwrite ? [y/N] y
Stream mapping:
Stream #0:1 -> #0:0 (prores (native) -> h264 (h264_qsv))
Stream #0:2 -> #0:1 (pcm_s24le (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_qsv # 0x55b3bb6e8800] Selected ratecontrol mode is unsupported
[h264_qsv # 0x55b3bb6e8800] Low power mode is unsupported
[h264_qsv # 0x55b3bb6e8800] Current frame rate is unsupported
[h264_qsv # 0x55b3bb6e8800] Current picture structure is unsupported
[h264_qsv # 0x55b3bb6e8800] Current resolution is unsupported
[h264_qsv # 0x55b3bb6e8800] Current pixel format is unsupported
[h264_qsv # 0x55b3bb6e8800] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!
I cannot get ANYTHING to work. I can transcode other h264 files without issue. I cannot seem to transcode this prores file.
Here is a link to the source file if anyone can help I would REALLY appreciate it...
https://www.dropbox.com/s/ejrfzad20yzaifm/10minute_Pipeline_Test.mov?dl=1
I use H264_QSV daily, and I find you have to declare the QSV device as being available.
Try this:
ffmpeg -err_detect ignore_err -hide_banner -loglevel verbose -init_hw_device qsv:qsv,child_device_type=qsv ^ -hwaccel qsv -hwaccel_output_format qsv -i "input.mov" -q:v 30 -preset slow -c:a aac output.mp4
There are many more options that can be added to improve efficiency, change the quality (the -q:v setting), etc.
I've found that QSV speeds things up so much that you can use a -preset of slow or very slow to get more compression for a given quality setting without significantly increasing the time it takes to convert the file.
I may not have done the copy as well as I should have.
This is a more complete copy of how I use ffmpeg.
ffmpeg -err_detect ignore_err -hide_banner -loglevel verbose -stats -benchmark -init_hw_device qsv:qsv,child_device_type=qsv ^
-hwaccel qsv -hwaccel_output_format qsv ^
-i "input file" ^
-c:a aac -q:a 1.9 -strict normal -sws_flags lanczos ^
-vf "vpp_qsv=cw=704:ch=480:cx=11:cy=0:w=640:h=480" ^
-async_depth 128 -q:v 28 -c:v h264_qsv -preset veryslow (a bunch of optimization options on how I want the compression to be done go in here, which can be discussed separately) -movflags +faststart "output file.mp4"
This is on Windows so the carat "^" is the command line continuation character.
-err_detect suppresses some of the more useless messages. -hide_banner suppresses things that I normally don't need to see at all.
-loglevel is usually set to "info" or "quiet", but if you want to know exactly which codecs are being used, set it to "verbose" as it is here.
This is the simple answer to the original question, "Am I using the QSV codec?".
-strict normal is optional, but I found some applications didn't do well with some of the newer optimizations. It does not appear to increase file size to any significant extent, and I don't run into problems running videos on old equipment.
I put the audio processing first as it seems to work better that way.
I let the codec choose the bit rate by setting the quality, as with the video (see below).
I have also included an example of the vpp_qsv video processing filter, as I find it speeds up many operations. It can, of course, be left out if you don't need it. I put it before the compression codec: ffmpeg will process them in the proper order, but I find it's easier to keep track of what's going on if I put the commands in about the same order as they will eventually be processed. When I put the commands in this order and "verbose" is on, ffmpeg reports that the output of the vpp_qsv filter remains in video memory as the input to the h264_qsv codec. This speeds things up in my tests: or, at least, it reduces the CPU load so other programs can run at the same time.
-async_depth is optional, increases the number of frames that are read before compression is done; I find this also usually makes things go a bit faster. -q:v is the compression quality setting: I've found 28 to 30 gives me good results for watching videos on a reasonably large TV, but you will have to make tests for yourself to see what setting is right for you. Doing this is much, much better than guessing what bitrate you need, the codec can do better optimizations, and so on. You will, in most cases, get variable bit rate compression, and sometimes variable frame rates. This improves compression for parts of the video that don't have much going on, while still providing higher bit rates when needed. You may be surprised at how low a bit rate can be produced this way and still have a good quality video.
I put -movflags +faststart in ALL of my MP4 videos. This moves a copy of the MOOV atom from the end of the video to the beginning. This does at least two things. First, for many players, the video will start playing faster as the information the player needs about the video is read immediately. Second, if an MP4 file ever gets truncated and the MOOV atom is missing, you will not be able to play the file at all. There are programs that pretend to be able to recover the missing information, but I have yet to see one actually work. But if the MOOV atom is also included at the beginning of the video, you will at least be able to start processing the video, and should at least get to the point where the file is damaged. It's cheap insurance, and only takes a moment or two. (This won't work if your output is a live stream, the video has to be "finished" before the atom is created.)
-stats and -benchmark are optional, I like to see how fast processing is going and be able to compare it to other times I process videos to see if any changes I make to the options are helping or not.
If there is an interest in the various vpp_qsv filter options, or in what other compression settings I use, or what settings will allow videos to work with Roku Media Player, let me know which topic I should post that in.

Sample accurate audio slicing in ffmpeg?

I need to slice an audio file in .wav format into 10 second chunks.
These chunks need to be exactly 10 seconds, not 10.04799988232 seconds.
the current code I am using is
ffmpeg -i test.wav -ss 0 -to 10 -c:a libfdk_aac -b:a 80k aac/test.aac
ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers
built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.2.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-opencl --disable-lzma --enable-nonfree --enable-vda
libavutil 55. 34.100 / 55. 34.100
libavcodec 57. 64.101 / 57. 64.101
libavformat 57. 56.100 / 57. 56.100
libavdevice 57. 1.100 / 57. 1.100
libavfilter 6. 65.100 / 6. 65.100
libavresample 3. 1. 0 / 3. 1. 0
libswscale 4. 2.100 / 4. 2.100
libswresample 2. 3.100 / 2. 3.100
libpostproc 54. 1.100 / 54. 1.100
Guessed Channel Layout for Input Stream #0.0 : stereo
Input #0, wav, from '/Users/chris/Repos/mithc/client/assets/audio/wav/test.wav':
Duration: 00:04:37.62, bitrate: 2307 kb/s
Stream #0:0: Audio: pcm_s24le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s32 (24 bit), 2304 kb/s
Output #0, adts, to '/Users/chris/Repos/mithc/client/assets/audio/aac/test.aac':
Metadata:
encoder : Lavf57.56.100
Stream #0:0: Audio: aac (libfdk_aac), 48000 Hz, stereo, s16, 80 kb/s
Metadata:
encoder : Lavc57.64.101 libfdk_aac
Stream mapping:
Stream #0:0 -> #0:0 (pcm_s24le (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
size= 148kB time=00:00:15.01 bitrate= 80.6kbits/s speed=40.9x
video:0kB audio:148kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
This code does not produce exact slices, any ideas how can this be accomplished?
Not possible*. AAC audio is stored in frames which decode to 1024 samples. So, for a 48000 Hz feed, each frame has a duration of 0.02133 seconds.
If you store the audio in a container like M4A which indicates duration per-packet, the duration of the last frame is adjusted to satisfy the specified t/ss-to. But the last frame still contains the full 1024 samples. See the readout below of the last 3 frames of a silent stream specified to be 10 seconds in a M4A. Compare the packet size(s) vis-a-vis the duration.
stream #0:
keyframe=1
duration=0.021
dts=9.941 pts=9.941
size=213
stream #0:
keyframe=1
duration=0.021
dts=9.963 pts=9.963
size=213
stream #0:
keyframe=1
duration=0.016
dts=9.984 pts=9.984
size=214
If this stream were originally stored in .aac, total duration would not be 10.00 seconds. Now whether M4A does the trick for you will depend on your player.
*there is a variant of AAC which decodes to 960 samples. So, a 48 kHz audio could be encoded to a stream exactly 10 seconds long. FFmpeg does not sport such an AAC encoder. AFAIK, many apps including itunes will not play such a file correctly. If you want to encode to this spec, there's an encoder available at https://github.com/Opendigitalradio/ODR-AudioEnc

Concat multiple (self-generated) videos using ffmpeg on raspbian linux

I am a very talented sleep talker, so I decided to write a solution that records the things I talk at night to make funny videos with subtitles of it. The project is nearly done, but I got a big problem with concating videos I generated before.
The video parts are generated from single png frames using this command:
ffmpeg -y -framerate 15 -i "${images_file_path}" -c:v libx264 -r 30 -pix_fmt yuv420p "${video_file_path}"
Then the sound is added using this command (got this from #9049970 and #11779490):
ffmpeg -y -i "${video_file_path}" -i "${mp3_file_path}" -map 0:v -map 1:a -vcodec copy -acodec copy -shortest "${final_video_file_path}"
All this is causing no problems so far, but I think it may be relevant to know how the videos are generated. I can watch all this and get valid video and sound - the full source code of this first part can be found here.
Now I added a feature that is able to generate "full videos" containing a title and a various number of previously generated "video parts" using this command:
ffmpeg -f concat -i "${video_list_path}" -filter_complex "${filter_string} concat=n=${input_file_counter}:v=1:a=1 [v] [a]" -map "[v]" -map "[a]" "${full_video_path}"
But something is wrong with it and I get this error:
Invalid file index 1 in filtergraph description [0:v:0] [1:v:0] [2:v:0] [2:a:0] [3:v:0] [4:v:0] [4:a:0] [5:v:0] [6:v:0] [6:a:0] [7:v:0] concat=n=8:v=1:a=1 [v] [a].
The full output is:
ffmpeg version N-77213-g7c1c453 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.9.2 (Raspbian 4.9.2-10)
configuration: --enable-shared --enable-gpl --prefix=/usr --enable-nonfree --enable-libmp3lame --enable-libfaac --enable-libx264 --enable-version3 --disable-mmx
libavutil 55. 10.100 / 55. 10.100
libavcodec 57. 17.100 / 57. 17.100
libavformat 57. 20.100 / 57. 20.100
libavdevice 57. 0.100 / 57. 0.100
libavfilter 6. 20.100 / 6. 20.100
libswscale 4. 0.100 / 4. 0.100
libswresample 2. 0.101 / 2. 0.101
libpostproc 54. 0.100 / 54. 0.100
[mov,mp4,m4a,3gp,3g2,mj2 # 0xc2e900] Auto-inserting h264_mp4toannexb bitstream filter
Input #0, concat, from '/usr/sleeptalk/records-rendered/3enguzpuu2gw0ogk8wkko/videos.txt':
Duration: N/A, start: 0.000000, bitrate: 61 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 58 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 2 kb/s
Metadata:
handler_name : SoundHandler
Invalid file index 1 in filtergraph description [0:v:0] [1:v:0] [2:v:0] [2:a:0] [3:v:0] [4:v:0] [4:a:0] [5:v:0] [6:v:0] [6:a:0] [7:v:0] concat=n=8:v=1:a=1 [v] [a].
I also wrote a test case so you can reproduce this on your local machine. Download the files from my dropbox. Also, the full script that renders the "final move" can be found here.
Would be great to get an Idea, got struggle to fix this the last two days.
You're using both the concat demuxer as well as the concat filter. Skip the latter, because
a) it's unnecessary and
b) I don't believe the demuxer is inducting all input files as separate inputs so the indices beyond 0 don't make sense. Also, the concat filter needs equal number of streams per input file, and their input assignment has to be pair-wise i.e. [0:v:0] [0:a:0] [1:v:0] [1:a:0] [2:v:0] [2:a:0]....
Instead, use
ffmpeg -f concat -i "${video_list_textfile}" -c copy "${full_video_path}"
where ${video_list_textfile} is a text file of the form
file 'file1.mp4'
file 'file2.mp4'
file 'file3.mp4'
...

Use ffmpeg to stream live content to azure media services

I've been trying to stream content to azure media services using ffmpeg as it's one of the options described here : http://azure.microsoft.com/blog/2014/09/18/azure-media-services-rtmp-support-and-live-encoders/
My command is :
ffmpeg -v verbose -i 300.mp4 -strict -2 -c:a aac -b:a 128k -ar 44100 -r 30 -g 60 -keyint_min 60 -b:v 400000 -c:v libx264 -preset medium -bufsize 400k -maxrate 400k -f flv rtmp://nessma-****.channel.mediaservices.windows.net:1935/live/584c99f5c47f424d9e83ac95364331e7
I have made sure that the streaming endpoint has one active streaming unit, I also made sure that the channel is actually Ready and I even get it to start streaming (which makes a PublishURL available).
When I execute the ffmpeg command to start streaming, I keep getting the following error :
ffmpeg version 2.5.2 Copyright (c) 2000-2014 the FFmpeg developers
built on Dec 30 2014 11:31:18 with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
configuration: --prefix=/Volumes/Ramdisk/sw --enable-gpl --enable-pthreads --enable-version3 --enable-libspeex --enable-libvpx --disable-decoder=libvpx --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-avfilter --enable-libopencore_amrwb --enable-libopencore_amrnb --enable-filters --enable-libgsm --enable-libvidstab --enable-libx265 --arch=x86_64 --enable-runtime-cpudetect
libavutil 54. 15.100 / 54. 15.100
libavcodec 56. 13.100 / 56. 13.100
libavformat 56. 15.102 / 56. 15.102
libavdevice 56. 3.100 / 56. 3.100
libavfilter 5. 2.103 / 5. 2.103
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Routing option strict to both codec and muxer layer
[mov,mp4,m4a,3gp,3g2,mj2 # 0x7f9a0a002c00] overread end of atom 'colr' by 1 bytes
[mov,mp4,m4a,3gp,3g2,mj2 # 0x7f9a0a002c00] stream 0, timescale not set
[mov,mp4,m4a,3gp,3g2,mj2 # 0x7f9a0a002c00] max_analyze_duration 5000000 reached at 5003637 microseconds
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '300.mp4':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: mp42isomavc1
creation_time : 2014-01-11 05:39:32
genre : Trailer
artist : Warner Bros.
title : 300: Rise of an Empire - Trailer 2
encoder : HandBrake 0.9.9 2013051800
date : 2014
Duration: 00:02:33.24, start: 0.000000, bitrate: 7377 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 (1920x1088), 7219 kb/s, 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc (default)
Metadata:
creation_time : 2014-01-11 05:39:32
encoder : JVT/AVC Coding
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 157 kb/s (default)
Metadata:
creation_time : 2014-01-11 05:39:32
Stream #0:2: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 101x150 [SAR 72:72 DAR 101:150], 90k tbr, 90k tbn, 90k tbc
rtmp://nessma-****.channel.mediaservices.windows.net:1935/live/584c99f5c47f424d9e83ac95364331e7: Input/output error
The Azure blog post clearly states that this should be possible but I can't find a working example anywhere.
Environment :
MacOS Maverick
FFMPEG installed from official build
300.mp4 : 1080p trailer of the latest 300 movie
I figured out the missing piece here ...
At the end of the publishURL, you need to add /mystream1 at the end. Hopefully, this helps somebody.
You need to add stream key name in the end of your ingest url.
In azure stream key name can be anything. It is used for references and logs purpose only.
Ex:
rtmp://nessma-****.channel.mediaservices.windows.net:1935/live/584c99f5c47f424d9e83ac95364331e7/some_random_stream_name
When people works with azure, they usually use the same stream name every time they broadcast anything. Some people, change stream name to match event name like "...live/584c99f5c47f424d9e83ac95364331e7/tennis_game_x_against_y". When you have a lot of events it will help you with troubleshooting both on your and azure side.

ffmpeg segments only the first part of my audio file

I'm implementing a http live streaming server to send audio file to iOS devices.
No problem with Apple's tools, mediafilesegmenter, my files are valid and it works fine.
I'm trying now to segment the same file using ffmpeg. I've downloaded the last stable version which is the 0.10.2 for now.
Here is how I try to segment my mp3 file:
./ffmpeg -re -i input.mp3 -f segment -segment_time 10 -segment_list outputList.m3u8 -acodec libmp3lame -map 0 output%03d.mp3
It starts the mapping like expected but finish with only one .mp3 file.
Did I miss something in the process?
Thanks in advance.
edit
Ok here is my latest command line:
ffmpeg -i input.mp3 -c:a libmp3lame -b:a 128k -map 0:0 -f segment -segment_time 10 -segment_list outputlist.m3u8 -segment_format mp3 'output%03d.mp3'
It still gives me only one file but the file is the hole song, not only one part.
Here is the output of ffmpeg:
ffmpeg version 0.10.2 Copyright (c) 2000-2012 the FFmpeg developers
built on Apr 20 2012 07:08:29 with gcc 4.5.2
configuration: --enable-gpl --enable-version3 --enable-nonfree --enable-postproc --enable-libmp3lame
libavutil 51. 35.100 / 51. 35.100
libavcodec 53. 61.100 / 53. 61.100
libavformat 53. 32.100 /
53. 32.100
libavdevice 53. 4.100 / 53. 4.100
libavfilter 2. 61.100 / 2. 61.100
libswscale 2. 1.100 / 2. 1.100
libswresample 0. 6.100 / 0. 6.100
libpostproc 52. 0.100 / 52. 0.100
[mp3 # 0x8e4f120] max_analyze_duration 5000000 reached at 5015510
Input #0, mp3, from 'BeachHouse-Myth.mp3':
Metadata:
title : Myth
artist : Beach House
track : /
album : Bloom
disc : /
genre : Alternative
TSRC : USSUB1296501
Duration: 00:04:18.69, start: 0.000000, bitrate: 320 kb/s
Stream #0:0: Audio: mp3, 44100 Hz, stereo, s16, 320 kb/s Output #0, segment, to 'stream%03d.mp3': Metadata:
title : Myth
artist : Beach House
track : /
album : Bloom
disc : /
genre : Alternative
TSRC : USSUB1296501
encoder : Lavf53.32.100
Stream #0:0: Audio: mp3, 44100 Hz, stereo, s16, 128 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mp3 -> libmp3lame)
Press [q] to stop, [?] for help
Truncating packet of size 1024 to 105ate= 0.0kbits/s
Truncating packet of size 1024 to 1
size= 0kB time=00:04:18.71 bitrate= 0.0kbits/s video:0kB audio:4042kB global headers:0kB muxing overhead -100.000000%
Audio only might be a bug. I contacted the FFMPEG player bug list, and a bug is filed: http://ffmpeg.org/trac/ffmpeg/ticket/1290

Resources