I am trying to recreate a bug in a device that receives an mpeg-2 stream that happens when the provider suddenly changes the codec of the elementary audio stream.
I am trying to generate this by concatenating two .ts files with ffmpeg, which have the same video codec but different audio codecs. I am ending up with a .ts with the audio channel being the same codec.
One condition is that the audio PID must remain the same during the stream, it cannot be two different channels (the stream can be interrupted though).
I need to stream this both via rtp and via a usb dvb modulator (which takes files as input)
Which is the best way to simulate this situation? Is there any tools that make it easier? Am I stuck to manually changing the file that is being streamed (interrupting the stream)?
Command output when concatenating:
[tomi#tomi-note2 ~]$ ffmpeg -i snippet_aac.ts -i snippet_mp2.ts -filter_complex "[0:v:0][0:a:0][1:v:0][1:a:0]concat=n=2:v=1:a=1[outv][outa]" -map "[outv]" -map "[outa]" concat_stream.ts
ffmpeg version n4.4 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11.1.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Input #0, mpegts, from 'snippet_aac.ts':
Duration: 00:02:00.02, start: 1.410111, bitrate: 559 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, progressive), 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 49152 vbv_delay: N/A
Stream #0:1[0x101](eng): Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 132 kb/s
Input #1, mpegts, from 'snippet_mp2.ts':
Duration: 00:02:00.01, start: 1.422422, bitrate: 797 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #1:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, progressive), 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 49152 vbv_delay: N/A
Stream #1:1[0x101](eng): Audio: mp2 ([3][0][0][0] / 0x0003), 44100 Hz, stereo, fltp, 384 kb/s
Stream mapping:
Stream #0:0 (mpeg2video) -> concat:in0:v0
Stream #0:1 (aac) -> concat:in0:a0
Stream #1:0 (mpeg2video) -> concat:in1:v0
Stream #1:1 (mp2) -> concat:in1:a0
concat:out:v0 -> Stream #0:0 (mpeg2video)
concat:out:a0 -> Stream #0:1 (mp2)
Press [q] to stop, [?] for help
Output #0, mpegts, to 'concat_stream.ts':
Metadata:
encoder : Lavf58.76.100
Stream #0:0: Video: mpeg2video (Main), yuv420p(tv, bt709, progressive), 640x360 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 90k tbn (default)
Metadata:
encoder : Lavc58.134.100 mpeg2video
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
Stream #0:1: Audio: mp2, 44100 Hz, stereo, s16, 384 kb/s (default)
Metadata:
encoder : Lavc58.134.100 mp2
frame= 7200 fps=2169 q=31.0 Lsize= 23681kB time=00:04:00.00 bitrate= 808.3kbits/s speed=72.3x
video:9878kB audio:11252kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 12.074516%
Related
I'm trying to remove a superfluous audio track from a movie file. Specifically, this movie on archive.org has the original audio as track 2, and what appears to be a slavic (Russian?) voiceover in track 1. I'd like to get rid of the latter.
I found this question that suggested I should do the following:
ffmpeg -i Benjamín\ dúfa.avi -map 0 -map -0:a:0 -c copy Benjamín\ dúfa\ \(aðeins\ íslenskt\ tal\).avi
But this doesn't work: after removing the first audio track, audio and video are not in sync in the resulting file anymore (when played in VLC). Now, I was able to overcome this problem by also reencoding the remaining audio track along the way, using
ffmpeg -i Benjamín\ dúfa.avi -map 0 -map -0:a:0 -c:v copy Benjamín\ dúfa\ \(aðeins\ íslenskt\ tal\).avi
But while this works, I'd like to understand why copying the audio track instead does not. I suppose it's a somewhat philosophical question --- consider it a matter of intellectual curiosity, combined with an aversion to needless lossy re-encoding.
So if anyone can explain this to me, I'd very grateful. Thanks!
EDIT: as per Gyan's request, here's ffmpeg's output for the first command:
ffmpeg version 4.2 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9.1.1 (GCC) 20190807
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Input #0, avi, from 'Benjamín dúfa.avi':
Metadata:
encoder : VirtualDubModRus 1.5.10.2 (build 2542/release)
IAS1 : Islenska
Duration: 01:27:37.00, start: 0.000000, bitrate: 1313 kb/s
Stream #0:0: Video: mpeg4 (Advanced Simple Profile) (XVID / 0x44495658), yuv420p, 720x400 [SAR 1:1 DAR 9:5], 979 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, fltp, 192 kb/s
Stream #0:2: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, fltp, 128 kb/s
Output #0, avi, to 'b.avi':
Metadata:
IAS1 : Islenska
ISFT : Lavf58.29.100
Stream #0:0: Video: mpeg4 (Advanced Simple Profile) (XVID / 0x44495658), yuv420p, 720x400 [SAR 1:1 DAR 9:5], q=2-31, 979 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, fltp, 128 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:2 -> #0:1 (copy)
Press [q] to stop, [?] for help
[avi # 0000000002bf00c0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
frame=131425 fps=7616 q=-1.0 Lsize= 718952kB time=01:27:37.00 bitrate=1120.3kbits/s speed= 305x
video:628523kB audio:82141kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.166227%
And also for the second, for comparison:
ffmpeg version 4.2 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9.1.1 (GCC) 20190807
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Input #0, avi, from 'Benjamín dúfa.avi':
Metadata:
encoder : VirtualDubModRus 1.5.10.2 (build 2542/release)
IAS1 : Islenska
Duration: 01:27:37.00, start: 0.000000, bitrate: 1313 kb/s
Stream #0:0: Video: mpeg4 (Advanced Simple Profile) (XVID / 0x44495658), yuv420p, 720x400 [SAR 1:1 DAR 9:5], 979 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, fltp, 192 kb/s
Stream #0:2: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, fltp, 128 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:2 -> #0:1 (mp3 (mp3float) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
Output #0, avi, to 'b2.avi':
Metadata:
IAS1 : Islenska
ISFT : Lavf58.29.100
Stream #0:0: Video: mpeg4 (Advanced Simple Profile) (XVID / 0x44495658), yuv420p, 720x400 [SAR 1:1 DAR 9:5], q=2-31, 979 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream #0:1: Audio: mp3 (libmp3lame) (U[0][0][0] / 0x0055), 48000 Hz, stereo, fltp
Metadata:
encoder : Lavc58.54.100 libmp3lame
[avi # 00000000004b7440] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
frame=131425 fps=1090 q=-1.0 Lsize= 718952kB time=01:27:37.00 bitrate=1120.3kbits/s speed=43.6x
video:628523kB audio:82141kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.166233%
I am trying to add background music to a video with an existing audio-track. I settled for the amix filter, since other filters seemed to cause a delayed audio track. So far everything is working as intended, however the output video is cut off at 11:34min. Which is 3 seconds shorter than the input video. I already tried to increase the length of the input video by e.g. 5 seconds, which still leads to the same output length of 11:34min. Here is the exact command I am using:
ffmpeg -i video.mp4 -i bg_music.wav -filter_complex "[0:a]volume=1.5[a0];[1:a]volume=0.2[a1];[a0][a1]amix=duration=shortest[a]" -map 0:v -map [a] -c:v copy -c:a aac -ar 22050 out.mp4
And this is the output I am getting. In this call the video has a length of 11:42min and the audio a length of 36:59min. I am running macOS Catalina 10.15.3:
built with Apple clang version 11.0.0 (clang-1100.0.33.17)
configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.29.100
Duration: 00:11:42.14, start: 0.000000, bitrate: 9182 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 9196 kb/s, 4.89 fps, 2.92 tbr, 15360 tbn, 120 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 22050 Hz, stereo, fltp, 100 kb/s (default)
Metadata:
handler_name : SoundHandler
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, wav, from 'bg_music.wav':
Metadata:
encoder : Adobe Audition 13.0 (Macintosh)
date : 2020-02-28
creation_time : 15:49:40
time_reference : 0
Duration: 00:36:58.57, bitrate: 2822 kb/s
Stream #1:0: Audio: pcm_f32le ([3][0][0][0] / 0x0003), 44100 Hz, stereo, flt, 2822 kb/s
Stream mapping:
Stream #0:1 (aac) -> volume
Stream #1:0 (pcm_f32le) -> volume
Stream #0:0 -> #0:0 (copy)
amix -> Stream #0:1 (aac)
Press [q] to stop, [?] for help
Output #0, mp4, to 'out.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.29.100
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, q=2-31, 9196 kb/s, 4.89 fps, 2.92 tbr, 15360 tbn, 15360 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 22050 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
encoder : Lavc58.54.100 aac
frame= 3392 fps=510 q=-1.0 Lsize= 789403kB time=00:11:33.46 bitrate=9325.3kbits/s speed= 104x
video:778257kB audio:11032kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.014279%
[aac # 0x7fe32d00e800] Qavg: 333.046```
I want to run a command that will add an image as a thumbnail to a video. The video will be shared by WhatsApp with a thumbnail showing up.
The problem is that after running the following command thumbnails don't show up in Whatsapp though it shows on Windows 10.
ffmpeg -i path/to/Video.mp4 -i path/to/Video.png -map 0 -map 1 -c copy -c:v:1 png -disposition:v:1 attached_pic -strict -2 path/to/out.mp4
And another problem is that the video with a thumbnail doesn't play on Android, iPhone, Windows, nor on Linux.
Log:
ffmpeg version 4.1.4-1+rpt1~deb10u1 2000-2019 the FFmpeg developers
built with gcc 8 (Debian)
configuration: --prefix=/usr --extra-version='1+rpt1~deb10u1' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --arch=arm --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
[libaom-av1 # 0x5b7130] v1.0.0
Input #0, matroska,webm, from 'Video.mp4':
Metadata:
COMPATIBLE_BRANDS: iso6av01mp41
MAJOR_BRAND : dash
MINOR_VERSION : 0
ENCODER : Lavf58.20.100
Duration: 00:03:30.70, start: -0.007000, bitrate: 213 kb/s
Stream #0:0: Video: av1 (Main), yuv420p(tv, bt709), 256x144, 23.98 fps, 23.98 tbr, 1k tbn, 1k tbc (default)
Metadata:
HANDLER_NAME : ISO Media file produced by Google Inc.
DURATION : 00:03:30.627000000
Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
Metadata:
DURATION : 00:03:30.701000000
Input #1, image2, from 'Video.png':
Duration: 00:00:00.04, start: 0.000000, bitrate: 23403 kb/s
Stream #1:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 1280x720 [SAR 1:1 DAR 16:9], 25 tbr, 25 tbn, 25 tbc
[mp4 # 0x625260] track 2: codec frame size is not set
Output #0, mp4, to 'out.mp4':
Metadata:
COMPATIBLE_BRANDS: iso6av01mp41
MAJOR_BRAND : dash
MINOR_VERSION : 0
encoder : Lavf58.20.100
Stream #0:0: Video: mjpeg (mp4v / 0x7634706D), yuvj420p(pc, bt470bg/unknown/unknown), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 25 tbr, 12800 tbn, 25 tbc
Stream #0:1: Video: av1 (Main) (av01 / 0x31307661), yuv420p(tv, bt709), 256x144, q=2-31, 23.98 fps, 23.98 tbr, 16k tbn, 1k tbc (default)
Metadata:
HANDLER_NAME : ISO Media file produced by Google Inc.
DURATION : 00:03:30.627000000
Stream #0:2(eng): Audio: opus (Opus / 0x7375704F), 48000 Hz, stereo, fltp (default)
Metadata:
DURATION : 00:03:30.701000000
Stream mapping:
Stream #1:0 -> #0:0 (copy)
Stream #0:0 -> #0:1 (copy)
Stream #0:1 -> #0:2 (copy)
Press [q] to stop, [?] for help
frame= 1 fps=0.0 q=-1.0 q=-1.0 size= 3840kB time=00:02:47.56 bitrate= 187.frame= 1 fps=0.0 q=-1.0 Lq=-1.0 size= 5632kB time=00:03:30.68 bitrate= 219.0kbits/s speed= 324x
video:2275kB audio:3221kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.464468%
What am I doing wrong?
Note: It's okay if it works with python.
I noticed that when I transcode the videos to 30 fps, then I get a preview in WhatsApp.
Maybe this helps.
This is what I'm trying to do:
(1) Read rtmp stream with ffmpeg(python subprocess) and piping to rawvideo
(2) processing rawvideo pipe from (1) to opencv readable image using numpy
(3) process the image from (2) using opencv for whatever application
(4) Display or (Stream)to processed image from (3)
This is the code I've written:
FFMPEG_BIN = 'ffmpeg'
#SOURCE -> rtmp://yourdomain/rtmpappname/streamkey
SOURCE = 'YOUR RTMP INPUT STREAM HERE URL'
#DESTINATION -> rtmp://yourdomain/rtmpappname/streamkey
DESTINATION = 'YOUR RTMP PROCESSED OUTPUT STREAM URL'
#frame width and height of source stream
FRAME_WIDTH = 1280
FRAME_HEIGHT = 720
import subprocess as sp
#read from rtmp source and output to rawvideo pipe
cmd_video_only_in = [ FFMPEG_BIN, '-i', SOURCE, '-f', 'image2pipe', '-pix_fmt', 'bgr24', '-vcodec', 'rawvideo','-']
video_input_stream = sp.Popen(cmd_video_only_in, stdout=sp.PIPE, bufsize=FRAME_WIDTH*FRAME_HEIGHT*3, shell=False)
#(Optional) stream opencv processed frames to DESTINATION url with audio mapped from source, audio-video sync can be achieved by changing
#the float value 2.00 of -itsoffset to what suits you
#cmd_processed_stream_out = [FFMPEG_BIN,'-y','-thread_queue_size', '1024','-f', 'rawvideo', '-pix_fmt', 'bgr24', '-video_size', str(FRAME_WIDTH)+'X'+str(FRAME_HEIGHT),'-i','-','-itsoffset','2.00','-i', SOURCE,'-map','0:v:0','-map','1:a:0','-vcodec','libx264','-pix_fmt','yuv420p','-acodec','copy','-preset','ultrafast','-f','flv',DESTINATION]
#processed_output_stream = sp.Popen(cmd_processed_stream_out, stdin=sp.PIPE, shell=False)
import cv2
import numpy as np
import time
while (True):
raw_image = video_input_stream.stdout.read(FRAME_WIDTH*FRAME_HEIGHT*3)
image = np.fromstring(raw_image, dtype='uint8')
image = image.reshape((FRAME_HEIGHT,FRAME_WIDTH,3))
#Process the image
#any opencv based image processing can be done here eg. object detection, feature extraction, etc.,
#Display output
cv2.imshow('OUTPUT', image)
#(Optional)send the processed image frames (output) to DESITNATION rtmp url
#processed_output_stream.stdin.write(image.tostring())
video_input_stream.stdout.flush()
if cv2.waitKey(25) & 0xFF == ord('q'):
video_input_stream.terminate()
processed_output_stream.stdin.close()
processed_output_stream.terminate()
break
The output frames which i'm trying to Display (using cv2.imshow) or Stream to rtmp destination are scrolling vertically from bottom to top. How do I solve this vertical scrolling issue?
EDIT1: FULL LOG - streaming output to DESTINATION URL:
$python3 processing.py
ffmpeg version 2.8.11-0ubuntu0.16.04.1 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
libavutil 54. 31.100 / 54. 31.100
libavcodec 56. 60.100 / 56. 60.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 40.101 / 5. 40.101
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100
ffmpeg version 2.8.11-0ubuntu0.16.04.1 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
libavutil 54. 31.100 / 54. 31.100
libavcodec 56. 60.100 / 56. 60.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 40.101 / 5. 40.101
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100
Metadata:
Server NGINX RTMP (github.com/arut/nginx-rtmp-module)
width 1280.00
height 720.00
displayWidth 1280.00
displayHeight 720.00
duration 0.00
framerate 25.00
fps 25.00
videodatarate 0.00
videocodecid 7.00
audiodatarate 0.00
audiocodecid 10.00
Input #0, live_flv, from 'SOURCE URL':
Metadata:
Server : NGINX RTMP (github.com/arut/nginx-rtmp-module)
displayWidth : 1280
displayHeight : 720
fps : 25
profile :
level :
Duration: 00:00:00.00, start: 10435.038000, bitrate: N/A
Stream #0:0: Video: h264 (Main), yuv420p, 1280x720, 25 fps, 25 tbr, 1k tbn, 2k tbc
Stream #0:1: Audio: aac (LC), 44100 Hz, stereo, fltp
Output #0, image2pipe, to 'pipe:':
Metadata:
Server : NGINX RTMP (github.com/arut/nginx-rtmp-module)
displayWidth : 1280
displayHeight : 720
fps : 25
profile :
level :
encoder : Lavf56.40.101
Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24, 1280x720, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
Metadata:
encoder : Lavc56.60.100 rawvideo
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Input #0, rawvideo, from 'pipe:':
Duration: N/A, start: 0.000000, bitrate: 552960 kb/s
Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24, 1280x720, 552960 kb/s, 25 tbr, 25 tbn, 25 tbc
Metadata:
Server NGINX RTMP (github.com/arut/nginx-rtmp-module)
width 1280.00
height 720.00
displayWidth 1280.00
displayHeight 720.00
duration 0.00
framerate 25.00
fps 25.00
videodatarate 0.00
videocodecid 7.00
audiodatarate 0.00
audiocodecid 10.00
Input #1, live_flv, from 'SOURCE URL':
Metadata:
Server : NGINX RTMP (github.com/arut/nginx-rtmp-module)
displayWidth : 1280
displayHeight : 720
fps : 25
profile :
level :
Duration: 00:00:00.00, start: 10438.591000, bitrate: N/A
Stream #1:0: Video: h264 (Main), yuv420p, 1280x720, 25 fps, 25 tbr, 1k tbn, 2k tbc
Stream #1:1: Audio: aac (LC), 44100 Hz, stereo, fltp
[libx264 # 0x19c1820] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 # 0x19c1820] profile Constrained Baseline, level 3.1
[libx264 # 0x19c1820] 264 - core 148 r2643 5c65704 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[flv # 0x19c0880] Codec for stream 1 does not use global headers but container format requires global headers
Output #0, flv, to 'DESINATION URL':
Metadata:
encoder : Lavf56.40.101
Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuv420p, 1280x720, q=-1--1, 25 fps, 1k tbn, 25 tbc
Metadata:
encoder : Lavc56.60.100 libx264
Stream #0:1: Audio: aac ([10][0][0][0] / 0x000A), 44100 Hz, stereo
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Stream #1:1 -> #0:1 (copy)
frame= 3 fps=1.0 q=-0.0 size= 8100kB time=00:00:00.12 bitrate=552960.0kbits/[live_flv # 0x19b1900] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
frame= 45 fps=0.0 q=20.0 size= 427kB time=00:00:01.28 bitrate=2730.9kbits/s
frame= 123 fps= 34 q=-0.0 size= 332100kB time=00:00:04.92 bitrate=552960.0kbits/
frame= 58 fps= 57 q=21.0 size= 608kB time=00:00:01.80 bitrate=2766.4kbits/s
frame= 136 fps= 33 q=-0.0 size= 367200kB time=00:00:05.44 bitrate=552960.0kbits/
frame= 70 fps= 46 q=20.0 size= 737kB time=00:00:02.28 bitrate=2648.5kbits/s
frame= 149 fps= 32 q=-0.0 size= 402300kB time=00:00:05.96 bitrate=552960.0kbits/
The following stream belongs to a .mkv file with dual audio;
Stream #0:0: Video: h264 (High), yuv420p(progressive), 1280x960 [SAR 1:1 DAR 4:3], 29.97 fps, 29.97 tbr, 1k tbn, 59.94 tbc (default)
Stream #0:1(eng): Audio: mp3, 48000 Hz, stereo, s16p, 256 kb/s (default)
Stream #0:2(por): Audio: ac3, 48000 Hz, stereo, fltp, 192 kb/s
Stream #0:3(eng): Subtitle: ass (default)
Which ffmpeg command should I use to create an output with the language in Portuguese, already with the English subtitles?
I found some commands that can extract the language-based caption that I specify, but I didn't find anything related to selecting the audio type.
This is the entire output from ffmpeg:
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
libavutil 55. 41.101 / 55. 41.101
libavcodec 57. 66.108 / 57. 66.108
libavformat 57. 58.101 / 57. 58.101
libavdevice 57. 2.100 / 57. 2.100
libavfilter 6. 67.100 / 6. 67.100
libswscale 4. 3.101 / 4. 3.101
libswresample 2. 4.100 / 2. 4.100
libpostproc 54. 2.100 / 54. 2.100
Input #0, matroska,webm, from 'mydualmkv.mkv':
Metadata:
title : Avidemux
encoder : libebml v1.2.3 + libmatroska v1.3.0
creation_time : 2015-08-04T15:25:49.000000Z
Duration: 00:23:37.44, start: 0.000000, bitrate: 1908 kb/s
Stream #0:0: Video: h264 (High), yuv420p(progressive), 1280x960 [SAR 1:1 DAR 4:3], 29.97 fps, 29.97 tbr, 1k tbn, 59.94 tbc (default)
Stream #0:1(eng): Audio: mp3, 48000 Hz, stereo, s16p, 256 kb/s (default)
Stream #0:2(pot): Audio: ac3, 48000 Hz, stereo, fltp, 192 kb/s
Stream #0:3(eng): Subtitle: ass (default)
[mp4 # 0000000000c36700] track 1: codec frame size is not set
Output #0, mp4, to 'path-to-output.mp4':
Metadata:
title : Avidemux
encoder : Lavf57.58.101
Stream #0:0: Video: h264 (High) ([33][0][0][0] / 0x0021), yuv420p(progressive), 1280x960 [SAR 1:1 DAR 4:3], q=2-31, 29.97 fps, 29.97 tbr, 16k tbn, 1k tbc (default)
Stream #0:1(por): Audio: ac3 ([165][0][0][0] / 0x00A5), 48000 Hz, stereo, fltp, 192 kb/s
Stream #0:2(eng): Subtitle: mov_text ([8][0][0][0] / 0x0008) (default)
Metadata:
encoder : Lavc57.66.108 mov_text
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:2 -> #0:1 (copy)
Stream #0:3 -> #0:2 (ass (ssa) -> mov_text (native))
Press [q] to stop, [?] for help
frame= 5451 fps=0.0 q=-1.0 size= 39273kB time=00:03:01.89 bitrate=1768.7kbits/s speed= 363x
frame=11708 fps=11695 q=-1.0 size= 79743kB time=00:06:30.62 bitrate=1672.3kbits/s speed= 390x
frame=17902 fps=11926 q=-1.0 size= 122134kB time=00:09:57.60 bitrate=1674.2kbits/s speed= 398x
frame=24717 fps=12352 q=-1.0 size= 166885kB time=00:13:44.83 bitrate=1657.4kbits/s speed= 412x
frame=31995 fps=12792 q=-1.0 size= 211010kB time=00:17:47.74 bitrate=1618.9kbits/s speed= 427x
frame=38979 fps=12988 q=-1.0 size= 260758kB time=00:21:40.64 bitrate=1642.4kbits/s speed= 433x
frame=42478 fps=13076 q=-1.0 Lsize= 286844kB time=00:23:37.31 bitrate=1657.9kbits/s speed= 436x
video:252490kB audio:33218kB subtitle:11kB other streams:0kB global headers:0kB muxing overhead: 0.393387%
Use
ffmpeg -i in.mkv -map 0:v -map 0:m:language:por -map 0:s -c copy -c:a aac -c:s mov_text out.mp4
This will copy the video, the Portuguese audio and transcode the English subtitles into a MP4.