I have several videos and photos and need to merge them with the cross-dissolve effect. The algorithm is next:
Create videos from images and add silent audio to them (so they will also have a sound stream):
ffmpeg -y -f lavfi -i anullsrc -loop 1 -i /tmp/media/import-2020-Aug-19-Wednesday-05-40-34/ea5c93fd-d946-4742-b8f7-ea9ae4d43441.jpg -c:v libx264 -t 10 -pix_fmt yuv420p -vf scale=750:1280 /tmp/media/import-2020-Aug-19-Wednesday-05-40-34/ea5c93fd-d946-4742-b8f7-ea9ae4d43441.mp4
Combine all the videos and audios into one using this command:
ffmpeg
-i /tmp/media/import-2020-Aug-19-Wednesday-05-40-34/temp_68d437c0-f5e2-4651-b07e-91533480b6ef.mp4
-i /tmp/media/import-2020-Aug-19-Wednesday-05-40-34/temp_48f3c111-610d-40c7-ac71-6ce2fbb16184.mp4
-i /tmp/media/import-2020-Aug-19-Wednesday-05-40-34/temp_1593b5d8-7e16-417d-9372-2267581cd504.mp4
-i /tmp/media/import-2020-Aug-19-Wednesday-05-40-34/temp_1ac7f6be-1b12-4e31-b904-1491cc9b9494.mp4
-i /tmp/media/import-2020-Aug-19-Wednesday-05-40-34/temp_ea5c93fd-d946-4742-b8f7-ea9ae4d43441.mp4
-filter_complex
"[0:v]trim=start=0:end=8.032,setpts=PTS-STARTPTS[clip0];
[1:v]trim=start=2:end=13.047,setpts=PTS-STARTPTS[clip1];
[2:v]trim=start=2:end=13.558,setpts=PTS-STARTPTS[clip2];
[3:v]trim=start=2:end=13.186,setpts=PTS-STARTPTS[clip3];
[4:v]trim=start=2,setpts=PTS-STARTPTS[clip4];
[0:v]trim=start=9.032:end=10.032,setpts=PTS-STARTPTS[out0];
[1:v]trim=start=14.047:end=15.047,setpts=PTS-STARTPTS[out1];
[2:v]trim=start=14.558:end=15.558,setpts=PTS-STARTPTS[out2];
[3:v]trim=start=14.186:end=15.186,setpts=PTS-STARTPTS[out3];
[1:v]trim=start=0:end=2,setpts=PTS-STARTPTS[in1];
[2:v]trim=start=0:end=2,setpts=PTS-STARTPTS[in2];
[3:v]trim=start=0:end=2,setpts=PTS-STARTPTS[in3];
[4:v]trim=start=0:end=2,setpts=PTS-STARTPTS[in4];
[in1]format=pix_fmts=yuva420p,fade=t=in:st=0:d=2:alpha=1[fadein1];
[in2]format=pix_fmts=yuva420p,fade=t=in:st=0:d=2:alpha=1[fadein2];
[in3]format=pix_fmts=yuva420p,fade=t=in:st=0:d=2:alpha=1[fadein3];
[in4]format=pix_fmts=yuva420p,fade=t=in:st=0:d=2:alpha=1[fadein4];
[out0]format=pix_fmts=yuva420p,fade=t=out:st=0:d=2:alpha=1[fadeout0];
[out1]format=pix_fmts=yuva420p,fade=t=out:st=0:d=2:alpha=1[fadeout1];
[out2]format=pix_fmts=yuva420p,fade=t=out:st=0:d=2:alpha=1[fadeout2];
[out3]format=pix_fmts=yuva420p,fade=t=out:st=0:d=2:alpha=1[fadeout3];
[fadein1]fifo[fadein1fifo];
[fadein2]fifo[fadein2fifo];
[fadein3]fifo[fadein3fifo];
[fadein4]fifo[fadein4fifo];
[fadeout0]fifo[fadeout0fifo];
[fadeout1]fifo[fadeout1fifo];
[fadeout2]fifo[fadeout2fifo];
[fadeout3]fifo[fadeout3fifo];
[fadeout0fifo][fadein1fifo]overlay[crossfade0];
[fadeout1fifo][fadein2fifo]overlay[crossfade1];
[fadeout2fifo][fadein3fifo]overlay[crossfade2];
[fadeout3fifo][fadein4fifo]overlay[crossfade3];
[clip0][crossfade0][clip1][crossfade1][clip2][crossfade2][clip3][crossfade3][clip4]concat=n=9[output];
[0:a][1:a]acrossfade=d=10:c1=tri:c2=tri[A1];
[A1][2:a]acrossfade=d=10:c1=tri:c2=tri[A2];
[A2][3:a]acrossfade=d=10:c1=tri:c2=tri[A3];
[A3][4:a]acrossfade=d=10:c1=tri:c2=tri[audio] "
-vsync 0 -map "[output]" -map "[audio]" /tmp/media/final/some_filename_d0d2aab0-792a-4540-b2d3-e64abe98bf5c.mp4
And all works pretty well, but if I have, for example:
picture
video
video
picture
Then the sound from the second video is mapping to the first picture and sound from the third video to second video. And the third video actually goes without sound.
It seems like it's happening because the silent sound of the first picture is pretty short. An I right?
If so, how can I increase its duration?
I would much appreciate any help with this!
Assuming 5 inputs of 10 seconds each, all with audio streams*, with ffmpeg 4.3 or newer, use the xfade and acrossfade filters.
ffmpeg
-i in1.mp4
-i in2.mp4
-i in3.mp4
-i in4.mp4
-i in5.mp4
-filter_complex
" [0][1]xfade=transition=fade:duration=2:offset=8[V01];
[V01][2]xfade=transition=fade:duration=2:offset=16[V02];
[V02][3]xfade=transition=fade:duration=2:offset=24[V03];
[V03][4]xfade=transition=fade:duration=2:offset=32[video];
[0:a][1:a]acrossfade=d=2:c1=tri:c2=tri[A01];
[A01][2:a]acrossfade=d=2:c1=tri:c2=tri[A02];
[A02][3:a]acrossfade=d=2:c1=tri:c2=tri[A03];
[A03][4:a]acrossfade=d=2:c1=tri:c2=tri[audio]"
-vsync 0 -map "[video]" -map "[audio]" out.mp4
*if there's no existing audio stream, add one using the command in step 1.
If the existing audio stream of a file isn't 10 seconds long, use these filters on it before acrossfade.
[input]aresample=async=1:first_pts=0,apad,atrim=0:10[filtered]
and then use this filtered stream as input.
I am trying to merge multiple audio files into a single file but instead of concatenating which I can do using the following command:
ffmpeg -v debug -i file1.wav -i file2.wav -i file3.wav -filter_complex [0:0]concat=n=3:v=0:a=1[out] -map [out] output.wav
Though this command works fine for concatenating, I want to overlap let's say the last 100ms of the end of the first file and 100ms of the start of the next file.
I am now trying to use 'acrossfade' filter that ffmpeg provides but I am not having any success with it.
ffmpeg -v debug -i file1.wav -i file2.wav -i file3.wav -filter_complex [0:a]acrossfade=d=0.100:c1=exp:c2=exp,[1:a]acrossfade=d=0.100:c1=exp:c2=exp,[2:a]acrossfade=d=0.100:c1=exp:c2=exp
This is what I have come up till now, but does not work as it throws 'Buffer is too short (n=0) for frame_length=1' error.
The documentation is not very helpful, does anyone have any idea what can be done?
Thanks in advance!
acrossfade is meant to create a transition between two inputs. So, each pair of inputs has to have acrossfade applied with the result being used as an input for the next acrossfade.
ffmpeg -v debug -i file1.wav -i file2.wav -i file3.wav -filter_complex "[0:a][1:a]acrossfade=d=0.100:c1=exp:c2=exp[a01];[a01][2:a]acrossfade=d=0.100:c1=exp:c2=exp" out.wav
Edit: your inputs are 16000 Hz, and your crossfade duration is 0.1s (!), which is less than 2 audio frames at the input sampling rate. Default frame size is 1024 samples. So, frame size needs to be lowered.
ffmpeg -v debug -i file1.wav -i file2.wav -i file3.wav -filter_complex "[0:a]asetnsamples=256[0a];[1:a]asetnsamples=256[1a];[2:a]asetnsamples=256[2a];[0a][1a]acrossfade=d=0.100:c1=exp:c2=exp[a01];[a01][2a]acrossfade=d=0.100:c1=exp:c2=exp" out.wav
I need offset (some silence) in the start of file, so I tried:
./ffmpeg -itsoffset 100 -i 3.mp3 offset_test.mp3
But it doesn't work.
How to add offset to audio file with ffmpeg?
For formats without timestamps, you'll need to add silence, as #Brad mentioned.
ffmpeg -i 3.mp3 -af adelay=100000|100000 delayed.mp3
The adelay takes delay in milliseconds per channel, separated by |.
The easiest way I found so far for ffmpeg ver > 4.2:
ffmpeg -i audio_in.wav -af areverse,apad=pad_dur=1s,areverse audio_out.wav
This will add an offset of 1 second to the audio.
I noticed that ffmpeg amix filter doesn't output good result in specific situation. It works fine if input files have equal duration. In that case volume is dropped in constant value and could be fixed with ",volume=2".
In my case I'm using files with different duration. Resulted volume is not good. First mixed stream resulted in lowest volume, and last one is highest. You can see on image that volume is increased linearly withing a time.
My command:
ffmpeg -i temp_0.mp4 -i user_2123_10.mp4 -i user_2123_3.mp4 -i user_2123_4.mp4
-i user_2123_7.mp4 -i user_2123_5.mp4 -i user_2123_1.mp4 -i user_2123_8.mp4
-i user_2123_0.mp4 -i user_2123_6.mp4 -i user_2123_9.mp4 -i user_2123_2.mp4
-i user_2123_11.mp4 -filter_complex "[1:a]adelay=34741.0[aud1];
[2:a]adelay=18241.0[aud2];[3:a]adelay=20602.0[aud3];
[4:a]adelay=27852.0[aud4];[5:a]adelay=22941.0[aud5];
[6:a]adelay=13142.0[aud6];[7:a]adelay=29810.0[aud7];
[8:a]adelay=12.0[aud8];[9:a]adelay=25692.0[aud9];
[10:a]adelay=32143.002[aud10];[11:a]adelay=16101.0[aud11];
[12:a]adelay=40848.0[aud12];
[0:a][aud1][aud2][aud3][aud4][aud5][aud6][aud7]
[aud8][aud9][aud10][aud11]
[aud12]amix=inputs=13:duration=first:dropout_transition=0"
-vcodec copy -y temp_1.mp4
That could be fixed by applying silence at the beginning and end of each clip, then they will have same duration and volume will be at the same level.
Please suggest how I can use amix to mix many inputs and ensure constant volume level.
amix scales each input's volume by 1/n where n = no. of active inputs. This is evaluated for each audio frame. So when an input drops out, the volume of the remaining inputs is scaled by a smaller amount, hence their volumes increase.
Changing the dropout_transition for all earlier inputs, as suggested in other answers, is one approach, but I think it will result in coarse volume modulations. Better method is to normalize the audio after the amix.
At present, you have two options, the loudnorm or the dynaudnorm filter. The latter is much faster
Syntax is to add it after the amix, so
[aud11][aud12]amix=inputs=13:duration=first:dropout_transition=0,dynaudnorm"
Read the documentation, if you wish to tweak parameters for maximum volume or RMS mode normalization..etc
The latest version of FFMPEG includes the normalize parameter for the amix filter, which you can use to turn off the constantly changing normalization. Here's the documentation for it.
Your amix filter string can be changed to:
[aud12]amix=inputs=13:normalize=0
The solution I've found is to specify the volume for each track in a "descendant" order and use no normalization filter afterwards.
I use this example, where I concat the same audio file in different positions:
ffmpeg -vn -i test.mp3 -i test.mp3 -i test.mp3 -filter_complex "[0]adelay=0|0,volume=3[a];[1]adelay=2000|2000,volume=2[b];[2]adelay=4000|4000,volume=1[c];[a][b][c]amix=inputs=3:dropout_transition=0" -q:a 1 -acodec libmp3lame -y amix-volume.mp3
More details, see this image. The first track is the normal mixing, the second is the one with volumes specified; the third is the original track. As we can see the 2nd track looks to have a normal volume.
ffmpeg -vn -i test.mp3 -i test.mp3 -i test.mp3 -filter_complex "[0]adelay=0|0[a];[1]adelay=2000|2000[b];[2]adelay=4000|4000[c];[a][b][c]amix=inputs=3:dropout_transition=0" -q:a 1 -acodec libmp3lame -y amix-no-volume.mp3
ffmpeg -vn -i test.mp3 -i test.mp3 -i test.mp3 -filter_complex "[0]adelay=0|0,volume=3[a];[1]adelay=2000|2000,volume=2[b];[2]adelay=4000|4000,volume=1[c];[a][b][c]amix=inputs=3:dropout_transition=0" -q:a 1 -acodec libmp3lame -y amix-volume.mp3
I can't really understand why amix changes the volume; anyway; I was digging around since a while for a good solution.
The solution seems to be a combination of "pre-amp", or multiplication, as Maxim puts it, AND you have to set dropout_transition >= max delay + max input length (or a very high number):
amix=inputs=13:dropout_transition=1000,volume=13
Notes:
amix has to resample float anyway, so there is no downside with adding the volume filter (which by default resamples to float, too).
And since we're using floats, there's no clipping and (almost) no loss of precision.
H't to #Mulvya for the analysis but their solution is frustratingly non-mathematical
I was originally trying to do this with sox, which was too slow. Sox's remix filter has the -m switch which disables the 1/n adjustment.
While faster, ffmpeg seems to be using way more memory for the same task. YMMV - I didn't test this thoroughly, because I finally settled on a small python script which uses pydub's overlay function, and only keeps the final output file and one segment in memory (whereas ffmpeg and sox seem to keep all of the segments in memory).
I got the same problem but found a solution!
First the Problem: i had to mix a background music file with 3 different TTS voice pieces that start with different delay. At the end the background sound was extremely loud.
I tried the suggested answer but it did not work for me, the end volume was still much higher. So my thoughts were: "All inputs must have the same length so everytime the same amount of audio is active in the mix"
apad on all TTS inputs with whole_len set and -shortest option in combination did the work for me.
Example call:
ffmpeg -y
-nostats
-hide_banner
-v quiet
-hwaccel auto
-f image2pipe
-i pipe:0
-i bgAudio.aac
-i TTS1.mp3
-i TTS2.mp3
-i TTS3.mp3
-filter_complex [1:a]loudnorm=I=-16:TP=-1.5:LRA=11:linear=false[a0];[2:a]loudnorm=I=-16:TP=-1.5:LRA=11:linear=false:dual_mono=true,adelay=7680|7680,apad=whole_len=2346240[a1];[3:a]loudnorm=I=-16:TP=-1.5:LRA=11:linear=false:dual_mono=true,adelay=14640|14640,apad=whole_len=2346240[a2];[4:a]loudnorm=I=-16:TP=-1.5:LRA=11:linear=false:dual_mono=true,adelay=3240|3240,apad=whole_len=2346240[a3];[a0][a1][a2][a3]amix=inputs=4:dropout_transition=0,asplit=6[audio0][audio1][audio2][audio3][audio4][audio5];[0:v]format=yuv420p,split=6[1080p][720p][480p][360p][240p][144p]
-map [audio0] -map [1080p] -s 1920x1080 -shortest out1080p.mp4
-map [audio1] -map [720p] -s 1280x720 -shortest out720p.mp4
-map [audio2] -map [480p] -s 858x480 -shortest out480p.mp4
-map [audio3] -map [360p] -s 640x360 -shortest out360p.mp4
-map [audio4] -map [240p] -s 426x240 -shortest out240p.mp4
-map [audio5] -map [144p] -s 256x144 -shortest out144p.mp4
Hope someone helps this!
Try to use multiplication:
"amix=inputs="+ chunks.length + ":duration=first:dropout_transition=3,volume=" + chunks.length
Sorry, for not sending ffmpeg output.
After all we ended up by writing small util in C++ for mixing audio. But first we converted mp4 to raw(pcm) format. That worked just fine for us, even requires addition HDD space for raw intermediate files.
Code looks like this:
short addSounds(short a, short b) {
double da = a;
da /= 65536.0;
da += 0.5;
double db = b;
db /= 65536.0;
db += 0.5;
double z = 0;
if (da < 0.5 && db < 0.5) {
z = 2 * da*db;
}
else {
z = 2 * ( da + db ) - 2 * da* db - 1;
}
z -= 0.5;
z *= 65536.0;
return (short)z;
}
I will show you my code.
"amix="+inputs.size()+",volume="+(inputs.size()+1)/2+"[mixout]\""
I don't use the code dropout_transition=0 because it will cause the problem you meet.
but I also find the problem that volume will be lower as the size of inputs increases.
so I make the volume louder.
try to change dropout transition to the duration of the first input:
duration=first:dropout_transition=_duration_of_the_first_input_in_seconds_
here is my ffmpeg command:
ffmpeg -y -i long.wav -i short.wav -filter_complex "[1:a]adelay=6000|6000[a1];[1:a]adelay=10000|10000[a2];[1:a]adelay=14000|14000[a3];[1:a]adelay=18000|18000[a4];[1:a]adelay=21000|21000[a5];[1:a]adelay=25500|25500[a6];[0:a][a1][a2][a3][a4][a5][a6]amix=inputs=7:duration=first:dropout_transition=32[aout]" -map "[aout]" -ac 2 -b:a 192k -ar 44100 output.mp3
see two dropout transitions as screenshot
I am trying to cut a video in 2 parts then reassembling with ffmpeg but the final output has a small audio glitch right where the segments meet. I am using the following command to split the video 1.mp4 in 2 parts:
ffmpeg -i 1.mp4 -ss 00:00:00 -t 00:00:02 -async 1 1-1.mp4
and
ffmpeg -i 1.mp4 -ss 00:00:02 -t 00:00:02 -async 1 1-2.mp4
Once I have the 2 parts I am concatening them back together with:
ffmpeg -f concat -i files.txt -c copy output.mp4
files.txt is correctly listing both files. Can anyone point me to where the problem might be?
Thanks
The glitch is likely due to the audio priming sample showing up in between.
Since you're re-encoding the segments, you can do this in one command:
ffmpeg -i 1.mp4 -filter_complex
"[0]trim=duration=2[v1];[0]trim=2:4,setpts=PTS-STARTPTS[v2];
[0]atrim=duration=2[a1];[0]atrim=2:4,asetpts=PTS-STARTPTS[a2];
[v1][a1][v2][a2]concat=n=2:v=1:a=1[v][a]"
-map "[v]" -map "[a]" output.mp4
I had the same problem for about 3 weeks.
just merge the mp3 files using sox
sox in1.mp3 in2.mp3 in3.mp3 out.mp3
When I used concat with FFMPEG it made 12.5ms (I saw them on using Audacity) audio gaps. (I don't know why)
Maybe for your case it'll be better to extract the audio and video to two separate files using ffmpeg, merge them (video using FFMPEG and audio using sox) then put the files together into one container (mp4) file