for encoding a video in ffmpeg and stream to a server, i need to use the pipe "|" command to copy the video before re-encoding it and sending it to the server.
This command works perfectly in the shell :
./ffmpeg -f x11grab -s 640x480 -framerate 25 -i :0.0 -vcodec libx264 -framerate 25 -rtbufsize 2500k -s 640x480 -preset veryfast -pix_fmt yuv420p -crf 26 -force_key_frames 'expr:gte(t,n_forced*2) -minrate 850k -maxrate 850k -b:v 900k -bufsize 280k -f flv -
| ./ffmpeg -f flv -i - -c copy -f flv "rtmp://SERVER_ADRESS.twitch.tv/app/STREAM_KEY"
In the shell, i see the normal output of ffmpeg, wich contains multiple lines like this:
frame= 218 fps=0.0 q=-1.0 Lsize= 1860kB time=00:00:09.08 bitrate=1677.5kbits/s dup=1 drop=7 speed=11.2x
frame= 219 fps=0.0 q=-1.0 Lsize= 1860kB time=00:00:10.08 bitrate=1677.5kbits/s dup=1 drop=7 speed=11.2x
...
Now how to translate it to NODEJS with spawn ? If i do something like this :
var arguments = [
'-f', 'xgrab',
'-s', '640x480',
'-framerate', '25',
'-i', ':0.0',
'-vcodec', 'libx264',
'-framerate', '25'
'-rtbufsize', '2500k',
'-framerate', framerate,
'-s', '640x4',50
'-preset', 'veryfast',
'-pix_fmt', 'yuv420p',
'-crf', '26',
'-force_key_frames', 'expr:gte(t,n_forced*2)',
'-minrate', 850 +'k',
'-maxrate',850+'k',
'-b:v', 900+'k',
'-bufsize', 280+'k',
'-f', 'flv',
'-', '|',
'./ffmpeg', '-f','flv', '-i', '-',
'-c', 'copy',
'-f', 'flv', 'rtmp://SERVER_ADRESS.twitch.tv/app/STREAM_KEY'
]);
var childProcess = spawn(cmd, arguments);
childProcess.stdout.on('data', function(data){
console.log('stream: '+data.toString());
});
childProcess.stderr.on('data', function(data){
console.log('stream: '+data.toString());
});
I only get the output from the first part of the command, before the "|"
and the 2nd part never runs.
Also, i think something disastrous is happening in background because i get multiple instances of ffmpeg on my computer when i check the running processes.
The pipe is a shell construct, so you'll have to do something like:
spawn('/bin/sh', '-c', cmd_plus_arguments_and_pipes);
Related
I have a standard mp4 (audio + video)
I am trying to merge a 1.4 second mini mp4 clip into this track, replacing the video for the length of the mini clip but merging the audios together at a specific time
Would anyone know how to do this using ffmpeg?
I've tried quite a few different filters, however can't seem to get what I want
V <------->
miniclip.mp4 A <=======>
V <-----------> ↓ + ↓ <--->
standard.mp4 A <=========================>
Example to show miniclip.mp4 (1.4 seconds long) at timestamp 5.
ffmpeg -i main.mp4 -i miniclip.mp4 -filter_complex "[0:v]drawbox=t=fill:enable='between(t,5,6.4)'[bg];[1:v]setpts=PTS+5/TB[fg];[bg][fg]overlay=x=(W-w)/2:y=(H-h)/2:eof_action=pass;[1:a]adelay=5s:all=1[a1];[0:a][a1]amix" output.mp4
drawbox covers the main video with black. Only needed if miniclip.mp4 has a smaller width or height than main.mp4. You can omit it if miniclip.mp4 width and height is ≥ than main.mp4. Alternatively you could use the scale2ref filter to make miniclip.mp4 have the same width and height as main.mp4.
setpts add a 5 second offset to miniclip.mp4 video.
overlay overlays miniclip.mp4 video over main.mp4 video.
adelay adds a 5 second delay to miniclip.mp4 audio.
amix mixes miniclip.mp4 and main.mp4 audio.
More info
See FFmpeg Filter Documentation for info on each filter.
How to get video duration
Edited (now I understood the question):
First Get 1.4 seconds of standard.mp4 and audio1.mp3
-ss is the start for get the small video that will be 1.4 seconds of length (with -t option you can specify the duration, in this case 1.4 seconds) summary: cut video from min 5, 1.4 seconds
-an is for audio none copy, because you want to add a new audio1.mp3
video_only.mp4
ffmpeg -ss 00:05:00 -i standard.mp4 -t 1.4 -map 0:v -c copy -an small_only_video.mp4
audio_only.mp4
ffmpeg -ss 00:05:00 -i audio1.mp3 -t 1.4 -c copy small_only_audio.mp3
now you can to create a small_clip_audiovideo.mp4
ffmpeg -i small_only_video.mp4 -c:a mp3 -i small_only_audio.mp3 -c copy -map 0:v -map 1:a:0 -disposition:a:0 default -disposition:a:1 default -strict -2 -sn -dn -map_metadata -1 -map_chapters -1 -movflags faststart small_clip_audiovideo.mp4
V <------->
miniclip.mp4 A <=======>
V <-----------> ↓ + ↓ <------->
standard.mp4 A <=============================>
|--|--|--|--|--|--|--|--|--|--|
0 1 2 3 4 5 6 7 8 9 10
standard.mp4 have 10 seconds (aprox) of duration, have audio and video
miniclip.mp4 have 03 seconds (aprox) of duration, have video and audio
ffmpeg -i standard.mp4 |
} have same codes of video and audio?*
ffmpeg -i miniclip.mp4 |
if are not a same audio code or video code of files standard.mp4 and miniclip.mp4, you will be to recode for continue, if you want a good work.
ffmpeg -ss 00:00:00 -i standard.mp4 -t 4 -c copy 01.part_project.mp4
and 7 to 10, in 03.part_project.mp4
ffmpeg -ss 00:00:04.000 -i standard.mp4 -t 3.0000 -c copy 03.part_project.mp4
changue name or create a copy of miniclip.mp4 to 02.part_project.mp4
cp miniclip.mp4 02.part_project.mp4
(the part of 4 second to 7 seconds, of standard.mp4 will be used if you choice the OPTION 2 copy only the audio, santadard_part2_audio.mp4)
NOW THE OPTION N 1: IS TO CONTACT (UNITED) the 3 video parts
make a folder "option1" and copy 01.part_project.mp4 02.part_project.mp4 03.part_project.mp4
mkdir option1 && cp 01.part_project.mp4 02.part_project.mp4 03.part_project.mp4 ./option1 && cd ./option1
now you concat 01.part_project.mp4 + 02.part_project.mp4 + 03.part_project.mp4 into a unique file fin_option1.mp4
ffmpeg -f concat -safe 0 -i <(for f in ./*.mp4; do echo "file '$PWD/$f'"; done) -c copy fin_option1.mp4
V <------->
miniclip.mp4 A <=======>
V <-----------> ↓ + ↓ <------->
standard.mp4 A <============XXXXXXXXX========>
|--|--|--|--|--|--|--|--|--|--|
0 1 2 3 4 5 6 7 8 9 10
THE SECOND OPTION IS TO CONTACT (UNITED) the 3 video parts, BUT MIX
THE AUDIO OF miniclip.mp4 with santadard_part2_audio.mp4
get the audio stream from santadard_part2_audio.mp4 and get the audio
file only from miniclip.mp4
ffmpeg -i santadard_part2_audio.mp4 -map 0:a -c copy -vn -strict -2 mix_audio_santadad.mp4
ffmpeg -i miniclip.mp4 -map 0:a -c copy -vn -strict -2 mix_audio_miniclip.mp4
MIX ALL AUDIOS** IN ONE AND PUT THE VIDEO FROM miniclip.mp4
ffmpeg -i mix_audio_miniclip.mp4 -i mix_audio_santadad.mp4 -filter_complex amix=inputs=2:duration=longest -strict -2 audio_mixed_miniclip.mp4
get only video from miniclip.mp4
ffmpeg -i miniclip.mp4 -c copy -an miniclip_video.mp4
and get miniclip but with mixed audios, I think that it is the solution that you are looking for
ffmpeg -i miniclip_video.mp4 -i audio_mixed_miniclip.mp4 -c copy -map 0:v -map 1:a:0 -disposition:a:0 default -disposition:a:1 default -strict -2 -sn -dn -map_metadata -1 -map_chapters -1 -movflags faststart 02.part_project_OPTION2.mp4
santadard_part2_audio.mp4
+
audio_miniclip.mp4
V <------->
miniclip.mp4 A <MMMMMMMM> (audio miniclip mixed with standard.mp4)
V <-----------> ↓ + ↓ <------->
standard.mp4 A <============ ========>
|--|--|--|--|--|--|--|--|--|--|
0 1 2 3 4 5 6 7 8 9 10
make a folder "option2" and copy 01.part_project.mp4 02.part_project_OPTION2.mp4 03.part_project.mp4
mkdir option2 && cp 01.part_project.mp4 02.part_project_OPTION2.mp4 03.part_project.mp4 ./option2 && cd ./option2
ffmpeg -f concat -safe 0 -i <(for f in ./*.mp4; do echo "file '$PWD/$f'"; done) -c copy fin_option2.mp4
NOTES
** YOU CAN USE A LOT OF AUDIO MANIPULATIONS https://trac.ffmpeg.org/wiki/AudioChannelManipulation
I run FFMpeg as follows:
#!/bin/bash
fc="[1]scale=iw/2:ih/2 [pip]; [pip] zmq=bind_address=tcp\\\://127.0.0.1\\\:1235,[0]overlay=x=0:y=0"
ffmpeg -v verbose -re -y -i test.mkv -i test2.mkv -filter_complex "$fc" -f mpegts -codec:v libx264 -preset ultrafast resultzmq.mp4
I then start a Python 3 app to send zmq commands to FFMpeg:
import zmq
import time
import sys
from multiprocessing import Process
context = zmq.Context()
port = "1235"
print("Connecting to server with port {}".format(port))
socket = context.socket(zmq.REQ)
socket.connect("tcp://localhost:{}".format(port))
for request in range (20):
print("Sending request ", request, "...")
socket.send_string("Parsed_overlay_2 x 200")
message = socket.recv()
print("Received reply ", request, "[", message, "]")
time.sleep (1)
Which runs fine up until about 40 seconds when I get this from Ffmpeg (it stops getting the command):
frame= 918 fps= 24 q=19.0 size= 12192kB time=00:00:38.82 bitrate=2572.6kbits
frame= 931 fps= 24 q=19.0 size= 12402kB time=00:00:39.30 bitrate=2585.1kbits
[Parsed_zmq_1 # 0x56185e089220] Processing command #8 target:Parsed_overlay_2 command:x arg:200
[Parsed_zmq_1 # 0x56185e089220] Sending command reply for command #8:
0 Success
frame= 938 fps= 24 q=19.0 size= 12516kB time=00:00:39.82 bitrate=2574.1kbits/frame= 952 fps= 24 q=19.0 size= 12752kB time=00:00:40.33 bitrate=2590.0kbits/[Parsed_zmq_1 # 0x56185e089220] Processing command #9 target:Parsed_overlay_2 command:x arg:200
[Parsed_zmq_1 # 0x56185e089220] Sending command reply for command #9:
0 Success
frame= 963 fps= 24 q=19.0 size= 12932kB time=00:00:40.81 bitrate=2595.6kbits
frame= 976 fps= 24 q=19.0 size= 13121kB time=00:00:41.31 bitrate=2601.4kbits
frame= 992 fps= 24 q=19.0 size= 13434kB time=00:00:41.84 bitrate=2629.9kbits
frame= 1002 fps= 24 q=18.0 size= 13582kB time=00:00:42.34 bitrate=2627.2kbits
and this from the Python 3 client:
Sending request 8 ...
Received reply 8 [ b'0 Success' ]
Sending request 9 ...
Received reply 9 [ b'0 Success' ]
Sending request 10 ...
The disconnect always happens at the same time, no matter when I start the Python client. If I start it after 40 seconds, it won't send any commands at all.
On my actual application, the same thing happens but at about 60 seconds.
I tried setting up a simple Python server/client and the problem does not occur. So I assume the problem must have something to do with FFMpeg and its zmq plugin?
If you would like to test this yourself, just make sure test.mkv and test2.mkv is some video longer than 1 minute.
I would really appreciate any assistance!
After aimlessly changing the code for the better part of the day, I finally found the solution:
#!/bin/bash
fc="[1]scale=iw/2:ih/2,[0]overlay=x=0:y=0,zmq=bind_address=tcp\\\://127.0.0.1\\\:1235 "
ffmpeg -v verbose -re -y -i test.mkv -i server_upgrade_2.mkv -filter_complex "$fc" -f mpegts -codec:v libx264 -preset ultrafast resultzmq.mp4
My guess is that even though the position of the zmq filter does not matter when you try to issue commands (you can issue commands to all the filters), when the input to the zmq filter ends, so does the zmq filter.
Using REQ/REP archetype in any seriously meant, production-grade distributed system is indeed a
Highway to Hell
Never opt in for a trivially looking false beauty of REQ/REP. Never. It can and will fall into an unsalvagable mutual deadlock. The matter is not if, but just when.
I have found not any explicit reason if / why FFMPEG used REP for any particular reason / if it can start to use any other, more suitable archetype as PAIR / PAIR for pipeline-filter-internode-processing or PUSH/PULL or some advanced, composite signalling/messaging layer compositions. Again, my other posts here on ZeroMQ bring more reasoning and examples.
This is a small python3 script reading off picam using OpenCV :
#picamStream.py
import sys, os
from picamera.array import PiRGBArray
from picamera import PiCamera
import time
import cv2
# initialize the camera and grab a reference to the raw camera capture
camera = PiCamera()
camera.resolution = (960, 540)
camera.framerate = 30
rawCapture = PiRGBArray(camera, size=(960, 540))
# allow the camera to warmup
time.sleep(0.1)
# capture frames from the camera
for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
image = frame.array
# ---------------------------------
# .
# Opencv image processing goes here
# .
# ---------------------------------
os.write(1, image.tostring())
# clear the stream in preparation for the next frame
rawCapture.truncate(0)
# end
And I am trying to pipe it to ffmpeg to Youtube stream
My understanding is that I need to reference below two commands to somehow come up with a new ffmpeg command.
Piping picam live video to ffmpeg for Youtube streaming.
raspivid -o - -t 0 -vf -hf -w 960 -h 540 -fps 25 -b 1000000 | ffmpeg -re -ar 44100 -ac 2 -acodec pcm_s16le -f s16le -ac 2 -i /dev/zero -f h264 -i - -vcodec copy -acodec aac -ab 128k -g 50 -strict experimental -f flv rtmp://a.rtmp.youtube.com/live2/[STREAMKEY]
Piping OPENCV raw video to ffmpeg for mp4 file.
python3 picamStream.py | ffmpeg -f rawvideo -pixel_format bgr24 -video_size 960x540 -framerate 30 -i - foo.mp4
So far I've had no luck. Can anyone help me with this?
This is the program I use in raspberry pi.
#main.py
import subprocess
import cv2
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)
command = ['ffmpeg',
'-f', 'rawvideo',
'-pix_fmt', 'bgr24',
'-s','640x480',
'-i','-',
'-ar', '44100',
'-ac', '2',
'-acodec', 'pcm_s16le',
'-f', 's16le',
'-ac', '2',
'-i','/dev/zero',
'-acodec','aac',
'-ab','128k',
'-strict','experimental',
'-vcodec','h264',
'-pix_fmt','yuv420p',
'-g', '50',
'-vb','1000k',
'-profile:v', 'baseline',
'-preset', 'ultrafast',
'-r', '30',
'-f', 'flv',
'rtmp://a.rtmp.youtube.com/live2/[STREAMKEY]']
pipe = subprocess.Popen(command, stdin=subprocess.PIPE)
while True:
_, frame = cap.read()
pipe.stdin.write(frame.tostring())
pipe.kill()
cap.release()
Youtube needs an audio source, so use -i /dev/zero.
I hope it helps you.
I have a video and an audio file. I'm trying joining them and slice a piece of video and it's working:
ffmpeg -ss 0:0:1.950 -i "video.avi" -ss 0:0:1.950 -i "audio.mp3" -target pal-dvd -bufsize 9175040 -muxrate 50400000 -acodec ac3 -ac 2 -ab 128k -ar 44100 -t 0:0:5.997 -y "output.mpg"
The problem is when I try resize the video using the -vf filter, example:
ffmpeg -ss 0:0:1.950 -i "video.avi" -ss 0:0:1.950 -i "audio.mp3" -vf scale="1024:420" -target pal-dvd -bufsize 9175040 -muxrate 50400000 -acodec ac3 -ac 2 -ab 128k -ar 44100 -t 0:0:5.997 -y "output.mpg"
It doesn't work because of the argument: -target pal-dvd. If I remove this argument, the video resize but doesn't keep the quality I want.
-target pal-dvd is equal to -c:v mpeg2video -c:a ac3 -f dvd -s 720x576 -r 25 -pix_fmt yuv420p -g 15 -b:v 6000000 -maxrate:v 9000000 -minrate:v 0 -bufsize:v 1835008 -packetsize 2048 -muxrate 10080000 -b:a 448000 -ar 48000. Your other options override these defaults, so you can simply use these options directly and remove the -s 720x576 and use your own size instead.
I'm not sure why you want to resize to 1024x420 and then use -target pal-dvd, but this option implies additional options. From ffmpeg_opt.c:
} else if (!strcmp(arg, "dvd")) {
opt_video_codec(o, "c:v", "mpeg2video");
opt_audio_codec(o, "c:a", "ac3");
parse_option(o, "f", "dvd", options);
parse_option(o, "s", norm == PAL ? "720x576" : "720x480", options);
parse_option(o, "r", frame_rates[norm], options);
parse_option(o, "pix_fmt", "yuv420p", options);
opt_default(NULL, "g", norm == PAL ? "15" : "18");
opt_default(NULL, "b:v", "6000000");
opt_default(NULL, "maxrate:v", "9000000");
opt_default(NULL, "minrate:v", "0"); // 1500000;
opt_default(NULL, "bufsize:v", "1835008"); // 224*1024*8;
opt_default(NULL, "packetsize", "2048"); // from www.mpucoder.com: DVD sectors contain 2048 bytes of data, this is also the size of one pack.
opt_default(NULL, "muxrate", "10080000"); // from mplex project: data_rate = 1260000. mux_rate = data_rate * 8
opt_default(NULL, "b:a", "448000");
parse_option(o, "ar", "48000", options);
Also, option placement matters. If you want to resize and use -target then place the filtering after -target. Note that this will probably resize twice.
Or omit -target and manually declare each option and modify them to your desired specifications.
I'm running nodejs on a raspberry pi and I want to run a child process to spawn a webcam stream.
Outside of node my command is:
raspivid -n -mm matrix -w 320 -h 240 -fps 18 -g 100 -t 0 -b 5000000 -o - | ffmpeg -y -f h264 -i - -c:v copy -map 0:0 -f flv -rtmp_buffer 100 -rtmp_live live "rtmp://example.com/big/test"
With child_process I have to break each argument up
var args = ["-n", "-mm", "matrix", "-w", "320", "-h", "240", "-fps", "18", "-g", "100", "-t", "0", "-b", "5000000", "-o", "-", "|", "ffmpeg", "-y", "-f", "h264", "-i", "-", "-c:v", "copy", "-map", "0:0", "-f", "flv", "-rtmp_buffer", "100", "-rtmp_live", "live", "rtmp://example.com/big/test"];
camera.proc = child.spawn('raspivid', args);
However it chokes on the | character:
error, exit code 64
Invalid command line option (|)
How do I use this pipe character as an argument?
This has been answered in another question: Using two commands (using pipe |) with spawn
In summary, with child.spawn everything in args should be an argument of your 'raspivid' command. In your case, the pipe and everything after it are actually arguments for sh.
A workaround is to call child.spawn('sh', args) where args is:
var args = ['-c', <the entire command you want to run as a string>];