Piping ffmpeg thumbail output to another program - node.js

I'm trying to capture frames from a live video stream (h.264) and pipe the resulting JPG images to a Node JS script, instead of saving these individual frames directly to .jpg files.
As a test, I created the following Node JS script, to simply capture the incoming piped data, then dump it to a file:
// pipe.js - test pipe output
var fs = require('fs');
var data = '';
process.stdin.resume();
process.stdin.setEncoding('utf8');
var filename = process.argv[2];
process.stdin.on('data', (chunk) => {
console.log('Received data chunk via pipe.');
data += chunk;
});
process.stdin.on('end', () => {
console.log('Data ended.');
fs.writeFile(filename, data, err => {
if (err) {
console.log('Error writing file: error #', err);
}
});
console.log('Saved file.');
});
console.log('Started... Filename = ' + filename);
Here's the ffmpeg command I used:
ffmpeg -vcodec h264_mmal -i "rtsp://[stream url]" -vframes 1 -f image2pipe - | node pipe.js test.jpg
This generated the following output, and also produced a 175kB file which contains garbage (unreadable as a jpg file anyway). FYI using ffmpeg to export directly to a jpg file produced files around 25kB in size.
...
Press [q] to stop, [?] for help
[h264_mmal # 0x130d3f0] Changing output format.
Input stream #0:0 frame changed from size:1280x720 fmt:yuvj420p to size:1280x720 fmt:yuv420p
[swscaler # 0x1450ca0] deprecated pixel format used, make sure you did set range correctly
Received data chunk via pipe.
Received data chunk via pipe.
frame= 1 fps=0.0 q=7.7 Lsize= 94kB time=00:00:00.40 bitrate=1929.0kbits/s speed=1.18x
video:94kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
Received data chunk via pipe.
Data ended.
Saved file.
You can see that the Node JS script is receiving piped data (per the "Received data via pipe" messages above. However, it doesn't seem to be outputting a valid JPG file. I can't find a way to specifically request that ffmpeg output JPG format, since there is no -vcodec option for JPG. I tried using -vcodec png and outputting to a .png file, but the resulting file was about 2MB in size and also unreadable as a png file.
Is this a problem caused by using utf8 encoding, or am I doing something else wrong?
Thanks for any advice.
UPDATE: OK I got it to send a single jpg image correctly. The issue was in the way Node JS was capturing the stream data. Here's a working script:
// pipe.js - capture piped binary input and write to file
var fs = require('fs');
var filename = process.argv[2];
console.log("Opening " + filename + " for binary writing...");
var wstream = fs.createWriteStream(filename);
process.stdin.on('readable', () => {
var chunk = '';
while ((chunk = process.stdin.read()) !== null) {
wstream.write(chunk); // Write the binary data to file
console.log("Writing chunk to file...");
}
});
process.stdin.on('end', () => {
// Close the file
wstream.end();
});
However, now the problem is this: when piping the output of ffmpeg to this script, how can I tell when one JPG file ends and another one begins?
ffmpeg command:
ffmpeg -vcodec h264_mmal -i "[my rtsp stream]" -r 1 -q:v 2 -f singlejpeg - | node pipe.js test_output.jpg
The test_output.jpg file continues to grow as long as the script runs. How can I instead know when the data for one jpg is complete and another one has started?
According to this, jpeg files always start with FF D8 FF and end with FF D9, so I guess I can check for this ending signature and start a new file at that point... any other suggestions?

Related

first m3u8 ts segment not working after mp4 to m3u8 conversion by node js

index.js
const ffmpegPath = require('#ffmpeg-installer/ffmpeg').path;
const ffmpeg = require('fluent-ffmpeg');
const process = require('process');
const args = process.argv.slice(2);
if (args.length !== 4) {
console.error('Incorrect number of arguments');
process.exit(1);
}
const startTime = args[0];
const timeDuration = args[1];
const inputFile = args[2];
const outputFile=args[3];
ffmpeg.setFfmpegPath(ffmpegPath);
ffmpeg(inputFile)
.setStartTime(startTime)
.setDuration(timeDuration)
.output(outputFile)
.outputOptions('-hls_list_size 0')
.on('end', function(err) {
if(!err) { console.log('conversion Done') }
})
.on('error', function(err){
console.log('error: ', err)
}).run();
Here is the index.js and I'm running it by hitting the command on the terminal
node index.js 5 40 ./input.mp4 ./output.m3u8
Here 5 is for starting time and 40 is the time duration in seconds. The process creates m3u8 with ts files but the first ts file isn't getting created properly. It's been created in kb format while all the other files in mb format.
the output_test0 isn't getting generated properly and so that's why while playing the m3u8 file, the first few seconds is just static picture. This issue has been happening with the first ts output only. Any trick on how to fix it?
Following your comment about how it seems to be caused by the use of input seeking instead of output seeking:
Use seek() or seekOutput() instead of setStartTime(). The documentation describes the difference:
seek(time): seek output
Aliases: seekOutput().
Seeks streams before encoding them into the output. This is different from calling seekInput() in that the offset will only apply to one output. This is also slower, as skipped frames will still be decoded (but dropped).
The time argument may be a number (in seconds) or a timestamp string (with format [[hh:]mm:]ss[.xxx]).
ffmpeg('/path/to/file.avi')
.seekInput('1:00')
.output('from-1m30s.avi')
.seek(30)
.output('from-1m40s.avi')
.seek('0:40');
setStartTime() is an alias for seekInput(). From the same documentation:
seekInput(time): set input start time
Alias: setStartTime().
Seeks an input and only start decoding at given time offset.
Note that seek() or seekOutput() should be applied to the output and not on the input as seekInput(), i.e. after output().

Pass multiple input files to ffmpeg using a single stream in Node

I'm trying to use ffmpeg to merge multiple video files. Every file has the same encoding, and they just need to be stitched together. The problem I'm having is that I'd like to do this using streams, but ffmpeg only supports one input stream per command.
Since the files have the same encoding, I thought I could merge them into a single stream, and feed it as an input to ffmpeg.
const CombinedStream = require("combined-stream")
const ffmpeg = require("fluent-ffmpeg")
const AWS = require("aws-sdk")
const s3 = new AWS.S3()
const merge = ({ videos }) => {
const combinedStream = CombinedStream.create();
videos //I take my videos from S3 and merge them
.map((video => {
return s3
.getObject({
Bucket: "myAWSBucketName",
Key: video
})
.createReadStream()
}))
.forEach(stream => {
combinedStream.append(stream)
})
ffmpeg()
.input(combinedStream)
.save("/tmp/file.mp4")
}
merge({ videos: ["video1.mp4", "video2.mp4"]})
I was hoping ffmpeg could read the files from the single stream and output them together, but I got this error instead:
Error: ffmpeg exited with code 1: pipe:0: Invalid data found when processing input
Cannot determine format of input stream 0:0 after EOF
Error marking filters as finished
Conversion failed!
Can anyone help me?

merge multiple videos to a single .mp4 files using fluent-ffmpeg

Version information
fluent-ffmpeg version: 2.1.2
ffmpeg version:4
OS:linux mint
Code to reproduce
var fluent_ffmpeg = require("fluent-ffmpeg");
var mergedVideo = fluent_ffmpeg();
mergedVideo
.mergeAdd('./Video1.mp4')
.mergeAdd('./Video2.mp4')
// .inputOptions(['-loglevel error','-hwaccel vdpau'])
// .outputOptions('-c:v h264_nvenc')
.on('error', function(err) {
console.log('Error ' + err.message);
})
.on('end', function() {
console.log('Finished!');
})
.mergeToFile('./mergedVideo8.mp4', '/tmp');
When I run this code, then I get conversion failed error.
Observed results
Error ffmpeg exited with code 1: Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #3:0
Conversion failed!
I have tried the same conversion using the command line:
ffmpeg -f concat -i textfile -c copy -fflags +genpts merged8.mp4
Where textfile has the following content-
file 'Video1.mp4'
file 'Video2.mp4'
And I was able to concatenate the video file. But I want to get the same result using fluent-ffmpeg.

FFmpeg Stream RTSP input and save to file at the same time using nodejs

I am using node-rtsp-stream module to stream RTSP to web with nodejs.
I am streaming RTSP source with ffmpeg, for example RTSP SOURCE - EXAMPLE
I know that I can save one or many inputs to many outputs but I dont know if there is option to stream the input and save it to file at the same time without executing two process of ffmpeg.
With the following example I am able to stream the RTSP source
ffmpeg -i rtsp-url -rtsp_transport tcp -f mpeg1video -b:v 800k -r 30
On the module is look like that:
this.stream = child_process.spawn("ffmpeg", [ "-i", this.url, "-rtsp_transport", "tcp",'-f', 'mpeg1video', '-b:v', '800k', '-r', '30', '-'], {
detached: false
});
ff =child_process.spawn("ffmpeg", [ "-i", this.url, '-b:v', '800k', '-r', '30', '1.mp4'], {
detached: false
});
this.inputStreamStarted = true;
this.stream.stdout.on('data', function(data) {
return self.emit('mpeg1data', data);
});
this.stream.stderr.on('data', function(data) {
return self.emit('ffmpegError', data);
});
As you can see I am using two process of ffmpeg to do what I want but
If anyone faced with this issue and solve it with one command ( process ), I would like to get some suggestions.
How to stream RTSP source and save it to file at the same time.
for more information about the module I use:
node-rtsp-stream
try the code: (it will read RTSP and save to a jpg file (overwrite it every 3 seconds))
var fs = require('fs');
var spawn = require('child_process').spawn;
var rtspURI = 'rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov';
var fps = 1/3;
//avconv -i rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov \
// -r 1/3 -an -y -update 1 test.jpg
var ffmpeg = spawn('avconv', ['-i',rtspURI,'-r',fps,'-an','-y','-update','1','test.jpg']);
// var ffmpeg = spawn('avconv',
// ['-i','rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov',
// '-r','1/3','-an','-y','-update','1','test.jpg']);
ffmpeg.stdout.on('data', function (data) {
console.log('stdout: ' + data);
});
ffmpeg.stderr.on('data', function (data) {
console.log('stderr: ' + data);
});
ffmpeg.on('close', function (code) {
console.log('child process exited with code ' + code);
});

fluent-ffmpeg thumbnail creation error

i try to create a video thumbnail with fluent-ffmpeg here is my code
var ffmpeg = require('fluent-ffmpeg');
exports.thumbnail = function(){
var proc = new ffmpeg({ source: 'Video/express2.mp4',nolog: true })
.withSize('150x100')
.takeScreenshots({ count: 1, timemarks: [ '00:00:02.000' ] }, 'Video/', function(err, filenames) {
console.log(filenames);
console.log('screenshots were saved');
});
}
but i keep getting this error
"mate data contains no duration, aborting screenshot creation"
any idea why,
by the way am on windows, and i put the ffmpeg folder in c/ffmpeg ,and i added the ffmpeg/bin in to my environment varableļ¼Œ i dont know if fluent-ffmpeg need to know the path of ffmpeg,but i can successfully create a thumbnail with the code below
exec("C:/ffmpeg/bin/ffmpeg -i Video/" + Name + " -ss 00:01:00.00 -r 1 -an -vframes 1 -s 300x200 -f mjpeg Video/" + Name + ".jpg")
please help me!!!
I think the issue can be caused by the .withSize('...') method call.
The doc says:
It doesn't interract well with filters. In particular, don't use the size() method to resize thumbnails, use the size option instead.
And the size() method is an alias of withSize().
Also - but this is not the problem in your case - you don't need to set either the count and the timemarks at the same time. The doc says:
count is ignored when timemarks or timestamps is specified.
Then you probably could solve with:
const ffmpeg = require('fluent-ffmpeg');
exports.thumbnail = function(){
const proc = new ffmpeg({ source: 'Video/express2.mp4',nolog: true })
.takeScreenshots({ timemarks: [ '00:00:02.000' ], size: '150x100' }, 'Video/', function(err, filenames) {
console.log(filenames);
console.log('screenshots were saved');
});
}
Have a look at the doc:
https://github.com/fluent-ffmpeg/node-fluent-ffmpeg#screenshotsoptions-dirname-generate-thumbnails
FFmpeg needs to know the duration of a video file, while most videos have this information in the file header some file don't, mostly raw videos like a raw H.264 stream.
A simple solution could be to remux the video prior to take the snapshot, the FFmpeg 0.5 command for this task it's quite simple:
ffmpeg -i input.m4v -acodec copy -vcodec copy output.m4v
This command tells FFmpeg to read the "input.m4v" file, to use the same audio encoder and video encoder (no encoding at all) for the output, and to output the data into the file output.m4v.
FFmpeg automatically adds all extra metadata/header information needed to take the snapshot later.
Try this code to create thumbnails from Video
// You have to Install Below packages First
var ffmpegPath = require('#ffmpeg-installer/ffmpeg').path;
var ffprobePath = require('#ffprobe-installer/ffprobe').path;
var ffmpeg = require('fluent-ffmpeg');
ffmpeg.setFfmpegPath(ffmpegPath);
ffmpeg.setFfprobePath(ffprobePath);
var proc = ffmpeg(sourceFilePath)
.on('filenames', function(filenames) {
console.log('screenshots are ' + filenames.join(', '));
})
.on('end', function() {
console.log('screenshots were saved');
})
.on('error', function(err) {
console.log('an error happened: ' + err.message);
})
// take 1 screenshots at predefined timemarks and size
.takeScreenshots({ count: 1, timemarks: [ '00:00:01.000' ], size: '200x200' }, "Video/");

Resources