I am testing the libomxil-bellagio-master which is a implementation of openmax-il.
in this code, I found the source component(path in the source code is libomxil-bellagio-master/src/base/) only have one out port.
I have a file with audio and video . And I want to write a file reader to demux this file. how many out ports should this file-reader have?
If you wish to implement a demuxer as an OMX component, then your component would ideally have 2 ports viz., one each for audio and video.
Related
TL;DR
I'm unsure the best way to recognise when encoding videos have finished with Chokidar. Given the different methods encoders build their video files, what's the best way to accomodate all of them?
Context
I've developed a workflow for our office that allows us to quickly queue encode jobs in Adobe Premiere Pro. To queue them locally, I made use of Premiere's CEP API. I can easily send a job to Adobe Media Encoder (on the same machine) and it will automatically encode the video file to the relative project directory. This works great.
To queue encode jobs onto LAN workstations, I've taken a different approach, as the CEP API doesn't allow for any extensibility beyond the local machine. Instead I made use of Adobe Media Encoder's watch folders to detect added Premiere project files to a subfolder on our NAS (everything is on the NAS). This works great too.
Unfortunately, I'm unaware of a way for the queued encodes to be output to the relative project directory in the same way queuing locally does. I'm trying to find a way to do this by watching a common directory and moving finished files.
Since each video filename I'm queuing has this structure:
"projectName_sequenceName_givenName_renderType.mp4/.mxf" I've been able to move the files with this information easily. However, I'm struggling to accomodate for the different methods different encoding processes use. Different encoders - X264, MainConcept H264, etc - encode to disk differently.
Using Chokidar, I watched how different encoders build their files:
Example #1:
If I start a DnXHR MXF encode, it will first create the final .MXF container and then fill it. When it finishes, it writes the sidecar .XMP file. If the encode fails or is cancelled, the sidecar file will not be written.
Example #2:
If I start a TMPG x264 encode, it will first create the final .mp4 container, then create a temporary file: '.mp4_00_' appended. It will then write some initial metadata to the final container, start encoding to the '.mp4_00_' and depending on file size, create additional temporary files, '_.mp4_01_', etc. Finally it writes some additional information to the container, then to the temporary files and then deletes the temporary files. If the encode fails or is cancelled, the files are deleted.
Example #3:
If I start a MainConcept H264 encode (Premiere's default), it will first create the audio temp file, in this case '.aac'. Then create another temp file '.mkv.md0'. Halfway through encoding, it will create the video container '.m4v', start encoding to that, create some more temporary files '.md7/md6', create the final container '.mp4', along with 'sbjo.tmp', copy the '.mkv' file and '.aac' into the '.mp4' container, add a '00' file, very quickly delete it and then finish writing the '.mp4' metadata. Some of this happens very quickly and Chokidar has not always picked it up. Unless the encoder is being inconsistent.
These are the three encode types I've observed, and they're the three we need and use. I suppose I could watch each of them differently, but my concern would be if we ever switched encoders, I'd have to rewrite the code to accomodate them. The watch folder feature that Adobe Media Encoder has recognises when files have finished encoding before attempting to use them. I haven't tested every format, but a good deal. Would Media Encoder be accomodating each unique encoding process? Simply polling locked files? Or is there something I'm missing?
The code I have currently works fine for DNxHR MXFs provided they don't fail or are cancelled. It struggles with the h264/x264 examples. Since the file is created and left untouched while encoding to the temporary files, chokidar will register 'add'. Since the file is locked the move fails. Obviously this works fine when simply copying or moving a finished video file.
const watcher = chokidar.watch(['Z:/NETWORKRENDER/Finished/*.{mp4,mxf}'], {
persistent: true,
// On start, works on existing files
ignoreInitial: false,
followSymlinks: true,
interval: 1000,
awaitWriteFinish: {
stabilityThreshold: 5000,
pollInterval: 20000
},
});
We use ffprobe, I want to know if mpegdash is supported by ffprobe. I wnt to extract technical metadata of the .mpd file for example bitrate. I do not want to stream or encode. I just want to read the data
When I run the command ffprobe -formats.I don't see .mpd extension being supported by ffprobe. Is there any library or extension that I need to add.
Or ffmpeg/ffprobe would not be enough to extract the data. Is thee any other tool or library available for this.
The mpd file is a simple text file that acts like an index and points to the different audio and video streams.
You can actually just read it directly if you are just looking for the bitrate - for example a mid with 5 different ABR bit rates versions of the vide stream will have the different 'representations' listed including their bitrates - e.g. it will include a line like:
<Representation id="2" width="1280" height="720" frameRate="30/1" bandwidth="2499968" codecs="avc1.4D4029">
Well as nobody has answered till now and I have found a solution. I will answer myself.
if we run the command :
ffmpeg -re -i <file_name.mpd> -f dash -
FFmpeg reads the XML file required to read. But it also reads the chunks and a lot of other information too. Which makes it difficult to extract technical metadata. The easiest option I concluded is to write the customised code to read the mpeg_dash.mpd file. as the XML file contains all the information.
To read more about the format of mpeg_dash file: https://www.brendanlong.com/the-structure-of-an-mpeg-dash-mpd.html
You can use existing MPD parser like https://github.com/carlanton/mpd-tools
https://javalibs.com/artifact/io.lindstrom/mpd-parser
I hope this helps.
I am trying to create snapshots from a video stream using the "scene" video filter. I'm on Windows for now, but this will run on Linux I don't want the video output window to display. I can get the scenes to generate if I don't use the --vout=dummy option. When I include that option, it does not generate the scenes.
This example on the Wiki indicates that it's possible. What am I doing wrong?
Here is the line of code from the LibVLCSharp code:
LibVLC libVLC = new LibVLC("--no-audio", "--no-spu", "--vout=dummy", "--video-filter=scene", "--scene-format=jpeg", "--scene-prefix=snap", "--scene-path=C:\\temp\\", "--scene-ratio=100", $"--rtsp-user={rtspUser}", $"--rtsp-pwd={rtspPassword}");
For VLC 3, you will need to disable hardware acceleration which seems incompatible with the dummy vout.
In my tests, it was needed to do that on the media rather than globally:
media.AddOption(":avcodec-hw=none");
I still have mainy "Too high level or recursion" errors, and for that, I guess you'd better open an issue on videolan's trac.
I'm trying to compile ffmpeg into javascript so that I can decode H.264 video streams using node. The streams are H.264 frames packed into RTP NALUs so any solution has to be able to accept H.264 frames rather than a whole file name. These frames can't be in a container like MP4 or AVI because then the demuxer needs to needs the timestamp of every frame before demuxing can occur, but I'm dealing with a real time stream, no containers.
Streaming H.264 over RTP
Below is the basic code I'm using to listen on a udp socket. Inside the 'message' callback the data packet is an RTP datagram. The data portion of the data gram is an H.264 frame (P-frames and I-frames).
var PORT = 33333;
var HOST = '127.0.0.1';
var dgram = require('dgram');
var server = dgram.createSocket('udp4');
server.on('listening', function () {
var address = server.address();
console.log('UDP Server listening on ' + address.address + ":" + address.port);
});
server.on('message', function (message, remote) {
console.log(remote.address + ':' + remote.port +' - ' + message);
frame = parse_rtp(message);
rgb_frame = some_library.decode_h264(frame); // This is what I need.
});
server.bind(PORT, HOST);
I found the Broadway.js library, but I couldn't get it working and it doesn't handle P-frames which I need. I also found ffmpeg.js, but could get that to work and it needs a whole file not a stream. Likewise, fluent-ffmpeg doesn't appear to support file streams; all of the examples show a filename being passed to the constructor. So I decided to write my own API.
My current solution attempt
I have been able to compile ffmpeg into one big js file, but I can't use it like that. I want to write an API around ffmpeg and then expose those functions to JS. So it seems to me like I need to do the following:
Compile ffmpeg components (avcodec, avutil, etc.) into llvm bitcode.
Write a C wrapper that exposes the decoding functionality and uses EMSCRIPTEN_KEEPALIVE.
Use emcc to compile the wrapper and link it to the bitcode created in step 1.
I found WASM+ffmpeg, but it's in Chinese and some of the steps aren't clear. In particular there is this step:
emcc web.c process.c ../lib/libavformat.bc ../lib/libavcodec.bc ../lib/libswscale.bc ../lib/libswresample.bc ../lib/libavutil.bc \
:( Where I think I'm stuck
I don't understand how all the ffmpeg components get compiled into separate *.bc files. I followed the emmake commands in that article and I end up with one big .bc file.
2 questions
1. Does anyone know the steps to compile ffmpeg using emscripten so that I can expose some API to javascript?
2. Is there a better way (with decent documentation/examples) to decode h264 video streams using node?
To question 1:
Just follow the official doc
Consider the case where you normally build with the following
commands:
./configure
make
To build with Emscripten, you would instead use the following
commands:
# Run emconfigure with the normal configure command as an argument.
./emconfigure ./configure
# Run emmake with the normal make to generate linked LLVM bitcode.
./emmake make
# Compile the linked bitcode generated by make (project.bc) to JavaScript.
# 'project.bc' should be replaced with the make output for your project (e.g. 'yourproject.so')
# [-Ox] represents build optimisations (discussed in the next section).
./emcc [-Ox] project.bc -o project.js
To question 2: c/c++ libs can be called in a node environment. You can write some c/c++ glue code or use a proxy node module like node-ffi.
Using node-ffi to call exist libs may be easier.
May it help :)
Easiest way, specially if you need to run it in web browsers, is to utilize Media Source Extension. I managed to do it in just 3 days. Moreover, it is used automatically GPU hardware ( Cuda, Intel Qsv, ... ) acceleration as far as browser builtin support. It is important if it runs in the real World app. I have tested yesterday, just 5% of my old i7 machine CPU to decode 4K ( 4 times more than 1080p ) IP Camera H.264 nal raw live streaming. I am not sure it under server side js like nodejs, but I expect the result is similar. Contact me if you need further more. About H.265 / HEVC, you need emscripten it partially of ffmpeg or x265 or OpenH265 similarily, with minimal bc size ( less than 1,2,3 M depending on config. ). Good luck...
I want to stream my webcam in linux with VLC to the iPod. From what I've seen on the web, the easiest way is to use a web server and then access to it from the iPod like this:
NSString *url = #"http://www.example.com/path/to/movie.mp4";
MPMoviePlayerController *moviePlayer = [[MPMoviePlayerController alloc] initWithContentURL:[NSURL URLWithString:url]];
[moviePlayer play];
I have never used web services before and would like to know how i can achieve this whole process. Thank you
EDIT: After setting up the linux/vlc/segmenter, this is what i get in the terminal after running the comment from Warren and exiting vlc:
VLC media player 1.1.4 The Luggage (revision exported)
Blocked: call to unsetenv("DBUS_ACTIVATION_ADDRESS")
Blocked: call to unsetenv("DBUS_ACTIVATION_BUS_TYPE")
[0x87bc914] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Blocked: call to setlocale(6, "")
Blocked: call to sigaction(17, 0xb71840d4, 0xb7184048)
Warning: call to signal(13, 0x1)
Warning: call to signal(13, 0x1)
Warning: call to srand(1309581991)
Warning: call to rand()
Blocked: call to setlocale(6, "")
(process:4398): Gtk-WARNING **: Locale not supported by C library.
Using the fallback 'C' locale.
Warning: call to signal(13, 0x1)
Warning: call to signal(13, 0x1)
Blocked: call to setlocale(6, "")
Could not open input file, make sure it is an mpegts file: -1
Help me understanding all this? thks!
The URL you show assumes the video is prerecorded.
For live HTTP streaming to an iOS device, the URL will instead end in .m3u or .m3u8, which is a common playlist format type. (It is an extended version of the Icecast playlist format, documented in this IETF draft.) This playlist tells the iOS device how to find the other files it will retrieve, in series, in order to stream the video.
The first tricky bit is producing the video stream. Unlike all other iOS compatible media files, live HTTP streaming requires an MPEG-2 transport stream (.ts) container, rather than an MPEG-4 part 14 container (.mp4, .m4v). The video codec is still H.264 and the audio AAC, as you might expect.
A command something like this should work:
$ vlc /dev/camera –intf=dummy –sout-transcode-audio-sync –sout='#transcode{\
vcodec=h264,venc=x264{\
aud,profile=baseline,level=30,keyint=30,bframes=0,ref=1,nocabac\
},\
acodec=mp4a,ab=56,deinterlace\
}:\
duplicate{dst=std{access=file,mux=ts,dst=-}}' > test.ts
This is all one long command. I've just broken it up for clarity, and to work around SO's formatting style limits. You can remove the backslashes and whitespace to make it a single long line, if you prefer. See the VLC Streaming HOWTO for help on figuring out what all that means, and how to adjust it.
The /dev/camera bit will probably have to be adjusted, and you may want to fiddle with the A/V encoding parameters based on Apple's best practices tech note (#TN 2224) to suit your target iOS device capabilites.
The next tricky bit is producing this playlist file and the video segment files from the live video feed.
Apple offers a program called mediastreamsgementer which does this, but it's not open source, it only runs on OS X, and it isn't even freely downloadable. (It comes as part of Snow Leopard, but otherwise, you have to be in Apple's developer program to download a copy.)
Chase Douglas has produced a basic segmenter which builds against libavformat from ffmpeg. There is a newer variant here which has various improvments.
To combine this with the vlc camera capture and encoding command above, replace the > test.ts part with something like this:
| segmenter - 10 test test.m3u8 http://www.example.com/path/to/
This pipes VLC's video output through the segmenter, which breaks the TS up into 10 second chunks and maintains the test.m3u8 playlist file that tells the iOS device how to find the segment files. The - argument tells the segmenter that the video stream is being piped into its standard input, rather than coming from a file. The URL fragment at the end gets prepended onto the file names mentioned in the M3U file.
Having done all that, the only adjustment that should be needed for your Cocoa Touch code is that it should be accessing test.m3u8 instead of movie.mp4.