I try to open a rtsp stream using the class VideoCapture's funtion open.
I want to set timeout,but i do not know how to do.
I'm keeping this answer short, because I'm still looking for a better solution: there is a ffmpeg wrapper, which can be compiled to a ffmpeg dll file, which I guess replaces OpenCV's ffmpeg dll.
This ffmpeg wrapper is built in such a way, that a timeout can be set in milliseconds. I'll hopefully edit this when I have more information.
This is the wrapper, it's supposed to be easy to test in VS2012: Git
Related
I am trying to create snapshots from a video stream using the "scene" video filter. I'm on Windows for now, but this will run on Linux I don't want the video output window to display. I can get the scenes to generate if I don't use the --vout=dummy option. When I include that option, it does not generate the scenes.
This example on the Wiki indicates that it's possible. What am I doing wrong?
Here is the line of code from the LibVLCSharp code:
LibVLC libVLC = new LibVLC("--no-audio", "--no-spu", "--vout=dummy", "--video-filter=scene", "--scene-format=jpeg", "--scene-prefix=snap", "--scene-path=C:\\temp\\", "--scene-ratio=100", $"--rtsp-user={rtspUser}", $"--rtsp-pwd={rtspPassword}");
For VLC 3, you will need to disable hardware acceleration which seems incompatible with the dummy vout.
In my tests, it was needed to do that on the media rather than globally:
media.AddOption(":avcodec-hw=none");
I still have mainy "Too high level or recursion" errors, and for that, I guess you'd better open an issue on videolan's trac.
I'm trying to compile ffmpeg into javascript so that I can decode H.264 video streams using node. The streams are H.264 frames packed into RTP NALUs so any solution has to be able to accept H.264 frames rather than a whole file name. These frames can't be in a container like MP4 or AVI because then the demuxer needs to needs the timestamp of every frame before demuxing can occur, but I'm dealing with a real time stream, no containers.
Streaming H.264 over RTP
Below is the basic code I'm using to listen on a udp socket. Inside the 'message' callback the data packet is an RTP datagram. The data portion of the data gram is an H.264 frame (P-frames and I-frames).
var PORT = 33333;
var HOST = '127.0.0.1';
var dgram = require('dgram');
var server = dgram.createSocket('udp4');
server.on('listening', function () {
var address = server.address();
console.log('UDP Server listening on ' + address.address + ":" + address.port);
});
server.on('message', function (message, remote) {
console.log(remote.address + ':' + remote.port +' - ' + message);
frame = parse_rtp(message);
rgb_frame = some_library.decode_h264(frame); // This is what I need.
});
server.bind(PORT, HOST);
I found the Broadway.js library, but I couldn't get it working and it doesn't handle P-frames which I need. I also found ffmpeg.js, but could get that to work and it needs a whole file not a stream. Likewise, fluent-ffmpeg doesn't appear to support file streams; all of the examples show a filename being passed to the constructor. So I decided to write my own API.
My current solution attempt
I have been able to compile ffmpeg into one big js file, but I can't use it like that. I want to write an API around ffmpeg and then expose those functions to JS. So it seems to me like I need to do the following:
Compile ffmpeg components (avcodec, avutil, etc.) into llvm bitcode.
Write a C wrapper that exposes the decoding functionality and uses EMSCRIPTEN_KEEPALIVE.
Use emcc to compile the wrapper and link it to the bitcode created in step 1.
I found WASM+ffmpeg, but it's in Chinese and some of the steps aren't clear. In particular there is this step:
emcc web.c process.c ../lib/libavformat.bc ../lib/libavcodec.bc ../lib/libswscale.bc ../lib/libswresample.bc ../lib/libavutil.bc \
:( Where I think I'm stuck
I don't understand how all the ffmpeg components get compiled into separate *.bc files. I followed the emmake commands in that article and I end up with one big .bc file.
2 questions
1. Does anyone know the steps to compile ffmpeg using emscripten so that I can expose some API to javascript?
2. Is there a better way (with decent documentation/examples) to decode h264 video streams using node?
To question 1:
Just follow the official doc
Consider the case where you normally build with the following
commands:
./configure
make
To build with Emscripten, you would instead use the following
commands:
# Run emconfigure with the normal configure command as an argument.
./emconfigure ./configure
# Run emmake with the normal make to generate linked LLVM bitcode.
./emmake make
# Compile the linked bitcode generated by make (project.bc) to JavaScript.
# 'project.bc' should be replaced with the make output for your project (e.g. 'yourproject.so')
# [-Ox] represents build optimisations (discussed in the next section).
./emcc [-Ox] project.bc -o project.js
To question 2: c/c++ libs can be called in a node environment. You can write some c/c++ glue code or use a proxy node module like node-ffi.
Using node-ffi to call exist libs may be easier.
May it help :)
Easiest way, specially if you need to run it in web browsers, is to utilize Media Source Extension. I managed to do it in just 3 days. Moreover, it is used automatically GPU hardware ( Cuda, Intel Qsv, ... ) acceleration as far as browser builtin support. It is important if it runs in the real World app. I have tested yesterday, just 5% of my old i7 machine CPU to decode 4K ( 4 times more than 1080p ) IP Camera H.264 nal raw live streaming. I am not sure it under server side js like nodejs, but I expect the result is similar. Contact me if you need further more. About H.265 / HEVC, you need emscripten it partially of ffmpeg or x265 or OpenH265 similarily, with minimal bc size ( less than 1,2,3 M depending on config. ). Good luck...
I'm trying to write a program that uses GStreamer to connect to PulseAudio as a source so that I can intercept any audio that's being played. I have no need to play it back so my assumption is that my pipeline only needs a source and demuxer, I'm not sure of the latter though. The hello world example that I'm working off of is here, except that instead of using "filesrc" I'm using "pulsesrc".
Is there a good example that shows this out there already and I haven't found the right combination of terms to make Google give it to me? Do you have to do anything to PulseAudio to make it let you monitor its stream? Should I actually be trying to instead connect to a sink to monitor what's being played?
I think you will need to check the sources with e.g.:
pacmd list-sources | grep -e device.string -e 'name:'
and then connct to the source ending on ".monitor" by using the "device" property of pulsesrc.
I use the audio class to read MP3 file thanks to a little trick : replacing the ffmpegsumo.so of Node-Webkit with the chromium one. This enable the reading of MP3 on Windows but doesn't works on Mac OS. Does anyone know why ?
Here's the code :
player = new Audio()
player.src = '/path/to/the/audio.mp3';
player.play();
This seems to be dependant upon the dll/so being a 32 bit version. I am guessing that is why copying the file from Chrome doesn't work correctly for most people ( my 3 year old phone is the only 32-bit device I have left ).
I keep seeing this link --
https://github.com/rogerwang/node-webkit/wiki/Support-mp3-and-h264-in-video-and-audio-tag
.. but it is a blank page. I am guessing it was deleted since the info was likely not current or correct.
This issue thread has links to some rebuilt ffmpegsumo libraries for both Mac and Windows --
https://github.com/rogerwang/node-webkit/issues/1423
The alternative appears to be rebuilding ffmpegsumo, this thread has some config for doing that -- https://github.com/rogerwang/node-webkit/issues/1208
I am still confused about the licensing on it after you build the library, so that is probably worth some research. Everything about mpeg4-part10 is copyrighted and heavily patent encumbered. I think we all need to get smart enough to stop using mp4/h.264. Before I got this working correctly on node-webkit, it was easier to use ffmpeg to transcode the video to an ogv container using Theora and Vorbis codecs. At this point it seems like iOS is keeping h.264 alive, when it should probably die the horrible death it has earned.
I want to stream my webcam in linux with VLC to the iPod. From what I've seen on the web, the easiest way is to use a web server and then access to it from the iPod like this:
NSString *url = #"http://www.example.com/path/to/movie.mp4";
MPMoviePlayerController *moviePlayer = [[MPMoviePlayerController alloc] initWithContentURL:[NSURL URLWithString:url]];
[moviePlayer play];
I have never used web services before and would like to know how i can achieve this whole process. Thank you
EDIT: After setting up the linux/vlc/segmenter, this is what i get in the terminal after running the comment from Warren and exiting vlc:
VLC media player 1.1.4 The Luggage (revision exported)
Blocked: call to unsetenv("DBUS_ACTIVATION_ADDRESS")
Blocked: call to unsetenv("DBUS_ACTIVATION_BUS_TYPE")
[0x87bc914] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Blocked: call to setlocale(6, "")
Blocked: call to sigaction(17, 0xb71840d4, 0xb7184048)
Warning: call to signal(13, 0x1)
Warning: call to signal(13, 0x1)
Warning: call to srand(1309581991)
Warning: call to rand()
Blocked: call to setlocale(6, "")
(process:4398): Gtk-WARNING **: Locale not supported by C library.
Using the fallback 'C' locale.
Warning: call to signal(13, 0x1)
Warning: call to signal(13, 0x1)
Blocked: call to setlocale(6, "")
Could not open input file, make sure it is an mpegts file: -1
Help me understanding all this? thks!
The URL you show assumes the video is prerecorded.
For live HTTP streaming to an iOS device, the URL will instead end in .m3u or .m3u8, which is a common playlist format type. (It is an extended version of the Icecast playlist format, documented in this IETF draft.) This playlist tells the iOS device how to find the other files it will retrieve, in series, in order to stream the video.
The first tricky bit is producing the video stream. Unlike all other iOS compatible media files, live HTTP streaming requires an MPEG-2 transport stream (.ts) container, rather than an MPEG-4 part 14 container (.mp4, .m4v). The video codec is still H.264 and the audio AAC, as you might expect.
A command something like this should work:
$ vlc /dev/camera –intf=dummy –sout-transcode-audio-sync –sout='#transcode{\
vcodec=h264,venc=x264{\
aud,profile=baseline,level=30,keyint=30,bframes=0,ref=1,nocabac\
},\
acodec=mp4a,ab=56,deinterlace\
}:\
duplicate{dst=std{access=file,mux=ts,dst=-}}' > test.ts
This is all one long command. I've just broken it up for clarity, and to work around SO's formatting style limits. You can remove the backslashes and whitespace to make it a single long line, if you prefer. See the VLC Streaming HOWTO for help on figuring out what all that means, and how to adjust it.
The /dev/camera bit will probably have to be adjusted, and you may want to fiddle with the A/V encoding parameters based on Apple's best practices tech note (#TN 2224) to suit your target iOS device capabilites.
The next tricky bit is producing this playlist file and the video segment files from the live video feed.
Apple offers a program called mediastreamsgementer which does this, but it's not open source, it only runs on OS X, and it isn't even freely downloadable. (It comes as part of Snow Leopard, but otherwise, you have to be in Apple's developer program to download a copy.)
Chase Douglas has produced a basic segmenter which builds against libavformat from ffmpeg. There is a newer variant here which has various improvments.
To combine this with the vlc camera capture and encoding command above, replace the > test.ts part with something like this:
| segmenter - 10 test test.m3u8 http://www.example.com/path/to/
This pipes VLC's video output through the segmenter, which breaks the TS up into 10 second chunks and maintains the test.m3u8 playlist file that tells the iOS device how to find the segment files. The - argument tells the segmenter that the video stream is being piped into its standard input, rather than coming from a file. The URL fragment at the end gets prepended onto the file names mentioned in the M3U file.
Having done all that, the only adjustment that should be needed for your Cocoa Touch code is that it should be accessing test.m3u8 instead of movie.mp4.