I'm trying to view the MPEG-4 video stream from IP-camera which is sent by UDP multicast using gstreamer-java.
Each frame sent by camera is splitted into several UDP-packets.
After assembling complete frames from UDP-packets I get sequence of I-Frames and P-frames represented as byte arrays. I push each of these frames into "AppSrc" which is connected to "capsfilter" which is connected to "decodebin". The problem is: gstffmpegdec.c:2259:gst_ffmpegdec_frame: ffdec_mpeg4: decoding error (len: -1, have_data: 0).
I also tried the following chain: apprc => capsfilter => mpeg4videoparse => decodebin => videosink.
The result was the same.
What I must to do to workaround this problem?
Seems I found one way and now i'm testing it for stability. I have compiled from git
(git clone git://anongit.freedesktop.org/gstreamer/gst-plugins-good)
the gstreamer core, the "base", the "good" and the "bad" packages.
And then I applied the "xviddec" plugin which is now capable to process the buffers with caps "video/mpeg, mpegversion=4".
So the pipeline description is: "mpeg4videoparse name=parser ! xviddec ! ffmpegcolorspace ! capsfilter name=outfilter" which is connected to AppSrc object and VideoComponent object of "gstreamer-java" wrapper.
To specify the path to my compiled gstreamer binaries I pass the following JVM parameter:
"-Djna.library.path="path_to_my_home/bin/gstreamer/lib".
Note that compiling of "xviddec" requires the package "libxvidcore-dev" to be installed.
Related
I'm using SDL2 and the SDL mixer library to attempt to play a WAV file. After calling the Mix_LoadWAV_RW function, I get the following error string: "Unsupported block alignment" In case it matters, I'll note that I got the *SDL_RWops by calling SDL_RWFromMem on a WAV file I have embedded into my source code as an array of bytes. That is, the first several bytes are the WAV header containing the text "RIFF: and "WAVE".
I'm guessing this means I need to re-encode my WAV file to a sub-format with a supported block alignment, but that is mostly a guess. Given I go with that guess, it seems like I would need to know what block alignments are supported. I'm unable to find any relevant info in the docs about which alignments are supported.
I've also tried searching this github mirror of the source code for SDL mixer for "Unsupported block alignment" but I didn't get any results. Does that mean that the error is coming from some dependency of SDL Mixer or something?
Does anyone know more about this error, where I can find more information on this error, or even a reasonable guess as to how I should change my WAV file to work here?
For the record, I ended up just re-encoding the WAV file with default settings in Audacity, and it seems to have worked after that.
I'm trying to compile ffmpeg into javascript so that I can decode H.264 video streams using node. The streams are H.264 frames packed into RTP NALUs so any solution has to be able to accept H.264 frames rather than a whole file name. These frames can't be in a container like MP4 or AVI because then the demuxer needs to needs the timestamp of every frame before demuxing can occur, but I'm dealing with a real time stream, no containers.
Streaming H.264 over RTP
Below is the basic code I'm using to listen on a udp socket. Inside the 'message' callback the data packet is an RTP datagram. The data portion of the data gram is an H.264 frame (P-frames and I-frames).
var PORT = 33333;
var HOST = '127.0.0.1';
var dgram = require('dgram');
var server = dgram.createSocket('udp4');
server.on('listening', function () {
var address = server.address();
console.log('UDP Server listening on ' + address.address + ":" + address.port);
});
server.on('message', function (message, remote) {
console.log(remote.address + ':' + remote.port +' - ' + message);
frame = parse_rtp(message);
rgb_frame = some_library.decode_h264(frame); // This is what I need.
});
server.bind(PORT, HOST);
I found the Broadway.js library, but I couldn't get it working and it doesn't handle P-frames which I need. I also found ffmpeg.js, but could get that to work and it needs a whole file not a stream. Likewise, fluent-ffmpeg doesn't appear to support file streams; all of the examples show a filename being passed to the constructor. So I decided to write my own API.
My current solution attempt
I have been able to compile ffmpeg into one big js file, but I can't use it like that. I want to write an API around ffmpeg and then expose those functions to JS. So it seems to me like I need to do the following:
Compile ffmpeg components (avcodec, avutil, etc.) into llvm bitcode.
Write a C wrapper that exposes the decoding functionality and uses EMSCRIPTEN_KEEPALIVE.
Use emcc to compile the wrapper and link it to the bitcode created in step 1.
I found WASM+ffmpeg, but it's in Chinese and some of the steps aren't clear. In particular there is this step:
emcc web.c process.c ../lib/libavformat.bc ../lib/libavcodec.bc ../lib/libswscale.bc ../lib/libswresample.bc ../lib/libavutil.bc \
:( Where I think I'm stuck
I don't understand how all the ffmpeg components get compiled into separate *.bc files. I followed the emmake commands in that article and I end up with one big .bc file.
2 questions
1. Does anyone know the steps to compile ffmpeg using emscripten so that I can expose some API to javascript?
2. Is there a better way (with decent documentation/examples) to decode h264 video streams using node?
To question 1:
Just follow the official doc
Consider the case where you normally build with the following
commands:
./configure
make
To build with Emscripten, you would instead use the following
commands:
# Run emconfigure with the normal configure command as an argument.
./emconfigure ./configure
# Run emmake with the normal make to generate linked LLVM bitcode.
./emmake make
# Compile the linked bitcode generated by make (project.bc) to JavaScript.
# 'project.bc' should be replaced with the make output for your project (e.g. 'yourproject.so')
# [-Ox] represents build optimisations (discussed in the next section).
./emcc [-Ox] project.bc -o project.js
To question 2: c/c++ libs can be called in a node environment. You can write some c/c++ glue code or use a proxy node module like node-ffi.
Using node-ffi to call exist libs may be easier.
May it help :)
Easiest way, specially if you need to run it in web browsers, is to utilize Media Source Extension. I managed to do it in just 3 days. Moreover, it is used automatically GPU hardware ( Cuda, Intel Qsv, ... ) acceleration as far as browser builtin support. It is important if it runs in the real World app. I have tested yesterday, just 5% of my old i7 machine CPU to decode 4K ( 4 times more than 1080p ) IP Camera H.264 nal raw live streaming. I am not sure it under server side js like nodejs, but I expect the result is similar. Contact me if you need further more. About H.265 / HEVC, you need emscripten it partially of ffmpeg or x265 or OpenH265 similarily, with minimal bc size ( less than 1,2,3 M depending on config. ). Good luck...
im new at this forum and my english skills are not the best!
I have a website where i publish the videostreams of the cameras to show what happens inside during the nesting-time live! An guy with high IT-skills has build me a little Server for Restream it (Datarhei-Restreamer) But this guy has still no time and worse response-times...
To my Problem: The Restreamer dont support the "G.711" Audio-Codec from the cameras and the Livestream are still without audio at the website. So, i need to convert the Livestreams (RTSP and RTMP- in H.264) so that the audio changes to "aac" or something other supported. But i have no plan how to do this. I tried it with FFMPEG but i dont find the correct commands to get the my result. There is something with an Streaming-server to send the new created stream to - i dont get it into my head to do this (i need just a stream that are viewable with VLC player and then as input for my restreamer-server, jsut the same like ca
I want to change the source-stream into the correct codec (audio from G.711 to AAC, the rest like source) and then, put this "new" stream into my Restreamer-Server and it will work fine! (Tested with XSplitbroadcaster, but dont runs on Raspberry, only 1 instance runable but 2 livestreams needs to be encoded at same time) And this programm has annoying bugs (endless and not removeable error-messages, but running stream)
I have a new second raspberry that are planned as "live-encoder" for the restreamer-raspberry were the "new" streams are are going in (rtmp/rtsp-input on a graphical ui) I try it still with FFMPEG but still no result...
Sorry about this long text with all the language-issues but im really frustrated with it because i have purchased 2 new cameras with total 450 euros just to get the livestream with sound now :(
Finally, I found the best solution here and it works (https://github.com/datarhei/restreamer/issues/11). Inside the long discussion, use the solution written by svenerbeck on 4 Apr 2016. The essential part is written below.
Create a new live.json in /mnt/live.json with the upcoming modification:
"ffmpeg": {
"options": {
"native_h264": [
"-vcodec copy",
"-acodec aac",
"-f flv"
],
.....
Exec the container with
docker run ... -v /mnt/live.json:/restreamer/conf/live.json ....
I use the audio class to read MP3 file thanks to a little trick : replacing the ffmpegsumo.so of Node-Webkit with the chromium one. This enable the reading of MP3 on Windows but doesn't works on Mac OS. Does anyone know why ?
Here's the code :
player = new Audio()
player.src = '/path/to/the/audio.mp3';
player.play();
This seems to be dependant upon the dll/so being a 32 bit version. I am guessing that is why copying the file from Chrome doesn't work correctly for most people ( my 3 year old phone is the only 32-bit device I have left ).
I keep seeing this link --
https://github.com/rogerwang/node-webkit/wiki/Support-mp3-and-h264-in-video-and-audio-tag
.. but it is a blank page. I am guessing it was deleted since the info was likely not current or correct.
This issue thread has links to some rebuilt ffmpegsumo libraries for both Mac and Windows --
https://github.com/rogerwang/node-webkit/issues/1423
The alternative appears to be rebuilding ffmpegsumo, this thread has some config for doing that -- https://github.com/rogerwang/node-webkit/issues/1208
I am still confused about the licensing on it after you build the library, so that is probably worth some research. Everything about mpeg4-part10 is copyrighted and heavily patent encumbered. I think we all need to get smart enough to stop using mp4/h.264. Before I got this working correctly on node-webkit, it was easier to use ffmpeg to transcode the video to an ogv container using Theora and Vorbis codecs. At this point it seems like iOS is keeping h.264 alive, when it should probably die the horrible death it has earned.
I want to stream my webcam in linux with VLC to the iPod. From what I've seen on the web, the easiest way is to use a web server and then access to it from the iPod like this:
NSString *url = #"http://www.example.com/path/to/movie.mp4";
MPMoviePlayerController *moviePlayer = [[MPMoviePlayerController alloc] initWithContentURL:[NSURL URLWithString:url]];
[moviePlayer play];
I have never used web services before and would like to know how i can achieve this whole process. Thank you
EDIT: After setting up the linux/vlc/segmenter, this is what i get in the terminal after running the comment from Warren and exiting vlc:
VLC media player 1.1.4 The Luggage (revision exported)
Blocked: call to unsetenv("DBUS_ACTIVATION_ADDRESS")
Blocked: call to unsetenv("DBUS_ACTIVATION_BUS_TYPE")
[0x87bc914] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Blocked: call to setlocale(6, "")
Blocked: call to sigaction(17, 0xb71840d4, 0xb7184048)
Warning: call to signal(13, 0x1)
Warning: call to signal(13, 0x1)
Warning: call to srand(1309581991)
Warning: call to rand()
Blocked: call to setlocale(6, "")
(process:4398): Gtk-WARNING **: Locale not supported by C library.
Using the fallback 'C' locale.
Warning: call to signal(13, 0x1)
Warning: call to signal(13, 0x1)
Blocked: call to setlocale(6, "")
Could not open input file, make sure it is an mpegts file: -1
Help me understanding all this? thks!
The URL you show assumes the video is prerecorded.
For live HTTP streaming to an iOS device, the URL will instead end in .m3u or .m3u8, which is a common playlist format type. (It is an extended version of the Icecast playlist format, documented in this IETF draft.) This playlist tells the iOS device how to find the other files it will retrieve, in series, in order to stream the video.
The first tricky bit is producing the video stream. Unlike all other iOS compatible media files, live HTTP streaming requires an MPEG-2 transport stream (.ts) container, rather than an MPEG-4 part 14 container (.mp4, .m4v). The video codec is still H.264 and the audio AAC, as you might expect.
A command something like this should work:
$ vlc /dev/camera –intf=dummy –sout-transcode-audio-sync –sout='#transcode{\
vcodec=h264,venc=x264{\
aud,profile=baseline,level=30,keyint=30,bframes=0,ref=1,nocabac\
},\
acodec=mp4a,ab=56,deinterlace\
}:\
duplicate{dst=std{access=file,mux=ts,dst=-}}' > test.ts
This is all one long command. I've just broken it up for clarity, and to work around SO's formatting style limits. You can remove the backslashes and whitespace to make it a single long line, if you prefer. See the VLC Streaming HOWTO for help on figuring out what all that means, and how to adjust it.
The /dev/camera bit will probably have to be adjusted, and you may want to fiddle with the A/V encoding parameters based on Apple's best practices tech note (#TN 2224) to suit your target iOS device capabilites.
The next tricky bit is producing this playlist file and the video segment files from the live video feed.
Apple offers a program called mediastreamsgementer which does this, but it's not open source, it only runs on OS X, and it isn't even freely downloadable. (It comes as part of Snow Leopard, but otherwise, you have to be in Apple's developer program to download a copy.)
Chase Douglas has produced a basic segmenter which builds against libavformat from ffmpeg. There is a newer variant here which has various improvments.
To combine this with the vlc camera capture and encoding command above, replace the > test.ts part with something like this:
| segmenter - 10 test test.m3u8 http://www.example.com/path/to/
This pipes VLC's video output through the segmenter, which breaks the TS up into 10 second chunks and maintains the test.m3u8 playlist file that tells the iOS device how to find the segment files. The - argument tells the segmenter that the video stream is being piped into its standard input, rather than coming from a file. The URL fragment at the end gets prepended onto the file names mentioned in the M3U file.
Having done all that, the only adjustment that should be needed for your Cocoa Touch code is that it should be accessing test.m3u8 instead of movie.mp4.