I'm trying to extract mpeg4 from an rtp payload , format of the rtsp media (video) is MP4V-ES but I'm not able to extract the mp4 from the payload .
when I dump the extract into a raw file and use ffmpeg to convert it into .avi or .mpg its not working. I don't know what I'm missing here.
the code is written in java.
I want to extract each video frame from the rtp and save that in a file or retransmit it.
Thanks
Question UPDATED.....
Thanks for the inputs, actually I'm able to extract bytes from 000001b6 and sent it to ffmpeg , but it complains about not header information and then I constructed a header with 000001b0 00000000 and 000001B5 00000005 and sent that to the ffmpeg but no luck. can you help me here, because what I'm getting from RTP is 000001b6 [data] and again 000001b6 [data] I even followed 'Cipi' to just add 000001 but not working. am I missing something here ! and also want to know whether I need to decode/encode as what I get from the RTP is the actual mpeg4 data right then I don't know why to decode it , can I just save it a file and open with quicktime or VLC and it shoud show one frame right.
MPEG-4 Video RTP payload is described in RTP Payload Format for MPEG-4 Audio/Visual Streams.
See also How to process raw UDP packets so that they can be decoded by a decoder filter in a directshow source filter with a brief description of steps you need to reconstruct the MPEG-4 video stream.
Update: You might need to look for B0 and B5 in your session descriptor. So that you know where to look for, here is an example of MPEG-4 RTSP/SDP response:
RTSP/1.0 200 OK
CSeq: 2
Content-Base: rtsp://192.168.0.57/webcam/
Content-Type: application/sdp
Content-Length: 320
v=0
o=- 1 1 IN IP4 127.0.0.1
s=Test
a=type:broadcast
t=0 0
c=IN IP4 0.0.0.0
m=video 0 RTP/AVP 96
a=rtpmap:96 MP4V-ES/90000
a=fmtp:96 profile-level-id=1;config=000001B003000001B509000001000000012000C488BA9850584121463F
a=control:track0
m=audio 0 RTP/AVP 97
a=rtpmap:97 AMR/8000/1
a=fmtp:97
a=control:track1
Related
Similar to other questions I've seen I find that I cannot stream my HTTPS (and password protected via JWT/server-side auth) m3u8 TS streams to AirPlay. The files are being served via my own API/web app using HTML.
My HLS stream play fine on my iPhone (iPhone 8 w/ latest iOS at time of writing), but it's AirPlay that seems to fail. Apple's example m3u8 HTTP video stream files for TS and MP4 work as expected from my phone to AirPlay, but my own m3u8 file does not.
I imagine the answer may be within Apple's HTTP Live Streaming (HLS) specification documentation or their WWDC talk on HLS authoring, but I find the reference materials too opaque to follow as a person who is new to HLS streaming in general.
I've tried explicitly opting in to AirPlay using x-webkit-airplay="allow" on the video element as well as switching the server response Content-Type to application/vnd.apple.mpegurl from application/x-mpegURL, but with no success.
I'm using hls.js to play the videos, and the videos were transcoded like so using ffmpeg.
ffmpeg -i "${1}" \
-filter_complex "[0:v]split=1[v1]; [v1]scale=w=1080:h=-1[v1out]" \
-map "[v1out]" -c:v:2 libx264 -x264-params "nal-hrd=cbr:force-cfr=1" -b:v:2 5M -maxrate:v:2 5M -minrate:v:2 5M -bufsize:v:2 10M -preset slow -g 48 -sc_threshold 0 -keyint_min 48 \
-map "a:0" -c:a:0 aac -b:a:0 96k -ac 2 \
-f hls \
-hls_time 2 \
-hls_playlist_type vod \
-hls_flags independent_segments \
-hls_segment_type mpegts \
-hls_segment_filename "${2}/${3}/data%02d.ts" \
-var_stream_map "v:0,a:0" "${2}/${3}/stream.m3u8"
My iPhone seemingly shows that the video is currently playing, but AirPlay shows no signs that anything is being played.
Here is a snippet of my m3u8 file...
#EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:2
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-INDEPENDENT-SEGMENTS
#EXTINF:2.400000,
data00.ts
#EXTINF:1.600000,
data01.ts
#EXTINF:2.400000,
data02.ts
#EXTINF:1.60
...
...vs Apple's example...
#EXTM3U
#EXT-X-VERSION:6
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2218327,BANDWIDTH=2227464,CODECS="avc1.640020,mp4a.40.2",RESOLUTION=960x540,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc1",AUDIO="aud1",SUBTITLES="sub1"
v5/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=8144656,BANDWIDTH=8178040,CODECS="avc1.64002a,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc1",AUDIO="aud1",SUBTITLES="sub1"
v9/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=6307144,BANDWIDTH=6453202,CODECS="avc1.64002a,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc1",AUDIO="aud1",SUBTITLES="sub1"
v8/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=4775338,BANDWIDTH=5054232,CODECS="avc1.64002a,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc1",AUDIO="aud1",SUBTITLES="sub1"
v7/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=3240596,BANDWIDTH=3289288,CODECS="avc1.640020,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc1",AUDIO="aud1",SUBTITLES="sub1"
v6/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1292926,BANDWIDTH=1296989,CODECS="avc1.64001e,mp4a.40.2",RESOLUTION=768x432,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc1",AUDIO="aud1",SUBTITLES="sub1"
v4/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=914722,BANDWIDTH=922242,CODECS="avc1.64001e,mp4a.40.2",RESOLUTION=640x360,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc1",AUDIO="aud1",SUBTITLES="sub1"
v3/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=541239,BANDWIDTH=553010,CODECS="avc1.640015,mp4a.40.2",RESOLUTION=480x270,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc1",AUDIO="aud1",SUBTITLES="sub1"
v2/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2439704,BANDWIDTH=2448841,CODECS="avc1.640020,ac-3",RESOLUTION=960x540,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc1",AUDIO="aud2",SUBTITLES="sub1"
v5/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=8366033,BANDWIDTH=8399417,CODECS="avc1.64002a,ac-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc1",AUDIO="aud2",SUBTITLES="sub1"
v9/prog_index.m3u8
...
The issue is when the device playing the video tries to play it what it really does in the end is load the master.m3u8 playlist and then play the video. Since your m3u8 is protected the device does not have permission to load it. And the cookies in the main browser are not passed to the tv device so in reality it works as expected: a person you don’t know (the tv) is trying to play the video.
Look for ways to only protect the main playlist with a signature in the url itself and then having a middle ware creating cookies to protect the rest of the segments.
We have that working with AWS cloud front and lambda at edge
I am uploading videos to Youtube, and on my Android phone I am downloading them using Youtube Red. I am playing these downloaded videos in the background, when the screen is off.
This works with the wast majority of the videos, except the ones that I am uploading.
I did read the recommended upload formats, I tried several codecs, but no luck. My audios stop the second I am shutting off the screen.
What I finally found using youtube-dl -F is that my videos do not have audio-only tracks with webm extension, only as m4a (after Youtube processed them).
So my question is: what makes Youtube create webm audio files for some videos, but not for the others? Is there a way to force this (I suppose not). Is there a way to suggest it? As I mentioned, I tried a wide variety of codecs - video and audio, and the combinations - when generating my files to be uplaoded.
A sample output for a file which works:
format code extension resolution note
249 webm audio only DASH audio 52k , opus # 50k, 73.58KiB
250 webm audio only DASH audio 66k , opus # 70k, 92.62KiB
251 webm audio only DASH audio 114k , opus #160k, 161.14KiB
171 webm audio only DASH audio 115k , vorbis#128k, 161.27KiB
140 m4a audio only DASH audio 127k , m4a_dash container, mp4a.40.2#128k, 180.79KiB
and the output for one file which does not:
format code extension resolution note
139 m4a audio only DASH audio 49k , m4a_dash container, mp4a.40.5# 48k (22050Hz), 1.20MiB
140 m4a audio only DASH audio 129k , m4a_dash container, mp4a.40.2#128k (44100Hz), 3.20MiB
There are a number of audio files that have .m4a suffix and these are encoded in one of AAC or Apple Lossless (ALAC). I want to choose only audio files encoded in Apple Lossless of them. Is there any way to determine this? I tried FFmpeg, but it says all of them are encoded in AAC.
Edit: I am currently on Windows.
If you have the FFmpeg package, you should have ffprobe.
Give this a try:
ffprobe -v error -select_streams a:0 -show_entries stream=codec_name -of default=noprint_wrappers=1:nokey=1 file.m4a
-v error: to hide the startup text
-select_streams a:0: to select the first audio track
-show_entries stream=codec_name: to display only the codec type
-of default=noprint_wrappers=1:nokey=1: to remove extra formatting
This will print out just aac or alac. Perfect for scripting.
Here is a file that has a description of M4A (best I could find so far) on page 67:
http://iweb.dl.sourceforge.net/project/audiotools/audio%20formats%20reference/2.14/audioformats_2.14_letter.pdf
A typical M4A begins with an 'ftyp' atom indicating its file type...
10.2.1 the ftyp atom
[0 31] ftyp Length [32 63] 'ftyp' (0x66747970)
[64 95] Major Brand [96 127] Major Brand Version
[128 159] Compatible Brand₁ ...
The 'Major Brand' and 'Compatible Brand' elds are ASCII strings.
'Major Brand Version' is an integer.
At first I figured 'ftyp' would be where format is determined, but judging by this list that is more like the file type itself (already known as m4a):
http://www.ftyps.com/index.html
http://www.ftyps.com/what.html Describes a bit more of the format.
If ftyp doesn't differentiate, then I think that the 'Major Brand' field might refer to the fourcc's on this page:
http://wiki.multimedia.cx/index.php?title=QuickTime_container
The one for Apple Lossless being 'alac' and AAC is probably 'mp4a'
Apple's Lossless format open source page indicates that the ftype is 'alac' (slightly contradictory to above)
http://alac.macosforge.org/trac/browser/trunk/ALACMagicCookieDescription.txt
So far what I can tell is that the 4 bytes following ftyp are always (in a smallish sample size) 'M4A '.
Somewhere in the first ~200 (hex) bytes or so there is an ascii 'mp4a' for AAC compression or an 'alac' for Apple Lossless. The 'alac' always seems to come in pairs ~30 bytes apart ('mp4a' only once).
Sorry that's not more specific, if I find the exact location or prefix I'll update again. (My guess is the earlier part of the header has a size specified somewhere.)
You can do it with Core Audio.
Something like:
CFStringRef pathToFile;
CFURLRef inputFileURL = CFURLCreateWithFileSystemPath(kCFAllocatorDefault, pathToFile, kCFURLPOSIXPathStyle, false);
ExtAudioFileRef inputFile;
ExtAudioFileOpenURL(inputFileURL, &inputFile);
AudioStreamBasicDescription fileDescription;
UInt32 propertySize = sizeof(fileDescription);
ExtAudioFileGetProperty(inputFile,
kExtAudioFileProperty_FileDataFormat,
&propertySize,
&fileDescription);
if(fileDescription.mFormatID == kAudioFormatAppleLossless){
// file is apple lossless
}
On a Mac, you select the file you want and then right click. Find "Get Info" and click that and a window will pop up with extra information about the file you selected. It should say next to "Codecs:" "AAC" or "Apple Lossless"
I hope I helped those Mac users out there that had the same question (and possibly Windows users in some way even though I am not familiar with the OS.)
try using http://sourceforge.net/projects/mediainfo/
"MediaInfo is a convenient unified display of the most relevant technical and tag data for video and audio files." - sourceforge project description
This is how info is displayed.
General
Complete name : C:\Downloads\recit24bit.m4a
Format : MPEG-4
Format profile : Apple audio with iTunes info
Codec ID : M4A
File size : 2.62 MiB
Duration : 9s 9ms
Overall bit rate : 2 441 Kbps
Track name : 24 bit recital ALAC Test File
Performer : N\A
Comment : Test File
Audio
ID : 1
Format : ALAC
Codec ID : alac
Codec ID/Info : Apple Lossless Format
Duration : 9s 9ms
Bit rate mode : Variable
Bit rate : 2 438 Kbps
Channel(s) : 2 channels
Sampling rate : 22.7 KHz
Bit depth : 24 bits
Stream size : 2.62 MiB (100%)
Language : English
Check audio section for codec/encoding details.
Im trying to extract each frame from a rtsp mp4 stream, and convert that into a jpeg/gif using ffmpeg. I'm getting the sdp header from 000001b0.....000001b5, and adding that into an byte array then capturing a frame starting from 000001b6 and appending it to the byte array.
When I flush it to a file (.mpg) and use ffmpeg it throws errors and not converting.
my header looks like 000001B008000001B58913000001000000012000C488BA98514043C1463F and after this I'm appending a frame (starting from 000001b6).
I did something similar with FFMPEG, and it seems that the frame data you get from FFMPEG already contains the frame header, which is all you need to transcode the data. Please make sure that you decode the mp4 data to a raw format (RGB24 for instance), then encode it to the pixelformat the JPEG/GIF encoder expects (probably a YUV format) using libswscale, before passing the data to the encoder.
Depending on the Codec you may not have to add anything or you may have to add a lot..
This is referred to as de-packetization and MPEG4-ES has no packetization model... H264 has many depending on the profile.
Check out the RFC..
Either 3016 or 3640 should help you.
https://www.rfc-editor.org/rfc/rfc3640
https://www.rfc-editor.org/rfc/rfc3016
Greetings fellow amazonians
We have certain f4v files from webtvinteractive.com that are not working from a streaming distribution. The cloudfront diagnostic tool indicates the file is not found but from a download distribution they play all ok fine ??
When inspecting the codec the only difference we see is that the audio is encoded as MPEG audio layer 1/2/3 (mpga) instead of AAC (mp4a)
Also noticed that VP6F is used but that is a Abobe proprietary codec so it should stream all ok from Flash Media Server
Here is the test tool url:
http://d1k5ny0m6d4zlj.cloudfront.net/diag/CFStreamingDiag.html
s3dbhkulzq9ndp.cloudfront.net
teaser.f4v (ok)
EM2_H264.f4v (NOT ok)
StreamLen Response: 0 ??
Received an answer from AWS team member
Posted by: RonS#AWS
Posted on: Oct 10, 2011 7:29 AM
https://forums.aws.amazon.com/message.jspa?messageID=284676
Hi, I downloaded both of your video objects and ran flvcheck (http://help.adobe.com/en_US/FlashMediaServer/3.5_AdminGuide/WS5b3ccc516d4fbf351e63e3d119f29261b7-7ffc.html) against them.
The teaser.f4v is apparently OK according to flvcheck, but EM2_H264 gets this:
11-10-10 07:10:25 Error: -26 File appears to be FLV with wrong extension. EM2_H264.f4v
If I rename the file to have a .flv extension, I get "Found backward timestamp".
I suspect the FMS software we're using will have the same complaints about that file that flvcheck does.
flvcheck is available for download here (you'll need to register with Adobe): http://www.adobe.com/products/flashmediaserver/tool_downloads/