Apple recommends having hls segments of 10 seconds, however this means that seeking would be limited to every 10 seconds.
I have tried shorter segments of 3 seconds and this is better for seeking but this is not ideal or recommended.
Is there anyway of keeping the segments at 10 seconds but allowing for better seeking?
Would adding a key frame every 30 frames (1 key frames every second) allow for better seeking?
Ultimately, it depends upon your player. If I'm watching a video with the default iPad player, I can navigate via the progress bar on the bottom and seek to any point in the video, and it works very well, regardless of segment length or key frame cadence.
Some players support the #EXT-X-I-FRAMES-ONLY attribute. This implementation of trick play works by only playing back the intra frames. This was introduced in version 4 of the Pantos spec, and I have only seen it working well on newer iPads. A good sample clip can be found here.
Related
as I said in the title, I need to record my screen from an electron app.
my needs are:
high quality (720p or 1080p)
minimum size
record audio + screen + mic
low impact on PC hardware while recording
no need for any wait after the recorder stopped
by minimum size I mean about 400MB on 720p and 700MB on 1080p for a 3 to 4 hours recording. we already could achieve this by bandicam and obs and it's possible
I already tried:
the simple MediaStreamRecorder API using RecordRTC.Js; produces huge file sizes, like 1GB per hour for 720p video.
compressing the output video using FFmpeg; it can take up to 1 hour for 3 hours recording
save every chunk with 'ondataavailable' event and right after, run FFmpeg and convert and reduce the size and append all the compressed files (also by FFmpeg); there are two problems. 1, because of different PTS but it can be fixed by tunning compress command args. 2, the main problem is the audio data headers are only available in the first chunk and this approach causes a video that only has audio for the first few seconds
recording the video with FFmpeg itself; the end-users need to change some things manually (Stereo Mix), the configs are too complex, it causes the whole PC to work slower while recording (like fps drop; even if I set -threads to 1), in some cases after recording is finished it needs many times to wrap it all up
searched through the internet to find applications that can be used from the command line; I couldn't find much, the famous applications like bandicam and obs have command line args but there are not many args to play with and I can't set many options which leads to other problems
I don't know what else I can do, please tell me if u know a way or simple tool that can be used through CLI to achieve this and guide me through this
I end up using the portable mode of high-level 3d-party applications like obs-studio and adding them to our final package. I also created a js file to control the application using CLI
this way I could pre-set my options (such as crf value, etc) and now our average output size for a 3:30 hour value with 1080p resolution is about 700MB which is impressive
I have been analyzing the JSON file generated using chrome://webrtc-internal, while running webrtc on 2 PCS.
I looked at Stats API to verify how webrtc-internal computes the keyframe rate.
By looking at Stats API/ RTC Remote Inbound RTP Video Stream, it contains keyFramesDecoded which represents the total number of key frames, such as key frames in VP8, given that I set the codec to VP8.
keyFramesDecoded values are very small, i.e., 2 for a couple of minutes, and similarly for 3 and ...
My question is: How does the graph here make sense for keyFramesDecoded?
That looks right to me.
Chrome is configured to send a keyframe every 3000 frames. That means for 30fps you will see a keyframe every 100 seconds. The framesDecoded is being construct by lots of delta frames.
If you are in a unconstrained network and not dealing with lots of change in your video I would expect to see graphs like yours.
I need to play 4 audios through a browser web.
These audios last 150ms, 300ms, 450ms and 600ms.
I don't care about latency (if an audio is played 100 ms after it's not that important for my purpose).
But I do care about the duration of these audios: is the 150ms audio last exactly 150ms or there is an error due to the audio board or other components?
I know for sure that there is an error (I see a test using a Mac).
My question is: can anyone show me a paper, an article or anything that talks about the duration and test different setting or tell me if this error is always (Windows, Mac, old device, new device) very small (less than 10ms for example).
In other words: if I play an audio of 100ms how long does it really last (100ms? more? less?)?
In what manner is the sound not lasting the correct amount of time?
Does the beginning or the end get cut off?
Does the sound play back slower or faster than it should?
In my experience, I've never heard an error with playback rates caused by the browser or sound boards. But I have come across situations where a sound is played back with a different audio format than which is was encoded. For example, a sound encoded at 48000 fps played back at 44100 fps will take longer to execute, but will be very close to the original in pitch (maybe about a 1/2 step lower). I recommend as a diagnostic step to confirm the audio format used at each end. How to do so will depend on the systems being used.
I want to built a SoundWave sampling an audio stream.
I read that a good method is to get amplitude of the audio stream and represent it with a Polygon. But, suppose we have and AudioGraph with just a DeviceInputNode and a FileOutpuNode (a simple recorder).
How can I get the amplitude from a node of the AudioGraph?
What is the best way to periodize this sampling? Is a DispatcherTimer good enough?
Any help will be appreciated.
First, everything you care about is kind of here:
uwp AudioGraph audio processing
But since you have a different starting point, I'll explain some more core things.
An AudioGraph node is already periodized for you -- it's generally how audio works. I think Win10 defaults to periods of 10ms and/or 20ms, but this can be set (theoretically) via the AudioGraphSettings.DesiredSamplesPerQuantum setting, with the AudioGraphSettings.QuantumSizeSelectionMode = QuantumSizeSelectionMode.ClosestToDesired; I believe the success of this functionality actually depends on your audio hardware and not the OS specifically. My PC can only do 480 and 960. This number is how many samples of the audio signal to accumulate per channel (mono is one channel, stereo is two channels, etc...), and this number will also set the callback timing as a by-product.
Win10 and most devices default to 48000Hz sample rate, which means they are measuring/output data that many times per second. So with my QuantumSize of 480 for every frame of audio, i am getting 48000/480 or 100 frames every second, which means i'm getting them every 10 milliseconds by default. If you set your quantum to 960 samples per frame, you would get 50 frames every second, or a frame every 20ms.
To get a callback into that frame of audio every quantum, you need to register an event into the AudioGraph.QuantumProcessed handler. You can directly reference the link above for how to do that.
So by default, a frame of data is stored in an array of 480 floats from [-1,+1]. And to get the amplitude, you just average the absolute value of this data.
This part, including handling multiple channels of audio, is explained more thoroughly in my other post.
Have fun!
I am inserting timed metadata in a HLS (HTTP Live Stream) using id3taggenerator and mediafilesegmenter. I have followed the instructions from Jake's Blog.
First, I create the id3tag using id3taggenerator:
id3taggenerator -o text.id3 -t "video"
Then add the tag to the id3macro file:
0 id3 /path/to/file/text.id3
And segment the video and insert the id3 tags with mediafilesegmenter:
mediafilesegmenter -M /path/to/id3macro -I -B "my_video" video.mp4
However, the timed metadata is inserted at the wrong point in time. Instead of showing up at the beginning of the video (point in time 0), it is added with a delay of 10 s (give or take 0.05 seconds, sometimes more, sometimes less).
I've wrote a simple iOS player app that logs whenever it is notified of an id3 tag in the video. The app is notified after playing the video for around 10 seconds of the ID3 tag. I've also tried with another id3macro file, with multiple timed metadata inserted in the video (around 0s, 5s, 7s), all showing up with the same approximate delay. I have also changed with the duration of the segment to 5s, but each time it's the same result.
The mediafilesegmenter I am using is Beta Version 1.1(140602).
Can anyone else confirm this problem, or pin-point to what am I doing wrong here?
Cheers!
I can confirm that I experience the same issue, using the same version of mediafilesegmenter:
mediafilesegmenter: Beta Version 1.1(140602)
Moreover, I can see that the packet with ID3 is inserted in the right moment in the stream. Eg. if I specify a 10 second delay – I can see that my ID3 is inserted in the end of the first 10 second segment.
However, it appears 10 seconds later in iOS notifications.
I can see the following possible reasons:
mediafilesegmenter inserts metadata packet in the right place, but timestamp is delayed by 10 seconds for some reason. Therefore, clients (eg. iOS player) show the tag 10 seconds later. Apple tools are not well documented so it's hard to verify.
Maybe iOS player receives metadata in time (because I know the tag was included in previous segment file) but issues a notification with 10 second delay, for whatever reason.
I cannot dig further because I don't have any Flash/desktop HLS players that support in-stream ID3 tags. If I had one, I would check whether desktop player will display/process ID3 in time, without delay. Then, it would mean the problem is iOS, not mediafilesegmenter.
Another useful thing to do would be – extracting MPEG-TS frame with ID3 tag from the segment file, and checking headers, looking for any strange things there (eg. wrong timestamp).
Update:
I did some more research including reverse engineering of TS segments created with Apple tools, and it seems:
mediafilesegmenter starts PTS (presentation time stamps) from 10 seconds while, for example, ffmpeg starts from 0.
mediafilesegmenter adds ID3 frame at the correct place in TS file but with wrong PTS that is 10 seconds ahead of what was specified in meta file.
While the first issue doesn't seem to affect the playback (as far as I understand it's more important that PTS goes on continuously, not where it starts), the second is definitely an issue and the reason you/we are experiencing the problem.
Therefore, iOS player receives ID3 frame in time but since its PTS is 10 seconds ahead – it waits 10 seconds before issuing a notification. As far as I can say now – some other players simply ignore this ID3 frame because it's in the wrong place.
As a workaround, you can shift all ID3 files by 10 seconds in your meta file, but obviously, you won't be able to put anything in the beginning.