How to find resolution and framerate values in H.264 MPEG-2 TS? - resolution

Question

I'm working on a MPEG-2 TS video containing a H.264 stream, and I'm looking for video properties stored in the stream, by scanning PAT, PMT, PES, etc.
I'm able to read PAT, PMT, and elementary streams type and PID. Here I would like to find the resolution and the framerate (fps). Are they located in the PES header, or elsewhere? They are not in PAT or PMT.
Below, Transport Stream Packet Editor is able to find two different informations, one itself and the other from the Haali Media Decoder helper codec. How to get the first one:
Pseudo-code is welcomed.

Answer 1

I am not sure about the availability of the height width information in the MPEG2TS header. Because TS file can have multiple programs. But if you are targeting only TS files made of H.264 elementary stream then you an get these informations from the SPS of the H.264 elementary stream.
Every H.264 frame starts with four or three bytes sequence header 0x00 0x00 0x01 or 0x00 0x00 0x00 0x01. The frame is an SPS frame if doing the AND operation with next byte after start headers is equal to 0x07.
E.g. SPS frame 0x00 0x00 0x00 0x01 0x67 ... Doing AND operation (0x67 & 0x1F) = 0x07
Parsing the SPS header is also not an easy task but you can find the details in ffmpeg source code.
Hope this helps.

Answer 2

No, they were not present in PES header. To find resolution and frame rate from H.264 video in MPEG2-TS you need to parse SPS(Sequence parameter set)from H.264 stream.
These are the steps for parsing H.264 NAL(Network adaption layer) units:
Parse NAL unit prefix(NAL unit prefix is of 3(0x00,0x00,0x01) or 4(0x00,0x00,0x00,0x01) byte code) then Header(next byte after prefix code)
Check the type of NAL unit(last 5 bits) from Header byte.
If NAL unit is of type 7 means,this NAL unit is SPS NAL unit then parse the code
This ITU link gives the documentation about h.264 standard.
See section 7.3.2.1.1: Sequence parameter set data syntax gives the syntax to find the parameters in SPS.

Answer 3

I presume working code for this is resident inside of the ffprobe binary for the FFMPEG project, as it produces the desired output:
$ ffprobe -v quiet -show_streams output1.mp4
[STREAM]
index=0
codec_name=h264
... // A bunch of stream data
width=1280
height=1024
sample_aspect_ratio=1:1
display_aspect_ratio=5:4
....
r_frame_rate=30000/1001
avg_frame_rate=30000/1001
time_base=1/30000
...
[/STREAM]

Answer 4

The information you are looking for is inside the H.264 SPS NAL units.
You need to parse the PES data, extract the NALUs and then parse the SPS data. There you'll find the resolution. If SPS carries VUI information you have information about the desired frame rate.
MPEG2-TS is a transport stream, it transports something but does not carry detailed information about what it carries. It just wraps stuff.
What you could use from MPEG2-TS is PTS/DTS of the PES header and average the frame rate from the presentation time stamps provided.
To do it properly, parse the PES header, parse the NALU headers, parse the actual SPS NAL unit and if present the VUI it contains.