Supplemental enhancement information carries data that is not needed to decode the pictures: timing, HDR metadata, captions, checksums. H.265 has two SEI NAL unit types: prefix SEI (39), placed before the first slice of the access unit, and suffix SEI (40), placed after it. Each SEI NAL unit holds one or more messages, then the trailing bits. The messages are defined in Annex D of H.265; some come from Annexes F and G, and payload types 210 to 226 from ITU-T H.274.
Message framing
| Name | Coding | Description |
|---|---|---|
ff_byte | f(8), repeated | each FF adds 255 to the payload type |
last_payload_type_byte | u(8) | added to give payloadType |
ff_byte | f(8), repeated | each FF adds 255 to the payload size |
last_payload_size_byte | u(8) | added to give payloadSize in bytes |
sei_payload(payloadType, payloadSize) | the message |
A message that ends before its declared size, or leaves bits over, is padded with
payload_bit_equal_to_one and zeros to the byte boundary;
reserved_payload_extension_data may sit before that padding and lets later
editions add fields. The same type number means different things in prefix and suffix
units; a decoder that does not know a type skips payloadSize bytes.
Payload types
| Type | Message | Unit |
|---|---|---|
| 0 | buffering period | prefix |
| 1 | picture timing | prefix |
| 2 | pan-scan rectangle | prefix |
| 3 | filler payload (FF bytes) | prefix, suffix |
| 4 | user data registered by ITU-T T.35 | prefix, suffix |
| 5 | user data unregistered | prefix, suffix |
| 6 | recovery point | prefix |
| 9 | scene information | prefix |
| 15 | picture snapshot | prefix |
| 16 | progressive refinement segment start | prefix |
| 17 | progressive refinement segment end | prefix, suffix |
| 19 | film grain characteristics | prefix |
| 22 | post-filter hint | prefix, suffix |
| 23 | tone mapping information | prefix |
| 45 | frame packing arrangement | prefix |
| 47 | display orientation | prefix |
| 56 | green metadata (defined in ISO/IEC 23001-11) | prefix |
| 128 | structure of pictures information | prefix |
| 129 | active parameter sets | prefix |
| 130 | decoding unit information | prefix |
| 131 | temporal sub-layer zero index | prefix |
| 132 | decoded picture hash | suffix |
| 133 | scalable nesting | prefix |
| 134 | region refresh information | prefix |
| 135 | no display | prefix |
| 136 | time code | prefix |
| 137 | mastering display colour volume | prefix |
| 138 | segmented rectangular frame packing arrangement | prefix |
| 139 | temporal motion-constrained tile sets | prefix |
| 140 | chroma resampling filter hint | prefix |
| 141 | knee function information | prefix |
| 142 | colour remapping information | prefix |
| 143 | deinterlaced field identification | prefix |
| 144 | content light level information | prefix |
| 145 | dependent random access point indication | prefix |
| 146 | coded region completion | prefix, suffix |
| 147 | alternative transfer characteristics | prefix |
| 148 | ambient viewing environment | prefix |
| 149 | content colour volume | prefix |
| 150 | equirectangular projection | prefix |
| 151 | cubemap projection | prefix |
| 152 | fisheye video information | prefix |
| 154 | sphere rotation | prefix |
| 155 | region-wise packing | prefix |
| 156 | omnidirectional viewport | prefix |
| 157 | regional nesting | prefix |
| 158 | motion-constrained tile sets extraction information sets | prefix |
| 159 | motion-constrained tile sets extraction information nesting | prefix |
| 160 | layers not present (Annex F) | prefix |
| 161 | inter-layer constrained tile sets (Annex F) | prefix |
| 162 | bitstream partition nesting (Annex F) | prefix |
| 163 | bitstream partition initial arrival time (Annex F) | prefix |
| 164 | sub-bitstream property (Annex F) | prefix |
| 165 | alpha channel information (Annex F) | prefix |
| 166 | overlay information (Annex F) | prefix |
| 167 | temporal motion vector prediction constraints (Annex F) | prefix |
| 168 | frame-field information (Annex F) | prefix |
| 176 | 3D reference displays information (Annex G) | prefix |
| 177 | depth representation information (Annex G) | prefix |
| 178 | multiview scene information (Annex G) | prefix |
| 179 | multiview acquisition information (Annex G) | prefix |
| 180 | multiview view position (Annex G) | prefix |
| 181 | alternative depth information (Annex I) | prefix |
| 200 | SEI manifest | prefix |
| 201 | SEI prefix indication | prefix |
| 202 | annotated regions | prefix |
| 205 | shutter interval information | prefix |
| 210 | neural-network post-filter characteristics (H.274) | prefix, suffix |
| 211 | neural-network post-filter activation (H.274) | prefix, suffix |
| 212 | phase indication (H.274) | prefix |
| 213 | SEI processing order (H.274) | prefix |
| 214 | processing order nesting (H.274) | prefix, suffix |
| 215 | encoder optimisation information (H.274) | prefix, suffix |
| 216 | source picture timing information (H.274) | prefix |
| 218 | modality information (H.274) | prefix |
| 220 | digitally signed content initialisation (H.274) | prefix |
| 221 | digitally signed content selection (H.274) | prefix, suffix |
| 222 | digitally signed content verification (H.274) | suffix |
| 223 | generative face video (H.274) | suffix |
| 224 | generative enhancement face video (H.274) | suffix |
| 225 | AI usage restrictions request (H.274) | prefix |
| 226 | packed regions information (H.274) | prefix |
| other | reserved; skipped by decoders |
Every number not in this list is reserved in H.265, including numbers that H.264 uses
for its own messages; H.265 parses them as reserved_sei_message.
Buffering period (0)
| Name | Coding | Condition | Description |
|---|---|---|---|
bp_seq_parameter_set_id | ue(v) | the SPS whose HRD parameters apply | |
irap_cpb_params_present_flag | u(1) | no sub-picture HRD | 1: the two delay offsets and alternative delays follow |
cpb_delay_offset, dpb_delay_offset | u(v) × 2 | flag above is 1 | offsets used when RASL pictures are dropped; widths from au_cpb_removal_delay_length_minus1 and dpb_output_delay_length_minus1 + 1 |
concatenation_flag | u(1) | 1: the removal time follows the previous buffering period directly, as after splicing | |
au_cpb_removal_delay_delta_minus1 | u(v) | used with the concatenation flag | |
nal_initial_cpb_removal_delay[i], …offset[i] | u(v) × 2 | NAL HRD, per buffer specification | initial buffer delay and offset, in 90 kHz units; width initial_cpb_removal_delay_length_minus1 + 1 |
nal_initial_alt_cpb_removal_delay[i], …offset[i] | u(v) × 2 | sub-picture HRD or the IRAP flag | alternative values |
vcl_… | VCL HRD | the same four for the VCL HRD | |
use_alt_cpb_params_flag | u(1) | extension data present | 1: use the alternative parameters |
Picture timing (1)
| Name | Coding | Condition | Description |
|---|---|---|---|
pic_struct | u(4) | VUI frame_field_info_present_flag | see below |
source_scan_type | u(2) | the same | 0 interlaced, 1 progressive, 2 unknown, 3 reserved |
duplicate_flag | u(1) | the same | 1: this picture repeats an earlier one in output order |
au_cpb_removal_delay_minus1 | u(v) | HRD present | clock ticks from the buffering period to removal from the coded picture buffer, minus 1 |
pic_dpb_output_delay | u(v) | HRD present | clock ticks from removal to output |
pic_dpb_output_du_delay | u(v) | sub-picture HRD | the same in sub-picture ticks |
num_decoding_units_minus1 | ue(v) | sub-picture delays in this message | decoding units in the picture, minus 1 |
du_common_cpb_removal_delay_flag | u(1) | the same | 1: one delay increment applies to all decoding units |
du_common_cpb_removal_delay_increment_minus1 | u(v) | flag above is 1 | that increment, minus 1 |
num_nalus_in_du_minus1[i] | ue(v) | per decoding unit | NAL units in it, minus 1 |
du_cpb_removal_delay_increment_minus1[i] | u(v) | per decoding unit but the last, no common delay | its delay increment, minus 1 |
pic_struct | Display |
|---|---|
| 0 | progressive frame |
| 1 | top field |
| 2 | bottom field |
| 3 | top field, bottom field |
| 4 | bottom field, top field |
| 5 | top, bottom, top repeated |
| 6 | bottom, top, bottom repeated |
| 7 | frame doubling |
| 8 | frame tripling |
| 9 | top field paired with the previous bottom field |
| 10 | bottom field paired with the previous top field |
| 11 | top field paired with the next bottom field |
| 12 | bottom field paired with the next top field |
| 13–15 | reserved |
Values 1, 2 and 9 to 12 are used when field_seq_flag is 1 (each picture is a
field); 0 and 3 to 8 when it is 0.
User data (4, 5)
| Name | Coding | Description |
|---|---|---|
itu_t_t35_country_code | b(8) | registered (4): country code from ITU-T T.35, for example B5 for the United States |
itu_t_t35_country_code_extension_byte | b(8) | when the country code is FF |
itu_t_t35_payload_byte | b(8), to the end | data defined by the registered body |
uuid_iso_iec_11578 | u(128) | unregistered (5): a UUID naming the data’s owner; encoders such as x265 use it to store their settings as text |
user_data_payload_byte | b(8), to the end | the data |
Recovery point (6)
| Name | Coding | Description |
|---|---|---|
recovery_poc_cnt | se(v) | in picture order count units, how far after this picture (in output order) decoding started here becomes correct; 0 means this picture |
exact_match_flag | u(1) | 1: from the recovery point the pictures match those decoded from an earlier IRAP picture exactly |
broken_link_flag | u(1) | 1: decoding from an earlier IRAP picture may show artefacts, as after a splice |
Scene information (9)
| Name | Coding | Condition | Description |
|---|---|---|---|
scene_info_present_flag | u(1) | 1: the fields follow | |
prev_scene_id_valid_flag | u(1) | present | 1: the previous scene ID is still meaningful |
scene_id | ue(v) | present | scene identifier |
scene_transition_type | ue(v) | present | 0 none, 1 fade to black, 2 fade from black, 3 other transition to or from a constant colour, 4 dissolve, 5 wipe, 6 other mixture of two scenes |
second_scene_id | ue(v) | type above 3 | the scene being transitioned to |
Frame packing arrangement (45)
| Name | Coding | Condition | Description |
|---|---|---|---|
frame_packing_arrangement_id | ue(v) | identifier | |
frame_packing_arrangement_cancel_flag | u(1) | 1: cancels an earlier message | |
frame_packing_arrangement_type | u(7) | not cancel | 3 side by side, 4 top and bottom, 5 alternating frames; other values reserved (H.264’s checkerboard, column and row types 0 to 2 are not allowed) |
quincunx_sampling_flag | u(1) | not cancel | 1: each view is quincunx (checkerboard) sampled |
content_interpretation_type | u(6) | not cancel | 0 unspecified, 1 frame 0 is the left view, 2 frame 0 is the right view |
spatial_flipping_flag | u(1) | not cancel | 1: one view is mirrored (horizontally for type 3, vertically for 4) |
frame0_flipped_flag | u(1) | not cancel | which view is mirrored |
field_views_flag | u(1) | not cancel | 1: the two views are the two fields |
current_frame_is_frame0_flag | u(1) | not cancel | for type 5: this picture is view 0 |
frame0_self_contained_flag, frame1_self_contained_flag | u(1) × 2 | not cancel | 1: the view is predicted only from itself |
frame0_grid_position_x, …y, frame1_grid_position_x, …y | u(4) × 4 | not quincunx and type not 5 | sample position of each view, in sixteenths |
frame_packing_arrangement_reserved_byte | u(8) | not cancel | 0 |
frame_packing_arrangement_persistence_flag | u(1) | not cancel | 1: applies until cancelled or the sequence ends |
upsampled_aspect_ratio_flag | u(1) | 1: the aspect ratio in the VUI refers to each view after upsampling |
Display orientation (47)
| Name | Coding | Condition | Description |
|---|---|---|---|
display_orientation_cancel_flag | u(1) | 1: cancels an earlier message | |
hor_flip, ver_flip | u(1) × 2 | not cancel | mirror horizontally, vertically |
anticlockwise_rotation | u(16) | not cancel | rotation after flipping = 360 × value ÷ 65 536 degrees; 16 384 is 90° |
display_orientation_persistence_flag | u(1) | not cancel | 1: applies to following pictures too |
Structure of pictures information (128)
| Name | Coding | Condition | Description |
|---|---|---|---|
sop_seq_parameter_set_id | ue(v) | the SPS in use | |
num_entries_in_sop_minus1 | ue(v) | pictures in the structure, minus 1 | |
sop_vcl_nut[i] | u(6) | per picture | its NAL unit type |
sop_temporal_id[i] | u(3) | per picture | its temporal ID |
sop_short_term_rps_idx[i] | ue(v) | not IDR | its SPS reference picture set |
sop_poc_delta[i] | se(v) | after the first | picture order count step from the previous entry |
Active parameter sets (129)
| Name | Coding | Description |
|---|---|---|
active_video_parameter_set_id | u(4) | the VPS in use |
self_contained_cvs_flag | u(1) | 1: every parameter set used is sent before it is needed in the sequence |
no_parameter_set_update_flag | u(1) | 1: no parameter set content changes in the sequence |
num_sps_ids_minus1 | ue(v) | number of SPS IDs, minus 1 |
active_seq_parameter_set_id[i] | ue(v) | the SPSs in use |
layer_sps_idx[i] | ue(v) | for each further layer, which of those SPSs it uses |
Decoded picture hash (132, suffix)
| Name | Coding | Description |
|---|---|---|
hash_type | u(8) | 0 MD5, 1 CRC, 2 checksum; others reserved |
picture_md5[cIdx][i] | b(8) × 16 | MD5 (RFC 1321) of each colour plane |
picture_crc[cIdx] | u(16) | CRC of each plane |
picture_checksum[cIdx] | u(32) | checksum of each plane |
One value per colour plane: one for monochrome, three otherwise. Encoders add it so a decoder can check its output picture by picture.
Time code (136)
| Name | Coding | Condition | Description |
|---|---|---|---|
num_clock_ts | u(2) | number of time stamps, 1 to 3; one per field or frame shown, as given by pic_struct | |
clock_timestamp_flag[i] | u(1) | per time stamp | 1: a time stamp follows |
units_field_based_flag[i] | u(1) | 1: counts are in field units | |
counting_type[i] | u(5) | 0 no dropping and no offset, 1 no dropping, 2 dropping of zero frame counts, 3 dropping of the maximum count, 4 drop-frame (counts 0 and 1 dropped at each minute except every tenth), 5 and 6 unspecified dropping, 7 to 31 reserved | |
full_timestamp_flag[i] | u(1) | 1: seconds, minutes and hours all follow | |
discontinuity_flag[i] | u(1) | 1: this time stamp does not follow on from the previous one | |
cnt_dropped_flag[i] | u(1) | 1: counts were dropped just before this one | |
n_frames[i] | u(9) | frame count within the second | |
seconds_value[i], minutes_value[i], hours_value[i] | u(6), u(6), u(5) | full, or each behind its own flag | 0 to 59, 0 to 59, 0 to 23 |
seconds_flag[i], minutes_flag[i], hours_flag[i] | u(1) | not full, nested | 1: the value follows |
time_offset_length[i] | u(5) | bits in the offset | |
time_offset_value[i] | i(v) | length not 0 | signed offset in clock units |
HDR metadata (137, 144, 147, 148)
| Name | Coding | Description |
|---|---|---|
| Mastering display colour volume (137), SMPTE ST 2086 | ||
display_primaries_x[c], display_primaries_y[c] | u(16) × 6 | chromaticity of the three primaries in units of 0.00002 (5 to 37 000 valid); c = 0 green, 1 blue, 2 red |
white_point_x, white_point_y | u(16) × 2 | white point in units of 0.00002 |
max_display_mastering_luminance | u(32) | in units of 0.0001 cd/m² |
min_display_mastering_luminance | u(32) | in units of 0.0001 cd/m² |
| Content light level information (144) | ||
max_content_light_level | u(16) | MaxCLL in cd/m²; 0 unknown |
max_pic_average_light_level | u(16) | MaxFALL in cd/m²; 0 unknown |
| Alternative transfer characteristics (147) | ||
preferred_transfer_characteristics | u(8) | a transfer code that decoders which know it should use instead of the VUI one; the VUI then carries an older code for decoders that do not, such as 14 (BT.2020) with 18 (HLG) preferred |
| Ambient viewing environment (148) | ||
ambient_illuminance | u(32) | in units of 0.0001 lux |
ambient_light_x, ambient_light_y | u(16) × 2 | chromaticity in units of 0.00002 |
Other messages
The other payload types are listed above. Their fields are given in Annex D of H.265 (and Annexes F and G and ITU-T H.274 where marked); they are not repeated here.