HEIC: boxes and items

HEIF (ISO/IEC 23008-12) is built on the ISO base media file format (ISO/IEC 14496-12), the box structure of MP4. A still-image file holds items: coded images, derived images, thumbnails, alpha and depth planes, and metadata such as Exif. The meta box lists the items, says where each one’s bytes are and which properties apply to it. All numbers are big-endian. The properties are on Item properties and grids and overlays on Derived images.

The HEIF and ISOBMFF standards are sold by ISO; HEIF is not freely available. This page follows the layouts as the reference library libheif parses them, checked against the public AVIF specification, which builds on HEIF.

Box header

Offset	Size	Type	Name	Description
0	4	u32	size	Size of the whole box, header included. 1 means a 64-bit size follows; 0 means the box runs to the end of the file.
4	4	4CC	type	Four characters naming the box, such as `ftyp`.
8	8	u64	largesize	Only when size is 1.
8 or 16	16	UUID	usertype	Only when type is `uuid`: the box’s real identifier.

A full box adds one byte of version and three bytes of flags after the header. Readers skip boxes they do not know, using the size.

File layout

0ftyp brandsvaries
…
meta full box, version 0
1. hdlr handler, pictrequired
2. pitm primary itemrequired
3. iloc where each item’s data isrequired
4. iinf one infe per itemrequired
5. iref links between itemsoptional
6. iprp ipco properties and ipma associationsrequired
7. idat small item data kept inside metaoptional
8. grpl groups of itemsoptional
9. dinf data referencesoptional
varies
…mdat coded image datavaries
…moov tracks, for image sequences onlyoptional

A HEIF still-image file. Boxes may appear in other orders; ftyp comes first.

ftyp: file type

Offset	Size	Type	Name	Description
8	4	4CC	major_brand	The main specification the file follows.
12	4	u32	minor_version	Version of the major brand; usually 0.
16	4 each	4CC	compatible_brands	Every specification the file also conforms to, to the end of the box.

Brand	Meaning
`mif1`	HEIF image items, any codec. The structural brand every still-image file lists.
`msf1`	HEIF image sequence, any codec.
`heic`	HEVC images using the Main or Main Still Picture profile (8-bit 4:2:0).
`heix`	HEVC images using Main 10 or the range extensions (higher bit depths, 4:2:2, 4:4:4).
`heim`	Multiview HEVC (MV-HEVC) images.
`heis`	Scalable HEVC (SHVC) images.
`hevc`	HEVC image sequence (Main profiles).
`hevx`	HEVC image sequence (range extensions).
`hevm`	Multiview HEVC sequence.
`hevs`	Scalable HEVC sequence.
`avif`, `avis`	AV1 image and sequence: see AVIF.
`vvic`, `vvis`	VVC (H.266) image and sequence.
`evbi`, `evbs`	EVC image and sequence.
`j2ki`, `j2is`	JPEG 2000 image and sequence (ISO/IEC 15444-16).
`miaf`	Conforms to MIAF (ISO/IEC 23000-22), a stricter interoperability profile of HEIF.

A file is recognised by ftyp at offset 4 and one of these brands. A typical iPhone photo starts:

00 00 00 18 66 74 79 70 68 65 69 63 00 00 00 00 6D 69 66 31 …, a box of the stated size, ftyp, major brand heic, minor version 0, then compatible brands starting with mif1. (The size and the rest of the list vary.)

hdlr and pitm

Box	Size	Type	Name	Description
hdlr (full box, v0)	4	u32	pre_defined	0.
	4	4CC	handler_type	`pict` for images.
	12	u32 × 3	reserved	0.
	to end	string	name	Zero-terminated UTF-8, often empty.
pitm (full box)	2 or 4	u16 / u32	item_ID	The primary item: the image to show. 16-bit in version 0, 32-bit in version 1.

iinf and infe: item information

iinf (full box) holds entry_count, u16 in version 0 and u32 in later versions, then that many infe boxes. HEIF uses infe version 2 or 3:

Size	Type	Name	Description
flags bit 0	bit	hidden	Set for items that should not be shown on their own, such as grid tiles.
2 or 4	u16 / u32	item_ID	16-bit in version 2, 32-bit in version 3.
2	u16	item_protection_index	0 when the item is not protected.
4	4CC	item_type	What the item is, from the table below.
varies	string	item_name	Zero-terminated, often empty.
varies	string	content_type	Only for `mime` items: the media type.
varies	string	content_encoding	Only for `mime` items, optional: e.g. a compression.
varies	string	item_uri_type	Only for `uri` items.

Versions 0 and 1 have no item_type: a u16 item_ID, the protection index, then the name, content type and encoding strings.

item_type	Meaning
`hvc1`	An HEVC coded image; its decoder configuration is the `hvcC` property.
`hvt1`	An HEVC tile of a larger coded image.
`lhv1`	A layered (multiview or scalable) HEVC image.
`grid`	A derived image made by placing other images in a grid.
`iovl`	A derived image made by overlaying images on a canvas.
`iden`	A derived image that is another image with its own properties applied.
`Exif`	Exif metadata.
`mime`	Data of the media type in content_type; XMP is `application/rdf+xml`.
`uri`	Data identified by a URI.
`av01`	An AV1 coded image (AVIF).
`jpeg`	A JPEG coded image.
`vvc1`	A VVC coded image.
`j2k1`	A JPEG 2000 coded image.
`unci`	An uncompressed image (ISO/IEC 23001-17).

iloc: item locations

Size	Type	Name	Description
4 bits	u4	offset_size	Bytes in each extent_offset: 0, 4 or 8.
4 bits	u4	length_size	Bytes in each extent_length: 0, 4 or 8.
4 bits	u4	base_offset_size	Bytes in base_offset: 0, 4 or 8.
4 bits	u4	index_size / reserved	Bytes in each extent_index (versions 1 and 2); reserved in version 0.
2 or 4	u16 / u32	item_count	u32 in version 2.

Then, for each item:

Size	Type	Name	Description
2 or 4	u16 / u32	item_ID	u32 in version 2.
2	12 + 4 bits	reserved, construction_method	Versions 1 and 2 only: 0 means file offsets, 1 means offsets into `idat`, 2 means offsets into other items.
2	u16	data_reference_index	0 means this file; otherwise an entry in `dref`.
base_offset_size	uint	base_offset	Added to every extent offset.
2	u16	extent_count	Number of pieces; the item’s data is the pieces joined in order.
index_size	uint	extent_index	Per extent, only when index_size is not 0: which referenced item the extent is in.
offset_size	uint	extent_offset	Per extent.
length_size	uint	extent_length	Per extent; 0 means to the end of the file.

For an HEVC image item, the data is the coded picture as NAL units, each preceded by its length in (lengthSizeMinusOne + 1) bytes from hvcC; the parameter sets are in hvcC, not in the item.

iref: item references

iref (full box) holds one small box per reference, whose box type is the reference type:

Size	Type	Name	Description
8	box header	size, reference_type	The type, from the table below.
2 or 4	u16 / u32	from_item_ID	u32 when iref is version 1.
2	u16	reference_count	At least 1.
2 or 4 each	u16 / u32	to_item_ID	The items referred to, in order.

Type	From	To	Meaning
`dimg`	derived image	its inputs	The images a grid, overlay or identity item is built from, in order.
`thmb`	thumbnail	master image	A smaller version of the image.
`auxl`	auxiliary image	master image	An alpha plane, depth map or other auxiliary image; its kind is in `auxC`.
`cdsc`	metadata item	image	Exif or XMP that describes the image.
`prem`	colour image	alpha image	The colour is premultiplied by that alpha.
`base`	pre-derived image	its base images	A stored result of a derivation, such as an HDR image made from others.

iprp, ipco and ipma: properties

iprp holds ipco, a list of property boxes numbered from 1 in order, and one or more ipma boxes that say which properties belong to which item. An index of 0 means no property.

Size	Type	Name	Description
4	u32	entry_count	Items listed.
2 or 4	u16 / u32	item_ID	Per item; u32 when ipma is version 1.
1	u8	association_count	Per item.
1 or 2	bit + u7 / bit + u15	essential, property_index	Per association. With flags bit 0 clear, one byte: bit 7 essential, bits 0 to 6 the index. With it set, two bytes: bit 15 essential, bits 0 to 14 the index.

A reader that does not understand an essential property must not show the item. Transformative properties (clap, irot, imir) are applied in the order they are associated, after decoding.

idat, grpl, dinf and mdat

idat: raw bytes inside meta, addressed by items with construction_method 1. Grid and overlay descriptions often live here.
grpl: a list of entity groups. Each is a full box whose type is the grouping type, then group_id (u32), num_entities_in_group (u32) and that many entity IDs (u32). Types include altr (alternatives: show the first one you can), ster (a stereo pair, left then right) and pymd (a pyramid of resolutions).
dinf holds dref (full box, entry_count u32, then entries); a url entry with flags bit 0 set means “this file”, otherwise it carries a location string.
mdat is plain data, located only through iloc.
An image sequence (brand msf1) adds a moov box with tracks whose handler is pict, laid out as in MP4.

.HEIC File Format – Boxes and items