The level 1 structures of the HDF5 File Format Specification version 3.0 (sections III and VII, read 2026-11-13) hold the metadata that the superblock and object header messages point to. In every layout below, O is a field of the superblock's size of offsets (an address) and L one of its size of lengths; numbers are little-endian. The undefined address is every bit set. A “checksum” is the Jenkins lookup3 checksum of the structure before it. A structure's signature is a four-byte ASCII string, which lets a repair tool find it.
Version 1 B-trees (TREE)
Siblings on a level form a doubly linked list. A node has one more key than it has children. Leaf and internal nodes have the same layout; leaf children are symbol table nodes or raw data chunks, internal children are lower nodes.
| Field | Size | Meaning |
|---|---|---|
| Signature | 4 | TREE |
| Node type | 1 | 0: the tree points to group nodes. 1: it points to raw data chunks. |
| Node level | 1 | 0 for a leaf. |
| Entries used | 2 | The number of children N; the valid keys and pointers come first, the rest are undefined. |
| Left sibling | O | Undefined address at the left end of a level. |
| Right sibling | O | Undefined address at the right end. |
| Keys and child pointers | (2K+1) keys and 2K pointers, interleaved: key 1, child 1, key 2, … | With N entries used, N children and N+1 keys are valid. K comes from the superblock (or the B-tree ‘K’ values message). |
| Node type | Key | Ordering |
|---|---|---|
| 0, groups | One L: the offset into the local heap of the first object name in the subtree it describes. | The object of key[i] is the greatest in child[i−1]. Key 0 is often unused: it points at offset 0 of the heap, the empty string, less than any name. |
| 1, chunks | Size of chunk in bytes (4); filter mask (4; a set bit i means filter i of the pipeline was skipped for this chunk); then D+1 64-bit offsets of the chunk within the dataset (D dimensions, the last offset in the element, always 0). | The chunk of key[i] is the least in child[i]. Key[N] is sometimes unused, with an offset greater than any other and a chunk size of 0. |
Symbol table nodes, entries and the local heap
| Structure | Layout |
|---|---|
| Symbol table node (SNOD) | Signature SNOD (4), version 1 (1; there is no version 0), reserved (1), number of symbols (2), then 2K symbol table entries (K = group leaf node K), valid ones first. A full node of 2K entries splits into K and K+1. |
| Symbol table entry | Link name offset (O: into the group's local heap, the name NUL-terminated); object header address (O); cache type (4); reserved (4, zero); scratch-pad (16). |
| Cache type 0 | Nothing cached; guaranteed when the object header has a link count above one. |
| Cache type 1 | A group: the scratch-pad holds the address of the group's B-tree (O) and of its local heap (O). |
| Cache type 2 | A symbolic link: the first 4 bytes of the scratch-pad are the offset into the local heap of the link value; the object header address is undefined. |
| Local heap (HEAP) | Signature HEAP (4), version 0 (1), reserved (3), data segment size (L), offset to the head of the free list (L; undefined if no free block), address of the data segment (O). The data segment starts right after the header but may be moved, whole, when it must grow. Objects should be aligned on 8 bytes. |
| Local heap free block | The offset of the next free block (L; 1 marks the last), then the size of this block (L), counting these two fields, so a free block is at least 2 × L. |
Global heap (GCOL)
Shared data, such as the values of variable-length types and region references, lives in collections; the global heap is the set of all collections, which do not link to each other: one is found only from a reference. A collection is treated as one unit for I/O.
| Structure | Layout |
|---|---|
| Collection | Signature GCOL (4), version 1 (1; no version 0), reserved (3), collection size (L, whole collection; default and minimum 4,096 bytes, room for 127 16-byte objects), then objects 1 to N in any order without gaps, then object 0 (free space, if any). |
| Object | Heap object index (2; the smallest free number, 0 reserved for free space), reference count (2; 0 for object 0), reserved (4), object size (L), object data, padded to a multiple of 8 (the size counts the data only). Removing an object (count falling to 0) packs the later ones down and grows object 0. |
| Object 0 | The free space at the end; its size field is the free space including its own 16-byte header. When what is left is too small to hold a header, the header is implied and not written. |
| Global heap ID | Collection address (O), then the object index (4). |
| Block for virtual datasets | Version 0 (1), number of entries (L), then per entry the source file name, source dataset name, source selection and virtual selection (each a variable-size string or dataspace selection encoding), then a checksum. |
Fractal heap (FRHP)
Built on a doubling table: indirect blocks hold only addresses; direct blocks hold object data. Each indirect block has a constant number of entries per row, the table width. The first two rows of blocks are the starting block size, and each next row's blocks are twice as big as the row before. Blocks up to the maximum direct block size are direct; larger ones are indirect blocks with their own entries.
nrows = (log2(iblock_size) - log2(starting block size)) + 1
max_dblock_rows = (log2(max direct block size) - log2(starting block size)) + 2
K = MIN(nrows, max_dblock_rows) * table width (direct block entries)
N = 0 if nrows <= max_dblock_rows, else K - max_dblock_rows * table width (indirect)| Header field | Size | Meaning |
|---|---|---|
| Signature, version | 4, 1 | FRHP, 0. |
| Heap ID length | 2 | Bytes in an object ID of this heap. |
| I/O filters' encoded length | 2 | Size of the filter information below; 0 means no filters. |
| Flags | 1 | Bit 0: the ID for huge objects has wrapped around, so each new huge object needs an ID search. Bit 1: direct blocks are checksummed. Bits 2–7 reserved. |
| Maximum size of managed objects | 4 | Larger objects are huge and stored directly in the file. |
| Next huge object ID | L | |
| v2 B-tree address of huge objects | O | The tree's record type depends on whether address and length fit in an ID and whether filters are used. |
| Free space in managed blocks | L | Bytes. |
| Managed block free-space manager | O | See the free-space section. |
| Managed space in heap, allocated managed space | L, L | The upper bound of the linear address space, and the part actually allocated. |
| Direct block allocation iterator offset | L | Where the next direct block will be allocated; the address space grows a row of direct blocks at a time. |
| Managed, huge and tiny objects: counts and sizes | L × 5 | Number of managed objects; size and number of huge objects; size and number of tiny objects (those packed in IDs). |
| Table width | 2 | Columns in the doubling table; a power of two. |
| Starting block size, maximum direct block size | L, L | Powers of two. An object bigger than the maximum direct block size less the block prefix and suffix is huge. |
| Maximum heap size | 2 | log2 of the linear address space (the number of address bits). |
| Starting # of rows in root indirect block | 2 | 0 means as many rows as the maximum heap size needs. |
| Address of root block | O | Undefined if the heap is empty. |
| Current # of rows in root indirect block | 2 | 0 means the root block is a direct block. |
| Filtered root direct block size, I/O filter mask | L, 4 | Only if the filter length is above 0. |
| I/O filter information | variable | A filter pipeline message; present only with filters. |
| Checksum | 4 |
| Block | Layout |
|---|---|
| Direct block (FHDB) | Signature FHDB (4), version 0 (1), heap header address (O), block offset in the heap's address space (the minimum number of bytes that hold the maximum heap size in bits, rounded up), checksum (4, only if flag bit 1), then object data for the rest of the block. |
| Indirect block (FHIB) | Signature FHIB (4), version 0 (1), heap header address (O), block offset (as above), then K direct block entries each with the child address (O) and, only with filters, the filtered size (L) and filter mask (4); then N indirect entries each with an address (O); checksum (4). A child's unfiltered size follows from its position in the heap's address space. |
Heap IDs. The first byte holds the ID version in bits 6–7 (0), the type in bits 4–5 and, for tiny objects, a length in bits 0–3.
| Type | Layout |
|---|---|
| 0, managed | Byte 0, then the object's offset in the heap (the minimum number of bytes for the maximum heap size), then its length (the minimum number of bytes for the smaller of the maximum direct block size and the maximum managed object size). |
| 1, huge | Byte 0 (low bits reserved), then by sub-type: indirect, plain or filtered (when an address and length do not fit): a v2 B-tree key (L, taken from the next huge object ID); direct, plain: address (O), length (L); direct, filtered: address (O), length (L), filter mask (4), de-filtered size (L). |
| 2, tiny | Normal (ID length 18 or less): byte 0 with the length minus one in bits 0–3, then the data. Extended (longer IDs): bits 0–3 are bits 8–11 of a 12-bit length, the next byte bits 0–7 (also minus one), then the data. The data sits inside the ID. |
Version 2 B-trees (BTHD, BTIN, BTLF)
Each child pointer carries the number of records in that child and, above the lowest internal level, the total in the child and everything below it, so the nth record can be found by counting.
| Structure | Layout |
|---|---|
| Header | BTHD (4), version 0 (1), type (1), node size (4: of all nodes), record size (2), depth (2), split percent (1), merge percent (1), root node address (O; undefined for an empty tree), records in the root (2), total records in the tree (L), checksum (4). |
| Internal node | BTIN (4), version 0 (1), type (1), records 0 to N−1, then N+1 child entries: child address (O), records in the child (variable size), total records in the child and below (variable size, absent on the level just above the leaves), then checksum (4). |
| Leaf node | BTLF (4), version 0 (1), type (1), records, checksum (4). |
The width of "records in the child" is the fewest bytes (up to 8) that hold the largest possible record count of the child node: for a leaf, (node size − leaf overhead) divided by the record size; for an internal node, (node size − overhead − one pointer entry) divided by (record size + one pointer entry); and the "total" width follows the product of these maxima up the levels.
| Type | Indexes | Record layout |
|---|---|---|
| 0 | Testing only | Not for real files. |
| 1 | Indirect, unfiltered huge fractal heap objects | Address (O), length (L), object ID (L). |
| 2 | Indirect, filtered huge objects | Address (O), length (L), filter mask (4), de-filtered size (L), object ID (L). |
| 3 | Direct, unfiltered huge objects | Address (O), length (L). |
| 4 | Direct, filtered huge objects | Address (O), length (L), filter mask (4), de-filtered size (L). |
| 5 | Link names in an indexed group | Hash of the name (4, Jenkins lookup3), heap ID of the link record in the group's fractal heap (7 bytes). |
| 6 | Link creation order in an indexed group | Creation order (8), heap ID (7). |
| 7 | Shared object header messages | Message location (1: 0 in the shared message heap, 1 in an object header), then for location 0: hash (4), reference count (4), heap ID (8); for location 1: hash (4), reserved (1), message type (1), object header index (2: the message is the nth of its type in that header), object header address (O). |
| 8 | Attribute names (dense attributes) | Heap ID (8), message flags (1), creation order (4), hash of the name (4). |
| 9 | Attribute creation order | Heap ID (8), message flags (1), creation order (4). |
| 10 | Chunks of a dataset without filters and with more than one unlimited dimension | Address (O), then a 64-bit scaled offset per dimension, slowest first (the chunk offset divided by the chunk size). |
| 11 | The same, with filters | Address (O), chunk size (variable, at most 8), filter mask (4), a scaled offset per dimension. |
Free-space managers (FSHD, FSSE)
A manager records unused space inside a heap or the file. The header describes the space tracked; the section list holds the free sections themselves, grouped in sets of equal size.
| Structure | Layout |
|---|---|
| Header | FSHD (4), version 0 (1), client ID (1: 0 fractal heap, 1 file, 2 and up reserved), total space tracked (L), total sections (L), serialised sections (L), un-serialised sections (L), number of section classes (2), shrink percent (2), expand percent (2), address space size (2: log2 of it), maximum section size (L), address of the serialised section list (O), size used (L), size allocated (L), checksum (4). |
| Section list | FSSE (4), version 0 (1), manager header address (O), then record sets: number of records in the set (the fewest bytes holding the serialised section count), the section size for all of them (the fewest bytes for the maximum section size), then each record: offset in the client (the fewest bytes for the address space size), type (1), type-specific data; then a checksum (4). |
| Section types, file client | 0: a range of real bytes of the file; 1 and up reserved. No extra data. |
| Section types, fractal heap client | 0 single (no extra data), 1 first row (same data as indirect), 2 normal row (none), 3 indirect: the offset of the indirect block in the heap (the fewest bytes for the maximum heap size), the start row, the start column and the number of blocks, each as the heap's rows and columns need. Higher types reserved. |
Dataset chunk indexes
Without the latest format, chunks are indexed by a version 1 B-tree of type 1. With it, the library picks one of five, by how the dataset is dimensioned and extended (Appendix C):
| Index | Used when | Structure |
|---|---|---|
| Single chunk | Current, maximum and chunk dimensions are all equal | The one chunk's address (and, with filters, its size and filter mask) is stored in the version 4 data layout message. |
| Implicit | Fixed maximum dimensions, no filter, early allocation | An array of chunks allocated at creation; base address in the layout message. Chunk address = base + chunk size × chunk index. Scaled offset = chunk offset / chunk dims; nchunks = (current dims + chunk dims − 1) / chunk dims; the index is the sum over dimensions of scaled_offset × down_chunks, where down_chunks is the product of nchunks of all faster dimensions. For current dims [4,5] and chunk dims [3,2], offset [3,4] has index 5. |
| Fixed array | Fixed maximum dimensions | Below. |
| Extensible array | One unlimited dimension | Below. |
| Version 2 B-tree | More than one unlimited dimension | Record types 10 and 11 above. |
An element describes one chunk: client ID 0 (not filtered) is the chunk's address (O); client ID 1 (filtered) is the address (O), the chunk size (variable, at most 8 bytes) and the filter mask (4).
| Structure | Layout |
|---|---|
| Fixed array header (FAHD) | Signature, version 0 (1), client ID (1), entry size (1), page bits (1: bits for the maximum entries in a page), maximum number of entries (L), data block address (O), checksum (4). |
| Fixed array data block (FADB) | Signature, version 0 (1), client ID (1), header address (O), then, if paged, a page bitmap (which pages are initialised) or, if not paged, the elements; checksum (4). Paging starts when the entry count passes a threshold; the pages follow the data block in one contiguous allocation. |
| Fixed array data block page | Elements, checksum (4). |
| Extensible array header (EAHD) | Signature, version 0 (1), client ID (1), element size (1), max elements bits (1), index block elements (1), data block minimum elements (1), secondary block minimum data pointers (1), max data block page elements bits (1), then secondary blocks created (L), their size (L), data blocks created (L), their size (L), maximum index set (L), elements realised (L), index block address (O), checksum (4). |
| Extensible array index block (EAIB) | Signature, version 0 (1), client ID (1), header address (O), elements held directly, data block addresses held directly, secondary block addresses; checksum (4). |
| Extensible array secondary block (EASB) | Signature, version 0 (1), client ID (1), header address (O), block offset, page bitmap (if its data blocks are paged), data block addresses; checksum (4). |
| Extensible array data block (EADB) | Signature, version 0 (1), client ID (1), header address (O), block offset, elements (if not paged); checksum (4). A page is the elements plus a checksum. |
The extensible array is optimised for appends: the index block points to secondary blocks, which point to data blocks or pages, and a lookup takes only a few file operations; data blocks are paged once they pass a size threshold.
Driver information block (level 0B)
From section II.B of the HDF5 File Format Specification version 3.0 (source H5.format.3.0.dox, HDFGroup/hdf5 develop, read 2026-11-13). An optional region (or a driver info message in the superblock extension) holding what a file driver needs to reopen the file.
| Field | Size | Meaning |
|---|---|---|
| Version | 1 | 0. |
| Reserved | 3 | Zero. |
| Driver information size | 4 | Size in bytes of the driver information. |
| Driver identification | 8 | ASCII, not NUL-terminated. Predefined drivers: NCSA plus the first four letters of the driver name; if the block is not the original version the last letter(s) become an ASCII version number starting at 0. User-defined drivers also use 8 bytes and should avoid the NCSA prefix. |
| Driver information | variable | Defined by the driver. |
- Multi driver (
NCSAmulti): separates raw data and kinds of metadata into up to 6 files that the library sees as one virtual file with one address space. Information: six member mapping bytes, one per kind of data in order (superblock, B-tree, raw data, global heap, local heap, object header), each a value 1 to 6 saying which kind it is merged with (1 superblock, 2 B-tree, 3 raw data, 4 global heap, 5 local heap, 6 object header; if the third byte is 3 and the rest 1 there are two files, raw data and everything else); two reserved bytes (zero); then for each member file N an address (virtual start address) and an end-of-address (end of the allocated space), each drawn as a full-width field in the specification; then the member file names, each NUL-terminated and padded with NULs to a multiple of 8 bytes. Default names follow%s-X.h5where%sis the whole file's name and X iss,b,r,g,lorofor superblock, B-tree, raw data, global heap, local heap, object header. - Family driver (
NCSAfami, library 1.8 and later): splits the address space over several smaller files for systems with a 2 GB file limit; does not separate metadata from raw data. Information: one field, the size of each member file.
The widths of the multi driver address fields and of the family driver size field are not stated in the specification text (the tables draw them as full-width rows).
Appendix D: encodings of dataspace selections and references
From Appendix D of the HDF5 File Format Specification version 3.0 (same source). Field widths are those the layout tables draw: a full-width row is 4 bytes unless the row states a size; rows marked “this space inserted only to align the table” are not in the data.
Dataspace description (H5Sencode / H5Sdecode)
| Field | Meaning |
|---|---|
| Dataspace ID (1 byte) | The dataspace message ID, 1. |
| Encode version (1 byte) | H5S_ENCODE_VERSION, 0. |
| Size of size (1 byte) | Number of bytes used to store the size of an object. |
| Size of extent | Size of the dataspace message. |
| Dataspace message (variable) | The dataspace header message (dataspace message). |
| Dataspace selection (variable) | Below. |
Dataspace selection
A selection type and then its selection info: 0 H5S_SEL_NONE (nothing selected), 1 H5S_SEL_POINTS (a sequence of points), 2 H5S_SEL_HYPERSLABS (a hyperslab), 3 H5S_SEL_ALL (the entire extent).
| Selection info | Layout |
|---|---|
| NONE and ALL | Version (value 1), then 8 reserved zero bytes. |
| POINTS | Version, 1 or 2. Version 1: reserved (zero), Length (size in bytes from Length to the end of the selection info), Rank (number of dimensions), Num Points, then for each point its Rank coordinates (full-width fields). Version 2: Encode Size (1 byte: the width of the following fields: 2, 4 or 8), Rank, Num Points (Encode Size bytes), then each point's coordinates (Encode Size bytes each). |
| HYPERSLABS | Version, 1, 2 or 3. Version 1: reserved, Length (size from Rank to the end), Rank, Num Blocks, then for each block its starting offset in each dimension followed by its ending offset in each dimension (the offsets of the first and last element), ordered from the fastest-changing dimension to the slowest. Version 2: Flags (bit 0 set: regular hyperslab; always 0x1 currently), Length (size from Rank to the end), Rank, then for each dimension Start, Stride, Count, Block, each 8 bytes (start of the first block; elements to move; number of blocks; block size). Version 3: Flags (bit 0: regular if set, irregular otherwise), Encode Size (1 byte: 2, 4 or 8), Rank, then either the regular form (Start, Stride, Count, Block for each dimension, each Encode Size bytes) or the irregular form (Num Blocks (Encode Size bytes), then for each block its starting offsets and ending offsets in each dimension, each Encode Size bytes). |
Reference encoding (revised)
Object references without an external file (H5R_OBJECT2) store the reference header and block together as the dataset's raw data; object references with an external file, dataset region references (H5R_DATASET_REGION2) and attribute references (H5R_ATTR) store the header plus a global heap ID in the raw data, the reference block being in the global heap.
- Reference header: Reference type (1 byte): 2
H5R_OBJECT2, 3H5R_DATASET_REGION2, 4H5R_ATTR; Flags (1 byte): bit 0 set means the reference is to an external file, bits 1 to 7 reserved. - Reference block: Token size (1 byte) and the Token (the object's token); if flags bit 0: Length of external file name (2 bytes) and the External file name; if the type is dataset region: Size of dataspace selection, Rank of dataspace selection and the Dataspace selection information (as above); if the type is attribute: Length of attribute name (2 bytes) and the Attribute name. Size of dataspace selection and Rank of dataspace selection are 4 bytes each; Token size is 1 byte; the two name lengths are 2 bytes (as the layout tables draw them).
Reference encoding (backward compatibility)
H5R_OBJECT1: stored as the dataset's raw data: the Object Address (Size of Offsets bytes).H5R_DATASET_REGION1: the raw data holds a global heap ID; the heap object holds the Object Address (Size of Offsets bytes) followed by the Dataspace selection information.
File space allocation types
Appendix B of the specification names six basic types: H5FD_MEM_SUPER
(superblock), H5FD_MEM_BTREE, H5FD_MEM_DRAW (raw data),
H5FD_MEM_GHEAP, H5FD_MEM_LHEAP and H5FD_MEM_OHDR
(object header). Other types map onto them: the shared message index
(H5FD_MEM_SOHM_INDEX) to B-tree; huge fractal heap objects
(H5FD_MEM_FHEAP_HUGE_OBJ) to raw data; fractal heap direct blocks
(H5FD_MEM_FHEAP_DBLOCK) and the free-space section list
(H5FD_MEM_FSPACE_SINFO) to local heap; and the fractal heap header and
indirect blocks, the free-space manager header and the shared message table to
object header.