Python 3.10 .pyc files: marshal

The body of a .pyc file is one object in Python’s marshal format. An object is a type byte, then whatever that type needs; containers hold further objects, so the format nests. The whole body is a single code object, and the code objects of the module’s functions and classes are inside its constants. All numbers are little-endian.

Python 3.10 writes .pyc files with marshal version 4. The version is not stored in the file; the magic number implies it.

References and FLAG_REF

A type byte may have its top bit set, 0x80, called FLAG_REF. The object is then added to a list of saved objects, numbered from 0 in the order they are read. A later r object gives an index into that list and means the same object again, so a name or constant used many times is stored once.

A container takes its place in the list when its type byte is read, before its items. So the numbering follows the order of the type bytes.

When writing, CPython sets the flag on an object that may be used again: one with more than one reference to it, or an interned string. It does not set it for None, True, False, Ellipsis or StopIteration, which are single bytes. A type byte of 0xE9 is i with FLAG_REF, and 0xDA is Z with FLAG_REF.

Object types

The table gives the type byte without FLAG_REF. “i32” and “u32” are little-endian 32-bit numbers.

Byte	Char	Type	After the type byte	Notes
`0x30`	`0`	TYPE_NULL	nothing	Not a value: marks the end of a dictionary
`0x4E`	`N`	TYPE_NONE	nothing	`None`
`0x46`	`F`	TYPE_FALSE	nothing	`False`
`0x54`	`T`	TYPE_TRUE	nothing	`True`
`0x53`	`S`	TYPE_STOPITER	nothing	The `StopIteration` class
`0x2E`	`.`	TYPE_ELLIPSIS	nothing	`Ellipsis`
`0x69`	`i`	TYPE_INT	i32 value	An integer that fits in 32 bits, signed
`0x49`	`I`	TYPE_INT64	8-byte signed value	CPython reads it but no longer writes it; a larger integer is `l`
`0x6C`	`l`	TYPE_LONG	i32 digit count, then that many u16 digits	An integer of any size, in base 2¹⁵, lowest digit first. A negative count means a negative number; the digits are the absolute value
`0x66`	`f`	TYPE_FLOAT	u8 length, then that many ASCII characters	A float as text. Only written with `marshal` version 0 or 1, so never in a `.pyc` of this format
`0x67`	`g`	TYPE_BINARY_FLOAT	IEEE 754 double, 8 bytes	A float
`0x78`	`x`	TYPE_COMPLEX	u8 length and text for the real part, then the same for the imaginary part	A complex number as text. Only written with `marshal` version 0 or 1, so never in a `.pyc` of this format
`0x79`	`y`	TYPE_BINARY_COMPLEX	two IEEE 754 doubles, real then imaginary	A complex number
`0x73`	`s`	TYPE_STRING	i32 length, then the bytes	A `bytes` object. `co_code` and `co_linetable` are this type
`0x74`	`t`	TYPE_INTERNED	i32 length, then UTF-8 bytes	Interned text with non-ASCII characters
`0x52`	`R`	TYPE_STRINGREF	u32 index	A reference to an interned string in the old Python 2 scheme. Python 3 neither writes nor reads it, and the explorer reports an error if it meets one
`0x75`	`u`	TYPE_UNICODE	i32 length, then UTF-8 bytes	Text with non-ASCII characters, not interned. Lone surrogates are kept with the `surrogatepass` error handler
`0x61`	`a`	TYPE_ASCII	i32 length, then the characters	ASCII text of 256 characters or more, not interned
`0x41`	`A`	TYPE_ASCII_INTERNED	i32 length, then the characters	Interned ASCII text of 256 characters or more
`0x7A`	`z`	TYPE_SHORT_ASCII	u8 length, then the characters	ASCII text of fewer than 256 characters, not interned
`0x5A`	`Z`	TYPE_SHORT_ASCII_INTERNED	u8 length, then the characters	Interned ASCII text of fewer than 256 characters: most names
`0x28`	`(`	TYPE_TUPLE	i32 count, then that many objects	A tuple of 256 items or more
`0x29`	`)`	TYPE_SMALL_TUPLE	u8 count, then that many objects	A tuple of fewer than 256 items
`0x5B`	`[`	TYPE_LIST	i32 count, then that many objects	A list. The compiler does not put lists in constants
`0x7B`	`{`	TYPE_DICT	a key object, a value object, and so on, ended by `0`	A dictionary
`0x3C`	`<`	TYPE_SET	i32 count, then that many objects	A set. Items are written in the order Python iterates them, which can vary between runs
`0x3E`	`>`	TYPE_FROZENSET	i32 count, then that many objects	A frozen set, as the compiler makes for `x in {1, 2}`. Items are written in the same way
`0x63`	`c`	TYPE_CODE	the code object fields, in order	A code object. Usually with FLAG_REF, `0xE3`
`0x72`	`r`	TYPE_REF	u32 index	The saved object with this number; see references
`0x3A`	`:`	TYPE_SLICE	three objects: start, stop and step	Not part of this format: Python 3.10 neither writes nor reads it

CPython also defines ?, TYPE_UNKNOWN, an error marker written only when an object cannot be marshalled. CPython’s reader does not accept it.

Worked example

These are the bytes of marshal.dumps((1, 'a', b'x', None, 1.5, 2**40, -1, True, {'k': 1}, frozenset({2}))) under Python 3.10.

00000000  29 0A E9 01 00 00 00 DA 01 61 F3 01 00 00 00 78
00000010  4E E7 00 00 00 00 00 00 F8 3F EC 03 00 00 00 00
00000020  00 00 00 00 04 E9 FF FF FF FF 54 7B DA 01 6B 72
00000030  00 00 00 00 30 3E 01 00 00 00 E9 02 00 00 00

A small tuple of ten items, count 0x0A

Bytes	Object	Saved as
`29 0A`	Small tuple, 10 items	not saved
`E9 01 00 00 00`	Integer 1, FLAG_REF	#0
`DA 01 61`	Interned short text `'a'`, FLAG_REF	#1
`F3 01 00 00 00 78`	Bytes `b'x'`, FLAG_REF	#2
`4E`	`None`	never saved
`E7 00 00 00 00 00 00 F8 3F`	Float 1.5, FLAG_REF	#3
`EC 03 00 00 00 00 00 00 00 00 04`	Integer of 3 digits: 0, 0 and 0x0400, which is 1024 × 2³⁰ = 2⁴⁰, FLAG_REF	#4
`E9 FF FF FF FF`	Integer −1, FLAG_REF	#5
`54`	`True`	never saved
`7B`	Dictionary	not saved
`DA 01 6B`	The key, interned short text `'k'`, FLAG_REF	#6
`72 00 00 00 00`	The value, a reference to #0, the integer 1
`30`	End of the dictionary
`3E 01 00 00 00`	Frozen set of 1 item	not saved
`E9 02 00 00 00`	Integer 2, FLAG_REF	#7

.PYC File Format – Marshal

References and FLAG_REF

Object types

Worked example