.PY File Format – Lexical structure

Physical lines

A physical line ends in LF, CR LF or CR. Each form is replaced by a single LF before anything else happens, even inside string literals, and a file may mix them. The end of the input also ends the last line.

Comments

A comment starts with # outside a string literal and runs to the end of the physical line (2.1.3). It ends the logical line unless the line is inside brackets. Python has no block comment. A coding declaration and a shebang are comments with a special use.

Joining lines

Explicit joining (2.1.5): a physical line that ends in a backslash outside a string or comment is joined to the next, and the backslash and line end are removed. A line ending in a backslash cannot carry a comment, and the backslash cannot split a token other than a string literal.

Implicit joining (2.1.6): inside ( ), [ ] or { }, lines join without a backslash. These lines may carry comments, may be blank, and may be indented freely, and no NEWLINE token is produced between them. This is the preferred way to split a long line.

total = (first_value    # implicit joining: brackets
         + second_value
         + third_value)

if year > 1900 and year < 2100 \
        and month <= 12:      # explicit joining: backslash
    print("in range")

Blank lines

A logical line with only spaces, tabs, form feeds and possibly a comment is ignored and produces no NEWLINE (2.1.7). In the interactive prompt a blank line can end a block, but this is a feature of the prompt.

Indentation

The white space at the start of a logical line sets its indentation level, and indentation is what groups statements (2.1.8). Tabs are replaced, from left to right, by one to eight spaces so the column reaches the next multiple of eight. The total count of spaces is the level. A form feed at the start of a line is ignored for this count.

The tokeniser keeps a stack of levels, starting with a single 0 that is never removed. For each logical line that is not blank, the level is compared with the top of the stack:

• If it is equal, nothing happens.
• If it is larger, it is pushed and one INDENT token is generated.
• If it is smaller, it must be a value already on the stack. Every larger value is popped, and one DEDENT token is generated for each.
• At the end of the file, one DEDENT is generated for each value on the stack above 0.

def f(x):
    if x:
        return 1
    return 0

NAME 'def'  NAME 'f'  OP '('  NAME 'x'  OP ')'  OP ':'  NEWLINE
INDENT  NAME 'if'  NAME 'x'  OP ':'  NEWLINE
INDENT  NAME 'return'  NUMBER '1'  NEWLINE
DEDENT  NAME 'return'  NUMBER '0'  NEWLINE
DEDENT  ENDMARKER

The compiler reports these errors:

Convention (PEP 8) is four spaces per level and no tabs.

Space between tokens

Except at the start of a logical line and inside string literals, spaces, tabs and form feeds separate tokens and are otherwise ignored (2.1.9). White space is needed between two tokens only when joining them would read as one, as with if x. A number directly followed by a keyword, as in 1or 2, still works but raises SyntaxWarning, and a future version will reject it.

Keywords

There are 35 keywords, which can never be used as names (2.3.1):

In alphabetical order, as the keyword module lists them: False None True and as assert async await break class continue def del elif else except finally for from global if import in is lambda nonlocal not or pass raise return try while with yield.

Soft keywords

A few words are keywords only in one place, and ordinary names elsewhere, so old code that uses them as names keeps working (2.3.2):

__debug__ cannot be assigned to, although it is not a keyword.

Reserved classes of names

Some names have a special meaning by their pattern of underscores (2.3.3):

Integers

integer:      decinteger | bininteger | octinteger | hexinteger | zerointeger
decinteger:   nonzerodigit (["_"] digit)*
bininteger:   "0" ("b" | "B") (["_"] bindigit)+
octinteger:   "0" ("o" | "O") (["_"] octdigit)+
hexinteger:   "0" ("x" | "X") (["_"] hexdigit)+
zerointeger:  "0"+ (["_"] "0")*

• Integers have unlimited precision, so a literal can be as long as memory allows.
• 0123 is an error, to avoid confusion with the C-style octal of Python 2. A number made only of zeros, such as 00 or 0_0, is valid.
• 1__0, _1 and 1_ are errors. _1 is a valid name, not a number.

Floating-point numbers

floatnumber:
    digitpart "." [digitpart] [exponent]
    | "." digitpart [exponent]
    | digitpart exponent
digitpart: digit (["_"] digit)*
exponent:  ("e" | "E") ["+" | "-"] digitpart

Valid examples are 3.14, 10., .001, 1e100, 3.14e-10 and 0e0. Leading zeros are allowed here: 077.010 means 77.01. Underscores are allowed between digits, as in 96_485.332_123, but not next to the . or the e. In CPython a float is an IEEE 754 double. There are no literals for infinity or NaN; use float("inf") and float("nan").

Imaginary numbers

imagnumber: (floatnumber | digitpart) ("j" | "J")

An imaginary literal has a real part of zero, as in 4.5j or 1_000j. A complex number with a real part is written as a sum: 3+4j. A base prefix cannot be used, so 0xFFj is an error.

Triple-quoted strings

A string opened with ''' or """ can contain unescaped newlines and quotes, and ends at the first matching triple quote. Line endings inside it are stored as LF. Whitespace is kept exactly. A backslash at the end of a line skips the line end. Common uses are multi-line text and docstrings.

Escape sequences

In ordinary and bytes literals, these sequences are recognised (2.5.4):

• Python keeps the backslash of any other sequence, so '\q' is two characters. These unrecognised escapes raise a SyntaxWarning since Python 3.12 and will be an error in a future version.
• In bytes literals \N, \u and \U are unrecognised escapes.
• An octal escape above \377 also raises SyntaxWarning since Python 3.12.
• In a string, \ud800 and other surrogates are allowed, although they cannot be encoded to UTF-8 without a special error handler.

Bytes literals

A bytes literal produces bytes, not text (2.5.5). It may contain only ASCII characters, whatever the source encoding, so byte values of 128 or more must use an escape.

signature = b'\x89PNG\r\n\x1a\n'
list(signature)        # [137, 80, 78, 71, 13, 10, 26, 10]

Raw strings

With an r prefix, a backslash is a plain character and no escapes are processed (2.5.6). A quote can still follow a backslash, but the backslash stays in the result: r"\"" is two characters. A raw literal cannot end in an odd number of backslashes, so r"\" is an error. Raw strings suit regular expressions and Windows paths.

import re
re.findall(r"\d{4}-\d{2}", "due 2026-11")     # ['2026-11']

Concatenation

Adjacent string literals are joined at compile time, even across lines inside brackets (6.2.3.2). Plain strings and f-strings can be mixed. Bytes cannot be mixed with strings, and t-strings cannot be mixed with either.

message = ("first part, "
           f"then {value!r}, "
           "and the end")

f-strings

A formatted string literal contains replacement fields in braces, and each one is an expression evaluated where the f-string appears (2.5.7, PEP 498, Python 3.6). Doubled braces outside a field stand for a literal brace.

{ expression [ = ] [ !conversion ] [ :format_spec ] }

name = "Ada"
width = 8
pi = 3.14159

f"Hello, {name}!"           # 'Hello, Ada!'
f"{name!r}"                 # "'Ada'"
f"{pi:.2f}"                 # '3.14'
f"{name:>{width}}"          # '     Ada'
f"{pi=:.3f}"                # 'pi=3.142'
f"{255:#x}, {1234567:,}"    # '0xff, 1,234,567'
f"{{{name}}}"               # '{Ada}'

Since Python 3.12 (PEP 701) the expression inside a field follows the normal rules: it may reuse the f-string's own quote character, contain backslashes and comments, span several lines, and nest f-strings to any depth in CPython.

• An empty expression is an error, and lambda and := must be in parentheses.
• Everything after a # in a field is a comment, including any closing brace, so a field with a comment must close on a later line.
• An f-string cannot be a docstring.

t-strings

A template string, or t-string, uses the same syntax as an f-string (2.5.8, PEP 750, Python 3.14). The difference is what it creates: a string.templatelib.Template object, which keeps the static text and the evaluated values apart so that code can process them, for example to escape values before building HTML.

name = "Ada"
template = t"Hello, {name!r}"

template.strings          # ('Hello, ', '')
template.values           # ('Ada',)
template.interpolations   # (Interpolation('Ada', 'name', 'r', ''),)

Each Interpolation has value, expression, conversion and format_spec. The conversion and format specifier are recorded but not applied. A Template can be added to another Template, but not to a str.