.PY File Format – Generators and async

Iterators

An iterable has __iter__, which returns an iterator. An iterator has __next__, which returns the next value or raises StopIteration, and an __iter__ that returns itself.

for x in items:
    body(x)

# does the same as
iterator = iter(items)
while True:
    try:
        x = next(iterator)
    except StopIteration:
        break
    body(x)

Two built-ins go with the protocol. iter(callable, sentinel) calls a function until it returns the sentinel. next(it, default) returns the default instead of raising StopIteration. The itertools module offers building blocks for iterators.

Generators

A function with yield in its body is a generator function. Calling it runs nothing and returns a generator, which is an iterator (PEP 255). Each next() runs the body up to the next yield, and the function's state is kept between calls.

def countdown(n):
    while n > 0:
        yield n
        n -= 1

list(countdown(3))      # [3, 2, 1]

def worker():
    received = yield "ready"
    print("got", received)
    try:
        yield "waiting"
    except ValueError:
        yield "recovered"
    return "finished"

g = worker()
next(g)                 # 'ready'
g.send("data")          # prints: got data, returns 'waiting'
g.throw(ValueError())   # 'recovered'
next(g)                 # StopIteration, whose value is 'finished'

def inner():
    yield 1
    return "result"

def outer():
    value = yield from inner()   # value is "result"
    yield value

list(outer())                    # [1, 'result']

Coroutines

async def defines a coroutine function (PEP 492, Python 3.5; Language Reference 8.9). Calling it returns a coroutine object without running the body. The body runs when the coroutine is awaited or scheduled on an event loop.

An awaitable is a coroutine, or an object whose __await__ returns an iterator. await pauses the current coroutine until the awaitable finishes and gives its result. await, async for and async with are valid only inside async def. A coroutine that is never awaited produces a RuntimeWarning.

import asyncio

async def fetch(name, delay):
    await asyncio.sleep(delay)
    return f"{name} done"

async def main():
    first = await fetch("a", 0.1)        # one after another

    results = await asyncio.gather(      # at the same time
        fetch("b", 0.1), fetch("c", 0.1))

    async with asyncio.TaskGroup() as group:
        task = group.create_task(fetch("d", 0.1))
    print(first, results, task.result())

asyncio.run(main())

The language defines the syntax only, and a library such as asyncio provides the event loop. asyncio.run starts it. TaskGroup and timeout arrived in Python 3.11. Since Python 3.14, asyncio.get_event_loop() no longer creates a loop implicitly. The old generator-based coroutines with @asyncio.coroutine were removed in Python 3.11.

async for

async_for_stmt: "async" for_stmt

An asynchronous iterable has __aiter__, and its iterator has an async __anext__ that raises StopAsyncIteration at the end (3.4.3, 8.9.2).

iterator = type(ITER).__aiter__(ITER)
running = True

while running:
    try:
        TARGET = await type(iterator).__anext__(iterator)
    except StopAsyncIteration:
        running = False
    else:
        SUITE
else:
    SUITE2

The built-ins aiter() and anext() arrived in Python 3.10.

async with

async_with_stmt: "async" with_stmt

An asynchronous context manager has async __aenter__ and __aexit__ methods (3.4.4, 8.9.3). It behaves as the with statement does, with each of those calls awaited.

manager = EXPRESSION
aenter = type(manager).__aenter__
aexit = type(manager).__aexit__
value = await aenter(manager)
hit_except = False

try:
    TARGET = value
    SUITE
except:
    hit_except = True
    if not await aexit(manager, *sys.exc_info()):
        raise
finally:
    if not hit_except:
        await aexit(manager, None, None, None)

Asynchronous generators

An async def with yield in its body is an asynchronous generator (PEP 525, Python 3.6), used with async for. It may not use yield from, and may not return a value. Both are syntax errors.

async def ticker(delay, count):
    for i in range(count):
        yield i
        await asyncio.sleep(delay)

async def main():
    return [i async for i in ticker(0.1, 3)]

Asynchronous comprehensions (PEP 530) are allowed inside async def. An asynchronous generator expression is allowed in a regular function, and it can only be consumed in asynchronous code.