CLJ file format: Concurrency

Clojure keeps identity apart from value: an identity is a reference that points to a succession of immutable values. There are four reference types, and each is for one kind of change. All of them are read with deref (@x), take a validator function that can reject new values, and take watches that are called after a change.

The reference types

Type	Change is	Made with	Changed with
Var	thread-local (dynamic bindings)	`def`, `binding`	`binding`, `set!`, `alter-var-root`
Atom	synchronous, independent	`atom`	`swap!` `reset!` `compare-and-set!` `swap-vals!` `reset-vals!`
Ref	synchronous, coordinated, in a transaction	`ref`	`alter` `commute` `ref-set` `ensure`, inside `dosync`
Agent	asynchronous, independent	`agent`	`send` `send-off` `send-via`

Atoms

An atom holds one value that changes atomically. (swap! a f args*) applies f to the current value and the arguments and sets the result; if another thread changed the atom in between, the function is called again, so f must have no side effects. Atoms suit state that never needs coordinating with other state, for example a memoisation cache.

Refs and the transaction system

Refs share state safely using a software transactional memory (STM) built on multiversion concurrency control with snapshot isolation. Changes to refs are made only inside (dosync body*). A transaction is atomic (all its changes happen or none), consistent (validators are checked before commit), and isolated (it sees a snapshot taken at its read point, plus its own changes).

alter sets a new value with a function; ref-set sets it directly; commute applies a commutative function whose result may be re-applied at commit time, so that it conflicts less; ensure protects a ref that is read but not changed.
Readers never block writers, and writers never block readers or commuters. A transaction may be retried, so it must not perform side effects; io! throws if it is run inside a transaction.
The values should be immutable, preferably persistent collections, because all work in a transaction is speculative. Agent sends made in a transaction are held until it commits.

Agents

(send agent f args*) returns at once; later, on another thread, f is applied to the agent's state and the arguments, the result goes through the validator, becomes the new state, and the watches are called. Actions to one agent run one at a time, in the order sent from one source. send runs the action in a thread from a thread pool (the pooled executor of the Agent class); send-off is for actions that may block and runs it in a separate thread (the solo executor); send-via takes an executor (since 1.5). An exception in an action is stored in the agent and every later use throws until the errors are cleared (agent-error, restart-agent, set-error-handler!, set-error-mode!, clear-agent-errors). Agents start non-daemon threads, so shutdown-agents is needed to let the JVM exit. await and await-for wait for the actions sent so far.

Futures, promises, delays and parallelism

Form	Does
`(future body*)`, `future-call`	runs the body on another thread; `@f` blocks until it is done; `future-done?` `future-cancel` `future-cancelled?` `future?` `realized?`
`(promise)`, `(deliver p v)`	a value that one thread delivers once and others wait for with `@p`
`(delay body*)`, `force`, `delay?`	runs the body the first time it is dereferenced and caches the result
`pmap` `pcalls` `pvalues`	apply a function, call functions or evaluate expressions in parallel (semi-lazily)
`(locking x body*)`	holds the Java monitor of an object; use it only to protect Java objects
`volatile!` `vreset!` `vswap!`	an unsynchronised mutable cell for use inside one thread, in transducers

Futures and the agent actions convey the thread's dynamic bindings (binding conveyance): (binding [*x* 2] (future (println *x*))) prints 2. bound-fn does the same for functions you hand to other threads.

(def counter (atom 0))
(swap! counter inc)           ; => 1
(add-watch counter :log (fn [k r old new] (println old "->" new)))

(def a (ref 100))
(def b (ref 0))
(dosync
  (alter a - 10)
  (alter b + 10))

(def worker (agent []))
(send worker conj :job)

An atom with a watch, a transaction over two refs, and an agent.

core.async

clojure.core.async is a library (not part of clojure.core) of channels for communicating sequential processes. A (chan) carries values; go and go-loop blocks run asynchronously and park instead of tying up an operating-system thread when they wait on <! (take), >! (put) or alt!/alts!; they should not perform operations that may block indefinitely, because that can deplete the fixed pool of go-block threads; the double-bang forms <!! and >!! block a thread; alts!, alt!, timeout, thread, pipe, pub/sub, mult/tap, mix and pipeline are among its other functions. The viewer colours these names in code that refers to them through a clojure.core.async alias.

Not described

Known, not yet covered

Reference: core.async is only summarised: its own documentation was not read (only its README and the public names of its source were read). Its channels with buffers (buffer, dropping-buffer, sliding-buffer) and transducers are named, not explained.
Reference: the thread pools (sizes, executors), STM internals (history queues, ref-min-history) are not covered.

Unknown

Reference: the sizes and kinds of the pools behind pooledExecutor and soloExecutor are in Java source that was not downloaded; the core.async rationale and API pages were not downloaded either.

.CLJ File Format – Concurrency