A field guide toActive Inference

In 2018 a small bulletin devoted to altered states of consciousness asked Karl Friston, the neuroscientist behind the free energy principle, a first question about his life. He answered with eight pages of "personal notes" under a title that is a question rather than a claim. His own answer, given twice, is careful: "I do not think I was really autistic." But looking back he sees "some distinctly autistic traits" in the child he was and in the adult he is. He avoids mobile phones and does not Skype. A disruption to his weekly routine is "rather nerve wracking". At international conferences he spends most of the time avoiding other human beings, especially in the morning; and yet he feels "most at home" when lecturing onstage, "close to lots of people who are, at the same time, comfortably distant."

What the notes actually record is the making of a particular kind of mind, and the pictures that decorate them are the reason this guide looks the way it does. As a boy he was "obsessed with the natural world" and recorded it "in a somewhat obsessional fashion": page after page of aquatic flora and fauna, drawn for a school project on ruled exercise-book paper, each specimen numbered, named in Latin and English, and measured to the inch. A scallop. A common starfish, a cushion star, a brittle star. A heart urchin drawn from the top and from the bottom. Squat lobster, common shrimp, hermit crab, acorn barnacle, mussels, periwinkles, a razor shell. "I must have spent hours on these," he writes, "at the expense of actually learning what I should have been learning."

Vignettes, in order

• Age five. Assessed by an educational psychologist to resolve "a confusing relationship" with his teacher: "a mildly traumatic experience" that took him out of the state system and into a private school run by Catholic nuns, "where I flourished".
• The second assessment. He was asked whether the puppets in Thunderbirds ever got hungry. "What on earth does a psychologist expect me to say?" After "several levels of recursive sophistication", he opted for yes.
• Age ten. A science project: a self-righting robot table, with mercury levels and feedback actuators, that could cross uneven ground. The psychologist wanted to know how he came up with it; he "somehow knew she was more interested in me than robots". The robot, he adds, set him up to understand feedback, optimal control and, much later, cybernetics.
• Walking to primary school past the grammar-school pupils waiting for their bus: "one day, I want to be in that queue", and then, "I must remind myself about this moment when I am grown—so I do not forget the insight, ambition and sophistication of childhood."
• Six schools by ten. His father built bridges for the motorways and the family moved with the roads. "The only constants in my life were my family—and the things inside my head."
• School. Self-esteem came from "drilling down into an area or problem in a somewhat perfectionist fashion" and from "isolating the problem domain—that felt like my little world". His teachers played games with him: could he find solutions to A-level mathematics questions more parsimonious than the examination boards' own? He derived the Schrödinger wave equation from scratch; his reward was to take the physics class while the physics teacher, Ged Proctor, amused himself in the stockroom (his primary passion was Morris dancing).
• Late teens. "Preoccupied with holistic explanations based on minimal assumptions", he spent hours in his bedroom above the cherry blossom, "convinced that there should be a singular explanation for the shape of things, just starting from the premise that something existed". His best picture of it was "some abstract point in an abstract space", which "in later life transpired to be a point attractor in a phase-space".
• Cambridge. After a year of medicine he chose psychology and physics, and spent a summer holiday, "to the exclusion of everything else", trying to get all of physics onto one page. He failed, but got quantum theory onto one page: a hand-lettered sheet of boxes and arrows, photographed for the essay under a brass clip. "Symptomatic of an obsessional drive to integration and simplification." He then forgot nearly all of it, but was never afterwards intimidated by the subject, foraging later in Wikipedia.
• A Christmas. "Thinking earnestly in the early hours over a nourishing coal fire in the family living room", he concluded that "all interesting things have to occupy a compact domain of phase-space and must therefore possess an attracting set", and that "the only invariance that lent 'shape to things' entailed correlations".
• Medicine, psychiatry, Blackwell's. He qualified as a doctor, trained in psychiatry, and found "a peaceful distraction" from a job in a therapeutic community for people with chronic schizophrenia, in an old Victorian asylum, in nursing one idea: that statistical invariance, correlations, had to be "transcribed into the physics of our brains". One Saturday at Blackwell's bookshop in Oxford, three hours into scouring the medical and mathematical shelves (this was before the web), he found Hebb, and "an ambivalent reaction: intense pleasure that the idea was valid and intense displeasure I had wasted several years on something that was already known." He tried to work out how old Hebb was, and whether they could ever have met. Then he waited, "patiently", until he could get into research proper, at about twenty-eight.

Reading the book with the notes in hand

The shape of Active Inference becomes legible once you have seen these pages. The adolescent demand for a single explanation "starting from the premise that something existed" is exactly the argument of the book's third chapter, where the whole theory is derived from nothing more than the fact that a thing persists. The coal-fire conclusion about compact domains and attracting sets is what the book calls a nonequilibrium steady state. The correlations that had to be transcribed into brains are Hebb's synapses, which reappear in the seventh chapter, derived rather than assumed, as a counting rule for learning. The little robot with its feedback actuators is the reflex arc that, in the eighth chapter, turns a prediction into a movement. And the one-page quantum chart is the ambition of the whole book: that the sprawl of psychology and neuroscience might be condensed into one principle. The last sheet of this guide is an attempt to do for Active Inference what the teenager did for quantum mechanics.

Two more things from the essay travel with the reader. First, the woodlouse. Friston mentions, in passing, a childhood observation of woodlice (told in full in an interview in the same issue) that made natural selection easy for him to accept: the apparent emergence of purpose from purposeless but "shaped" dynamics. The whole theory has that flavour. Its creatures look purposeful, curious and goal-directed, and none of that is put in by hand. Second, a confession from the book's own preface: "I did not write much of this book. Or, more precisely, I was not allowed to. This book's agenda calls for a crisp and clear writing style that is beyond me." The clarity is Parr's and Pezzulo's, who, he adds, have "theory of mind—in all senses". The ideas are shared. The sensibility on every page, the drive to put everything on one sheet, is his.

Every living thing is in constant exchange with its surroundings. It acts, and the world answers with sensations; the sensations change what it does next; the next action changes what it will sense. The book opens with this loop and with a question that sounds too plain to be scientific: how do living organisms persist, and act adaptively, while caught inside it? A bacterium following a nutrient gradient and a person planning a career are answers at different levels of sophistication and on different timescales, but they answer the same problem. To keep your bodily integrity you must exert control over the loop, soliciting sensations that correspond to what you need (nutrients and shelter for a simple organism, "friends and jobs for more complex ones") or that help you make sense of where you are.

Two temperaments

The authors borrow Roger Schank's division of researchers into neats and scruffies. Scruffies embrace heterogeneity: perception, attention, decision-making and social behaviour each get their own explanation, built bottom-up from data, using whatever works. Neats seek a unification that starts from first principles and derives as much as possible top-down. This is a book by neats, and its wager is that the diversity of behaviour, cognition and adaptation in living things is "amenable to a coherent explanation from first principles". It is careful about what that means. Active Inference does not aim to replace behavioural psychology, decision theory or reinforcement learning; it hopes "to embrace all those approaches that have proven so successful within a unified framework".

One imperative

The framework rests on a single idea: all facets of behaviour and cognition follow one imperative, minimizing the surprise of sensory observations. "Surprise" is a technical term from information theory, not a feeling. It measures how far an organism's current sensations are from the sensations that preserve its integrity: for a fish, being in water; for you, a body temperature near 37 °C. Two consequences follow at once, and the rest of the book unfolds from them.

First, surprise cannot be minimized by watching. To keep your sensations within bounds you have to act on the world; this is the active in Active Inference, and it is why perception and action will turn out to be two halves of one process rather than two faculties.

Second, surprise itself cannot be computed directly, for technical reasons that Sheet II makes plain. What an organism can do is minimize a proxy, a quantity called variational free energy, through neural activity, in response to sensations and in anticipation of them. That is the free energy principle in one sentence, and it explains the book's subtitle.

Field marks

Before the theory is built, the authors preview six ways it differs from its neighbours. They are worth keeping in view; every later sheet returns to at least one of them.

1. Perception and action serve one objective. Perception lowers free energy by changing your beliefs to fit the world; action lowers it by changing the world to fit your beliefs. Learning is perception at a slower timescale.
2. Planning is inference, scored by expected free energy. The score has two parts, the uncertainty a course of action is expected to resolve (exploration) and how well its outcomes fit the organism's goals (exploitation). The balance between them is automatic, not tuned by hand.
3. Everything is inference over a generative model. The model is an internal, probabilistic account of how hidden causes produce sensations. Under some assumptions the resulting dynamics are the familiar ones of predictive coding and the Helmholtz machine.
4. Perception and learning are active. The brain predicts rather than waits, and it seeks out the sensations that will resolve its uncertainty.
5. Action is goal-directed. It starts from a desired outcome, encoded as a prior prediction, and infers the sequence of actions that fulfils it. Stimulus–response habits are a special case, not the rule.
6. The constructs have biological analogues. Perceptual inference looks like changing synaptic activity; learning like changing synaptic efficacy; the precision of a prediction like the gain of an error unit; the precision of a plan like dopamine.

What the book leaves out

The book is about organisms and brains: bacteria to humans, and their behavioural, cognitive, social and neural processes. The wider free energy principle, which applies the same mathematics to cells, morphogenesis, evolution and culture, is left at the door, along with most of the physics and the philosophy. Part 1 (Sheets I–V) builds the theory; Part 2 (Sheets VI–X) shows how to use it.

1. Perception is inference

There is an old idea, from Hermann von Helmholtz in the 1860s and arguably from Kant before him, that perception is a kind of unconscious inference. The eye does not deliver the world; it delivers evidence, and the brain infers what caused it. In its modern form this is the Bayesian brain hypothesis: perception is not an outside-in transduction of sensations into pictures but an inside-out process in which the brain's own hypotheses are confirmed or disconfirmed by what arrives at the senses.

To infer this way an organism needs a generative model: a probabilistic account of how the hidden states of the world, x, generate observations, y. Formally it is a joint probability P(y, x), and it splits into two parts. The prior P(x) says what the organism expects before looking. The likelihood P(y | x) says which observations each hidden state would produce. Bayes' rule combines them into a posterior, the belief after the observation is in:

The book's worked example is deliberately silly so that the arithmetic is easy. A person is fairly sure she is looking at an apple: her prior gives 0.9 to apple and 0.1 to frog. Her likelihood says frogs jump (0.81) and apples almost never do (0.01). Then the thing jumps. Bayes' rule assigns the frog a posterior of 0.9 and the apple 0.1: one observation has overturned the prior, because the observation was eighty times more likely under the alternative. Experiment 1 lets you turn the dials.

2. Two kinds of surprise

The denominator in Bayes' rule, P(y), is the probability of the observation under the model as a whole: the model evidence, or marginal likelihood. In the example it is 0.1 × 0.81 + 0.9 × 0.01 = 0.09. Only nine times in a hundred does this model expect to see anything jump, so the model ought to be surprised, and the technical measure of that is surprise (sometimes surprisal): the negative logarithm of the evidence, −ln 0.09 ≈ 2.4 nats. (A nat is the natural-log cousin of a bit.) The larger the surprise, the worse the model is as an explanation of what was seen. A rival model that expected frogs all along would be surprised by only 0.2 nats; scoring models by their evidence in this way is Bayesian model comparison. Surprise can be written down for any family of distributions, whether the world is delivering categories (whose face is this?) or continuous quantities (how cold is it?), which is why a whole portfolio of distributions will be needed when models are built later.

Bayesian surprise is a different thing. It measures not how unlikely the observation was but how much you had to change your mind: the distance from prior to posterior, measured by the Kullback–Leibler (KL) divergence, the average difference between two log-probabilities. In the example it is about 1.8 nats. The two come apart cleanly. Commit absolutely to apples, with a prior of 1.0, and your Bayesian surprise when the thing jumps is zero, because you cannot update, while your surprise is enormous (4.6 nats), because an apple has just jumped. Hold that distinction; it becomes the difference between complexity and accuracy below.

3. The model is not the world

Two cautions. First, Bayesian inference is "optimal" only with respect to a cost function (it will turn out to be free energy), and exact Bayesian inference is beyond the resources of any real creature, so approximation is unavoidable. Second, and more interesting, the organism's generative model is not the generative process, the true causal structure by which the world makes its sensations. The world is in some true state x*; the brain entertains hypotheses x; the two need not live in the same space. The world may have five dimensions where the brain uses two; the brain may posit causes that do not exist; the model may never converge on the process, and need not. A simpler story can be a better explanation of the data at hand than the truth, as measured by surprise. Visual illusions are the example: "a mischievous psychophysicist" engineers a stimulus for which the brain's simpler explanation wins. Optimality is always relative to the creature's model, its memory and its compute.

4. Action is inference too

Everything so far is common to all Bayesian-brain theories. Here is the step that makes Active Inference different. Surprise depends on the model, but it also depends on the data. If you can act on the world to change how the data are generated, you can choose the data that are least surprising under your model. An organism with muscles lives in a loop: sense, infer, act to make the environment less surprising, sense again (in animals the mechanism is a motor reflex loop; the steps are really continuous). And the book insists on something stronger than "action is also inferential". Perception and action optimize one and the same quantity.

The plainest way to see it is as a discrepancy between prediction and sensation. She predicted red-and-still; she saw jumping. There are two ways to close the gap. She can change her mind (it is a frog): that is perception. Or she can turn her gaze to the nearest apple tree and change what she sees: that is action. The gaze example feels like a trick, so take the book's second one. Your thermoreceptors report a body temperature above the range you expect. Changing your mind ("evidently I run hot") is not adaptive; opening a window is. Here the prior is not a guess about the world but a specification of the states you must occupy. This is the move that lets the theory absorb homeostatic set-points: an organism's model assigns high probability to the observations it prefers, so that surprise means at once "this is unlikely" and "this is not where I should be". For the fish, the marginal likelihood of being in water is high; that is what makes it a fish.

Which direction of fit you use depends on confidence. Beliefs held loosely get updated; beliefs that underwrite your existence get enforced on the world. Experiment 2 turns that sentence into a slider.

5. Variational free energy: one quantity, three readings

Why not simply compute surprise and minimize it? Because computing P(y) means summing over every hidden state that could have produced y, and computing the posterior needs that sum too. For any model rich enough to be interesting the sum (or integral) is intractable. Variational inference replaces the two unknowables with two computable stand-ins. In place of the true posterior, an approximate posterior Q(x) that the organism chooses and adjusts (machine learners call it a recognition distribution). In place of the log evidence, the variational free energy F, a function of Q and of the data. Inference becomes optimization: make F as small as you can by moving Q. (In machine learning, −F is the evidence lower bound, ELBO. Same object, opposite sign.)

F can be rearranged in three ways, and each is a lens on what minimizing it means.

Energy minus entropy. The energy term rewards consistency between Q and the generative model (the word is inherited from statistical physics, where the log-probability of a configuration is proportional to minus its energy). The entropy term rewards uncertainty: absent data or precise priors, you should hold the most uncertain belief you can. Be no more sure than you have reason to be.

Complexity minus accuracy. Complexity here is the divergence from prior to posterior, exactly the Bayesian surprise of section 2: the amount you had to change your mind. Every accurate explanation costs some change of mind, and free energy charges for it, so the best explanation is the simplest accurate one. The trade-off recurs later as the basis of model comparison and of pruning a model in sleep.

Divergence plus surprise. F is an upper bound on surprise; the gap between them is the divergence between Q and the true posterior. Perception can close the gap, by improving Q. It cannot touch the surprise itself, because it cannot alter the sensations already gathered. Only action can do that, by changing what is sensed. So perception minimizes divergence, action minimizes surprise, and both are minimizing F. One more thing to notice: F is retrospective. It depends on past and present observations, never future ones. That is enough for perceiving. It is not enough for planning.

6. Expected free energy: planning as inference

To plan, an organism must consider observations it has not yet had and choose among courses of action. The book calls a course of action a policy and is careful to distinguish it from an action: a policy is "a hypothesis about a way of behaving"; an action is what actually touches the world. Planning is then inference in the same sense as before, except that the alternatives are not frog-or-apple but pond-or-orchard. Two ingredients are needed: a model of how hidden states change under each policy, and the likelihood as before. Chained together they let the model run counterfactual, "what if" simulations: what would I see if I went to the pond? Each policy is scored by its expected free energy, G, the free energy of the observations it is expected to bring about, summed over the planning horizon. Because G lives in the space of log-probabilities, turning scores into beliefs is a matter of exponentiating and normalizing: the lower a policy's G, the more probable it is, and the organism expects to pursue it. That expectation has sensory consequences (a plan to bend the arm predicts what biceps and triceps will feel), and the plan becomes a movement when reflexes make those predictions come true.

What is G made of? Like F, it can be rearranged, and two arrangements matter.

Information gain and pragmatic value. The first term, also called epistemic value, is how much a policy is expected to change your beliefs, which is to say how much uncertainty it would resolve. The second is how closely the outcomes it produces match your preferences, which enter as a prior over outcomes (the book writes it C). Both are measured in nats, so exploring and exploiting trade in one currency, and the classic exploration–exploitation dilemma of behavioural psychology is not so much solved as dissolved. The book's illustration: you want an espresso, and you know two good cafés, one open on weekdays and one at weekends, but you do not know what day it is. First an epistemic action (look at the calendar), then a pragmatic one (go to the right café). A planner that counts only pragmatic value "can only select policies by using random number generators—and will often miss out on their espresso". Information gain also penalizes aliased situations, states that look identical from where you stand, like two T-junctions in a maze; such places are avoided because nothing can be learned there.

Risk and ambiguity. The second arrangement splits G along the same seam as complexity and accuracy. Risk is expected complexity: the divergence between the outcomes a policy would bring and the outcomes you prefer. It captures the one-to-many mapping from policies to results, the slot machine whose payout distribution you know and cannot change. Ambiguity is expected inaccuracy: how dispersed outcomes remain even when the hidden state is known. A coin flip conditioned on the weather is maximally ambiguous; you will never learn about rain by watching for tails. Most real situations carry both, and G handles both automatically.

A twist worth remembering. In F the divergence is something to minimize; in G the corresponding term is something to maximize. The order in which expectations are taken flips the sign, because future outcomes now play the role of causes, "variables that are hidden in the future but explain decisions in the present". Variational free energy is minimized by observations that comply with what you already believe. Expected free energy is minimized by choosing the observations that will change your beliefs the most.

The table (after the book's Figure 2.6) is a family tree. Familiar decision schemes from psychology, control theory and machine learning are each what remains of G when some of its terms are thrown away, and each therefore applies only in the situations where the discarded terms would not have mattered. There is, the authors say, "no privileged role for epistemic (explorative) versus pragmatic (exploitative) imperatives—or for risk versus ambiguity. These (possibly false) dichotomies are just two sides of the same existential coin."

At the end of the low road

Two loops, one inside the other. The outer loop minimizes variational free energy; it is sufficient for perception and for holding beliefs about policies. A creature equipped with a model of the consequences of its own actions runs an inner loop that evaluates expected free energy and hands back probabilities over policies. Those probabilities are its intentions: "a creature's belief about policies directly corresponds to its intention—which it fulfills by acting."

The low road also changes the meaning of a familiar word. In ordinary Bayesian statistics a prior is a guess. In Active Inference the expected states are the preferred ones, including the states an organism has to occupy to survive, and surprising states are the dis-preferred ones. Fulfilling your expectations and staying alive become the same thing, which licenses one of the book's memorable lines: "the identity of an agent is isomorphic with its priors."

1. Start from existence

The high road inverts the argument of Sheet II. Instead of beginning with Bayes and discovering that free energy is a good approximation to inference, it begins with a fact about living things and discovers that they must, in effect, be doing approximate inference. The fact is this: to survive, an organism has to stay within a small subset of the states the universe allows. A fish has to stay in water. A human's temperature and heart rate have to stay within narrow ranges, "otherwise they will die (or more precisely will become something else, such as a corpse)". That acceptable range, the book says, "stipulatively defines the characteristic states something has to be in to be that thing". Organisms defend it at every level: physiology (sweating), psychology (buying a drink), culture (air conditioning). In the technical language, surprising states are the ones outside the comfort zone; to persist is to minimize surprise; and free energy minimization is proposed as the practical, biologically grounded way of doing so.

2. The Markov blanket

For there to be surprise there must be something to be surprised and something to be surprised about: a system and an environment that can be told apart. Judea Pearl's Markov blanket is the statistical way of drawing that line. The blanket of a variable is the set of variables that mediate all its interactions with everything else (its parents, its children, and the other parents of its children). Once you know the blanket, the outside tells you nothing more about the inside, and the inside nothing more about the outside. Applied to a creature, internal states (say, brain activity) and external states (the world) are separated by two kinds of blanket state: sensory states, through which the world influences the inside, and active states, through which the inside influences the world. Internal states cannot change external states directly, only through active states; external states cannot change internal states directly, only through sensory states. It is the action–perception loop of Sheet I restated as a fact about conditional independence, and it can be drawn around a cell, a brain, an organism, a couple or a community. Blankets nest.

Two things follow. The first is autonomy: internal states are partly insulated from the environment's dynamics and can resist them. The second is subtler. Because internal and external states both influence, and are influenced by, the same blanket, they come into a kind of synchrony. Huygens noticed in 1673 that two pendulum clocks hung from the same wooden beam fall into step; each becomes predictive of the other through the beam. In the same way the average internal state becomes informative about external states, and one can associate each internal state with a probability distribution over external ones. Internal states thereby look as if they represent, or model, the world. Nothing has been assumed about minds. The appearance falls out of the blanket.

3. Self-evidencing, and the necessary optimism

Push the synchrony one step and the internal states look like an approximate posterior belief about the outside, and their dynamics look like inference under a generative model of how sensory states arise. But the model cannot merely mirror the world, or the creature would follow the world's dissipative dynamics and disperse. It must also specify the conditions of its own existence: the states it has to visit to remain what it is. These enter the model as priors, which means the model implicitly assumes that its preferred sensations are the likely ones. The book calls this an optimism bias, and treats it as a necessity rather than a flaw: it is what turns a mirror into an agent. With the bias in place, maximizing the evidence for the model does two jobs at once. It explains sensations (the descriptive side of inference) and it brings about preferred sensations (the prescriptive side). Jakob Hohwy's word for the whole activity is self-evidencing: acting to garner sensory evidence for one's own model, which is to say for one's own existence. Since evidence is P(y) and surprise is −ln P(y), maximizing the one is minimizing the other.

4. Why surprise must fall: the ball in the bowl

Why must a persisting system minimize surprise at all? The book gives a physicist's sketch (the full treatment lives in Friston's 2019 papers). If a Markov blanket persists, the distribution of its states stays constant over time. Random fluctuations keep knocking the states toward improbable regions, so the average flow must keep pushing them back toward probable ones, like a ball rolling down a hill into a basin. Read the height of the hill as surprise and the picture is complete: at steady state, dynamics descend an energy function (a Hamiltonian, in the physics) that is negative log evidence. The book's Figure 3.3 shows two systems started at the same point with the same random kicks. One flows back toward the least surprising region and keeps its shape; the other, indifferent to surprise, wanders off and dissipates. Active Inference is about the first kind only. Experiment 3 reproduces the figure.

This is how living things resist, for a while, the second law of thermodynamics. Entropy, the dispersion of states, is the long-run average of surprise, so to occupy a small set of states with high probability is to hold your entropy at a particular level (not to drive it to zero: the walker on the left keeps a consistent spread rather than shrinking to a point). A physiologist recognizes homeostasis: sensor values that leave their range are pushed back by negative feedback. A control engineer recognizes a set-point, generalized to a whole distribution of preferred outcomes. The simplest such system, a random attractor like the left-hand walker, is a thermostat: one set-point, no learning, no planning. What distinguishes a thermostat from a person, on this account, is the shape of the attractor, from fixed points to increasingly complex and itinerant dynamics. Living organisms "constantly seek a compromise between excessive stability and excessive dispersion".

Nor does minimizing surprise mean rigidity. The book's Box 3.3 is worth summarizing because it answers the commonest objection (why would such a creature ever leave a dark, quiet room?). Advanced organisms pursue their homeostatic imperatives through open-ended repertoires of novel behaviour: "to produce and buy good wine to satisfy thirst and other needs", a predictive kind of regulation called allostasis. They also carve their niches, building refuges and cities against natural forces and laws and norms against social ones, so that the world becomes less surprising in the long run, and they accept short-term surprise to get there. And two entropies appear in the mathematics with opposite signs: the entropy of your beliefs should be high (keep your options open, commit only when you must), while the ambiguity of future outcomes should be low (resolve what can be resolved). Curiosity, on this account, is not in tension with surprise minimization; it is one of its instruments.

5. One thing with three names

E. T. Jaynes argued that inference, information theory and statistical physics are the same subject seen from three sides. The book adds cognition as a fourth and supplies a dictionary.

A warning that the book repeats (Box 3.2): "free energy" borrows the equations of thermodynamics but applies them to an agent's beliefs, not to the particles of its body. And the systems in question are open, nonequilibrium steady states in ceaseless exchange with their surroundings, not the closed equilibria of the textbook.

6. What perception can and cannot do

The bound of Sheet II, F = divergence + surprise, now reads as a division of labour. The motion of internal states minimizes the divergence, making the approximate posterior as good as it can be. But perception cannot touch the surprise, because it cannot change what has been sensed. The book's example: your model expects satiation, that is, high blood glucose. You sense low glucose. Perception infers that you are hungry, and closes the divergence; you now experience hunger. It can go no further. Only an action (eating "a dessert") changes the sensation and, with it, the surprise. The simplest systems have no divergence to close; their internal states are already the exact posterior. That is one way to say what cognition adds: cognitive systems must do the perceptual work before their actions are guaranteed to reduce surprise.

7. Beliefs in place of values

In optimal control, reinforcement learning and economics, behaviour optimizes a value function of states, usually learned by trial and error and computed through Bellman's equation. Active Inference replaces value with belief. Endowing an agent with expected free energy amounts to giving it an implicit prior that it will realize its preferences; its preference for a course of action just is its belief about what it will do and encounter. The authors admit this looks strange from either side (reinforcement learners keep value and belief apart; Bayesian statisticians keep value out of belief) and give three reasons it is powerful. It yields a self-consistent, teleological process model, and the preferred policy is something you can measure directly, whereas a value function must be inferred. Beliefs come with degrees and uncertainty, which value functions lack, and they handle itinerant sequences of behaviour more naturally. And optimal behaviour then follows Hamilton's principle of least Action: a path of least resistance to a steady state.

The analogy with Hamiltonian physics is meant at three levels. Like the Lagrangian and Hamiltonian reformulations of Newton, which derive the equations of motion from a conserved quantity, Active Inference specifies one quantity, free energy, from which neuronal and behavioural dynamics can be derived; different generative models give different dynamics as different Hamiltonians give different physics. Second, the energies in this book are all negative log-probabilities, so energy is simply improbability, and the conservation of energy and of probability are the same law. Third, both rest on the calculus of variations: a physical path minimizes an Action that is a functional of the path; a belief minimizes a free energy that is a functional of the belief.

8. Kinship

Three families of theory, usually at odds, turn out to be describing the same dynamics. Enactive theories (Maturana and Varela's autopoiesis) emphasize self-organization and the maintenance of a boundary; the Markov blanket is that boundary made formal. Cybernetic theories (Rosenblueth, Wiener, Ashby, Miller, Powers) describe behaviour as purposive: a mechanism tests whether a goal is met and steers corrective action; Active Inference says what is being minimized, a measurable statistical quantity that under some conditions is just a prediction error, and makes the control prospective. Model-based theories (Craik's mental models; Conant and Ashby's good regulator theorem, that every good regulator of a system must be a model of that system) hold that control rests on an internal model of the world; here the model is the generative model. Add ideomotor theory (Herbart, James): action begins as an imagined consequence, not as a response to a stimulus, which, the book notes, requires briefly attenuating the sensory evidence that would falsify the belief that triggers the action. Self-organization and teleology, representation and enaction: often treated as incompatible, here the same thing.

9. From life to agency

A line is drawn across the continuum of living things. Every self-evidencing system minimizes variational free energy; such systems can regulate themselves, achieve set-points and be enormously fit (viruses are the example). Some carry hierarchical models and can handle events at several timescales. But without temporal depth, a model that represents the consequences of its own actions, they live in the present. A temporally deep model opens the door to expected free energy, that is, to planning, and with it a primitive form of agency: the creature "becomes able to select among alternative futures as opposed to simply selecting how to deal with the sensed present". Temporal depth becomes psychological depth. Where each kind of creature falls along this line, the book says, is an empirical question.

This is the book's most technical chapter, and its authors say it may be skipped. This sheet keeps what a reader needs to hold on to for the rest: where the free energy bound comes from, and the two shapes a generative model takes.

1. Where the bound comes from

Bayes' rule is easy to state and hard to compute, because the evidence P(y) is a sum over every hidden state. The trick that makes it tractable is Jensen's inequality: for a concave function like the logarithm, the log of an average is always at least the average of the log. (If the logarithm is a hill, the chord between two points on it lies below the hill.) Multiply the intractable sum by Q(x)/Q(x), which is to say by one, read the result as an average under Q, take logs, and the inequality hands you a quantity that is never above the log evidence and that you can compute: minus the free energy. Making F small pulls the bound tight, and the gap is exactly the KL divergence from Q to the true posterior. There are many ways to measure the distance between two distributions; why this one? Because the goal is self-evidencing first and matching the posterior second, and Jensen fixes the choice.

2. Drawing a model

The book draws models as factor graphs: circles for the variables one holds beliefs about, squares for the probability distributions (factors) that connect them, arrows for the direction of causation. A clinical test is the example. The prevalence of a disease is the prior on the hidden state x; the test's sensitivity and specificity form the likelihood of the result y. Add a second disease that could also explain a positive result, a second test that measures the same disease, or a genetic predisposition sitting above the disease, and you have the shapes generative models take. What makes the models of Active Inference special is that they evolve as new observations are sampled, that the next observation depends (through action) on the current beliefs, and that they can include "how I am acting" among the hidden variables.

3. Beads on a string: discrete time

The first shape treats time as a sequence of events and states as categories: which object, which plan, which room. It is the partially observable Markov decision process (POMDP), and its whole content fits in five letters.

Because the transitions depend on the policy, each policy is a model of a behavioural sequence, and choosing among policies is model comparison. The prior over policies is the softmax (exponentiate and normalize) of −G: a free-energy-minimizing creature's self-consistent prior is that it will act to minimize expected free energy. Write out F for this model and minimize it, and two coupled updates fall out. Beliefs about states drift down the free energy gradient, driven by the difference between what is predicted and what is observed: perception. Beliefs about policies are the softmax of −(G + F): planning. Drawn as a network of nodes exchanging local messages, this is the message passing that Sheet V will map onto neurons; the mean-field assumption that beliefs factorize over time is what keeps the sums small.

The sign twist from Sheet II now has its formal home. In F, the KL term is between the approximate posterior and the exact one. In G, the analogous term compares the posterior you have now with the posterior you would have after seeing a whole trajectory of future outcomes, treating today's posterior as tomorrow's prior; averaged over outcomes that have not happened yet, its sign reverses. So G is minimized by selecting the observations that would cause the largest change in beliefs, while F is minimized when observations agree with beliefs already held. F is the primary quantity, minimized over time; G is a prerequisite, because it furnishes the prior over policies that F needs.

4. The stream: continuous time

The second shape suits sensory receptors and muscles, which vary continuously. States obey stochastic differential equations: the rate of change of x is some function f(x, v) plus noise, and the data are some function g(x, v) plus noise, where v is a slowly varying cause (it plays the role the policy played on the left). This is the generative model behind the Kalman–Bucy filters of engineering, with one biological amendment: real fluctuations are smooth, not white, so the model tracks not only where a state is but its velocity, acceleration and higher derivatives, its generalized coordinates of motion. These are the coefficients of a Taylor series around the present moment, letting the model extrapolate a short way into the past and the future. About six suffice.

Under the Laplace approximation, which treats every distribution as Gaussian near its peak, the free energy collapses to something a neuron could plausibly compute: a sum of squared prediction errors, each weighted by a precision (an inverse variance, a measure of how much to trust that error). There is a sensory error (data minus the prediction of data), a dynamical error (the motion of the state minus its predicted motion) and a prior error on the cause. Minimizing this is predictive coding: higher levels send predictions down, lower levels send errors up, and expectations move so as to cancel the errors. One elegant detail: at the minimum, "the motion of the mode should be the mode of the motion", so a belief keeps travelling at its believed velocity instead of freezing. Hierarchies are built by treating the cause v at one level as if it were data for the level above, with slower dynamics.

Action enters in one line. Its rate of change is set by the gradient of free energy with respect to action, and the only part of F that depends on action is the lowest sensory prediction error. So action simply makes sensations match descending predictions. "Active Inference is just predictive coding plus reflex arcs." Two things are conspicuously absent. There is no inverse model computing motor commands from goals, as there is in optimal control; the reflex arc is controller enough. And there is no cost function; whatever cost or value would have done is absorbed, more expressively, into the priors.

A principle is not a process theory. The principle says that free energy is minimized; a process theory says how a particular system, here the brain, does it, and in doing so makes predictions that data can refute. Chapter 5 offers the reading "most consistent with currently available evidence", and says plainly that it is not the last word. Its starting point is a structural coincidence. In a generative model, each variable depends on only a few others, so inference by message passing is sparse. In the brain, each neuron depends only on the neurons it shares synapses with, so connectivity is sparse. The proposal is that these two sparsities are the same one: messages are synaptic exchanges and gradient descent is neuronal activity.

1. The column

The neocortex is six layers deep and its wiring is stereotyped enough to repeat from region to region. Ascending inputs, from lower areas or from the primary thalamus, target the spiny stellate cells of layer IV. Ascending outputs leave from the superficial pyramidal cells of layers II and III. Descending inputs arrive from the deep pyramidal cells (layer VI) of higher areas, and the large layer V pyramidal cells (the Betz cells of the motor cortex) project to the basal ganglia, the thalamus and the spinal cord. Lay the predictive-coding network of Sheet IV over this anatomy and the pieces fit: ascending messages are prediction errors, descending messages are predictions, expectations live in the pyramidal cells. The asymmetry predicts something measurable. Turning an expectation into an error is a nonlinear operation, and nonlinearities push signals to higher frequencies (square a sine wave and its frequency doubles), so ascending messages should ride in faster bands (gamma) than descending ones (alpha and beta). That is what recordings show. For discrete models the messages passed between regions are mixtures of expectations rather than errors, but the shape survives: errors in layer IV, expectations in superficial and deep pyramidal cells. Where a drawing seems to disagree with the tissue (a descending inhibition that ought to land in layer III), the book notes that one arrow in a scheme may be two synapses in the cortex.

2. Motor commands are predictions

Follow layer V of the motor cortex down the pyramidal tract to the spinal cord. There the descending signal is subtracted from the incoming proprioceptive input; the difference drives the motor neurons; the muscles contract until the difference is gone. On this reading the corticospinal signal is not a command but a prediction of what the muscles are about to feel, and the movement is the reflex that makes the prediction true. "The only thing we should need to generate a movement is a prediction of the sensory consequences anticipated if that movement were to be executed." One anatomical oddity fits at once: the primary motor cortex has very few layer IV granular cells, which is to say almost no error units, because proprioceptive errors are resolved by reflexes rather than by changing one's mind.

There is a catch, which turns out to be a discovery. To begin moving you must believe "I am moving" while the proprioceptive data still say that you are not. Left to itself, that belief would be corrected at once. So the descending signals must predict not only the data but their precision, turning down the gain on proprioception long enough for the initially false belief to take hold and be made true. This is sensory attenuation, the complement of attention; saccadic suppression, the blindness during an eye movement, is an everyday instance. Its failures, the book proposes, lie behind passivity phenomena, catatonic states in schizophrenia and the failure to initiate movement in Parkinson's disease.

3. Deep structures: policies and dopamine

Layer V also projects to the striatum, the input nucleus of the basal ganglia. Its medium spiny neurons come in two kinds. Those bearing D1 dopamine receptors feed a direct pathway that disinhibits behaviour; those bearing D2 receptors feed an indirect pathway that suppresses it; dopamine enhances the first and damps the second. The book's mapping (its Figures 5.1 and 5.4): predicted outcomes and their divergence from preferred outcomes come together in layer V to compute expected free energy, G; the basal ganglia turn G into posterior beliefs over policies (read out in the internal globus pallidus and returned to cortex, probably through the thalamus). The direct pathway carries the fast, focused, goal-directed evaluation; the indirect pathway carries the fixed prior E, which says what one usually does in this context: habit. Reinforcement learning calls these model-based and model-free, a name the authors find slightly misleading since "both systems depend on models". Dopamine, on this account, sets the balance between inferring what to do and what not to do, and the quantity it encodes is the precision of beliefs about policies: confidence that one's plan will minimize expected free energy.

That reinterprets a famous finding. Dopamine neurons respond to unexpected fruit juice, and to a cue that predicts juice, in a way that looks like a "reward prediction error". In Active Inference, achieving a preferred outcome, or seeing a cue for one, raises the animal's confidence that it is on the right policy, and the rise in confidence is the dopamine spike. Parkinson's akinesia (no policy can be enacted) and the impulsivity produced by dopamine agonists sit at the two ends of the same dial. The thalamus gets a briefer treatment: its primary nuclei behave like lower cortical areas, relaying data and receiving predictions; its second-order nuclei (pulvinar, mediodorsal) may carry second-order statistics, that is, precision.

4. Chemicals as confidence

Precision is the multiplicative weight on a prediction error, in neural terms a synaptic gain, and different distributions in a generative model have different precisions: one can be more or less sure of one's senses, of how things will unfold, and of how one will act. The book's Table 5.1 assigns each a chemical.

Three payoffs. The mapping explains observed biology (the dopamine story above). It generates hypotheses: in Lewy body dementia, cholinergic loss would make the likelihood imprecise, so perception loses the constraint of sensation, which would explain that condition's complex visual hallucinations. And it suggests measurements: pupil size tracks noradrenaline and hence transition precision, blink rate tracks dopamine, so one can read parameters of a person's generative model from their eyes.

5. Learning as wiring

Fast changes of precision only turn a signal up or down. To change what a signal means you need learning: beliefs about the parameters of the model, whose substrate is the efficacy of synapses between the populations that represent states and outcomes. When an outcome is observed together with a belief that a particular state caused it, the connection between the two is strengthened. That is Hebb's edict, "cells that fire together, wire together", arrived at from statistics rather than assumed. A further prediction follows from Figure 5.1: likelihood connections enter and leave a column, while transition and dynamical connections stay inside it, so learning what things look like should change extrinsic connectivity and learning how things move should change intrinsic connectivity, which dynamic causal modelling of imaging data can test.

6. Continuous below, discrete above

Sensory receptors and muscles are continuous, so the lowest level of any neural hierarchy must be a continuous model. But categorical beliefs (what object, which person) live higher up, and policy selection in the basal ganglia is a choice among discrete movements. So movements are compositions of discrete trajectories, and the further one moves from the receptors the more discretized representations become; even a classical receptive field can be read as tiling a continuous state space into little categories. Sheet VIII builds the hybrid model this implies.

The chapter ends by listing what it has left out: most of the brain. Do parts of the amygdala do for autonomic policies what the basal ganglia do for skeletomotor ones? Does the pulvinar play the same role for mental policies? How should the very different microcircuits of the cerebellum and hippocampus be read? These are offered as questions, and as the kind of question a process theory makes askable.

The second part of the book turns from understanding to building, and its philosophy is top-down: "the challenge is not to emulate a brain, piece by piece, but to find the generative model that describes the problem the brain is trying to solve." Specify the model and the behaviour, the inference and the neural dynamics all follow, by minimizing free energy. Four questions do most of the work, and they are a genuine sequence, so they are numbered here.

1. Which system? Draw the blanket.

   Decide what is inside, what is outside, and what the interface is. The choice is real. Draw the blanket around a single neuron and its sensory states are receptor occupancies, its active states the rate of vesicle release, and the "world" it infers is the rest of the network. Draw it around the nervous system and the body becomes external, known only through interoception. Draw it around the whole organism and the skin depressing under a thumb is an inference about the thumb. Draw it around a person and their calculator, and the calculator is part of the machinery of inference (extended cognition). Blankets nest, from brains to organisms to communities. Fixing the line fixes what is inferred and what does the inferring; free energy is minimized only by the internal and active states, which see nothing but the sensory states.

2. What form of model? Three dichotomies.

   Discrete or continuous. Object identities, plans and other categories call for the bead-string model; positions, muscle lengths and luminance call for the stream (Sheet IV). A quantity can be discretized or a category smeared, so the choice is conceptual, but it is not idle: each form implies a different message passing and therefore a different prediction about neurons. Most current models put decisions in discrete variables and fine-grained perception and action in continuous ones, or combine the two.

   Shallow or deep. If the process has parts that change on separated timescales, use a hierarchy: slower above, faster below. Phonemes inside words inside sentences; "find keys, open door, enter" inside "go home". A useful licence follows: the inferences of lower, faster levels can be treated as the sensory states of the level you care about, so you need not model the whole brain to study one part of it.

   Temporal depth. Does the creature need to weigh alternative futures that depend on its own actions? A rat planning a route through a maze needs something like Tolman's cognitive map, a model of where each turn leads. Planning uses expected free energy, which is a functional of future observations, and future observations can only be predicted by a model that represents the consequences of action. Decide, too, how far ahead it plans. (A hierarchical model is not automatically a temporally deep one: predictive coding hierarchies consider only the present.)

3. Set up the model: variables, priors, and what is learned.

   Choose the hidden states, the observations and the actions. The concept of a jumping frog, in the book's example, is itself a small hierarchy: one amodal cause with visual (green, jumping), auditory (croaking), tactile (wrinkly) and interoceptive (racing heart) consequences, some of them contingent on action (touch it and it jumps). Factorize wherever factors are invariant to one another, such as what and where, instead of enumerating every combination. Then choose the priors. This is, the authors say, "the most challenging—but also the most creative—part of model design", and their practical heuristic is to model how an experimental psychologist would generate the stimuli. Behind the heuristic stands a theorem. Wald's complete class theorem says any decision procedure is Bayes-optimal under some set of priors, so "asking whether a behavior is (Bayes) optimal is meaningless. The important question is, What are the prior beliefs that would make this optimal?" Finally, decide what stays fixed and what is learned. Inference is the fast updating of beliefs about states and maps to neuronal dynamics; learning is the slow updating of beliefs about parameters and maps to synaptic plasticity; and giving a model learnable parameters gives it, for free, an appetite for novelty (Sheet VII). Parts of the model can be borrowed from machine learning (variational autoencoders, generative adversarial networks, "world models"), at the risk of losing the integration that makes the approach worth having.

4. Set up the process: the world the model lives in.

   In many of the book's simulations the process uses the same matrices as the model, but their meanings differ: the process's likelihood is an objective property of the environment (a measurement distribution), the model's is a subjective belief. The two can differ arbitrarily, and a game engine can serve as a process. The good regulator theorem says an effective regulator must be a model of what it regulates, but there is more than one way to be one. An explicit model resembles the environment (a map of the maze, with corridors and dead ends). An action-oriented model encodes only sensorimotor contingencies (whisker touch, turn) and can still escape the maze without representing a single corridor. And some control can be outsourced to the body: the passive dynamic walker, two legs and two arms with no sensors, motors or controllers, walks down an incline because its mechanics are tuned to the world. A creature can be a good model rather than merely have one.

Then run it. The standard routines (in the SPM software from University College London) simulate, plot, analyse and fit; the annotated code for the T-maze in the book's Appendix C fits on two pages. Two adjustments are common. One is to limit the planning horizon or sample policies, for large state spaces. The other is to ablate: remove a term of expected free energy and compare the creature with the impoverished cousins in Sheet II's family tree, or add habits or a learning rate, or turn a prior or a precision up or down to see a disorder appear. Sheets VII to IX do each of these.

1. A sessile listener

Start with a creature that cannot act, and a model with no policies: a hidden Markov model. An amateur pianist intends a sequence of notes (the hidden states) and produces the notes you hear (the outcomes). The likelihood A says she hits the note she intends 70 percent of the time; the transitions B say each note is followed by the next with 97 percent probability; the prior D says the piece starts on the first note. Now feed the listener a performance in which the third note comes out wrong. Its beliefs grow more confident with each note except at the third, where two explanations compete: the musician intended an unusual note (unlikely under B) or she slipped (likely under A). The simulated listener settles on the slip, but keeps a small probability for the alternative, and the capacity to report that residual uncertainty is the point of doing perception with probabilities. Add factors (pitch and loudness; in vision, what and where) and the same machinery scales without change.

2. The rat

Now let the creature act, and the hidden Markov model becomes a POMDP. The book's flagship example is a rat in a T-maze. One arm holds something attractive, the other something aversive, and a third arm holds a cue that says which is which. The rat can go straight to an arm and risk the aversive stimulus, or visit the cue first and then choose. Writing the model down takes five decisions, and they are the building blocks of every model in the chapter.

• Outcome modalities: two. Where I am (start, a leftward cue, a rightward cue, left arm, right arm) and what I feel (nothing, attractive, aversive).
• Hidden-state factors: two. Location (start, cue, left, right) and context (attractive on the left, or on the right). The likelihood A is therefore a tensor rather than a matrix; an arm yields the attractive outcome 98 percent of the time when the context matches.
• Transitions: four actions move the rat anywhere, except that the arms are absorbing: once in, it stays. Context never changes.
• Preferences: C is written as log-probabilities, +6 for the attractive outcome and −6 for the aversive one, so that the attractive outcome is e⁶ ≈ 400 times more probable than nothing under the rat's model. The rat's model is, in Sheet III's sense, optimistic.
• Initial state: certainly at the start; context fifty–fifty.

Nothing else is specified. The rat goes to the cue first, because that location has high epistemic value (it resolves the context), and then to the correct arm, because that has high pragmatic value. Its belief about the context jumps when it sees the cue, and the jump appears in the simulation as a local field potential of the kind one could record. Explore-then-exploit was not programmed. It fell out of minimizing G.

3. What epistemic value is made of

The epistemic value of a policy is a difference of two entropies. The first is the posterior predictive entropy: how uncertain I am about what I would see if I did this. The second is the expected ambiguity: how much of that uncertainty could never be resolved, because outcomes are only loosely tied to states. Their difference is the mutual information between states and outcomes under the policy, equivalently the expected Bayesian surprise, and it is exactly the score Lindley proposed in 1956 for choosing experiments. The best eye movement, like the best experiment, is toward something you are uncertain about and could actually learn from. Two simulations make the two terms visible. Four squares on a screen change colour from moment to moment; a viewer with no preferences samples all four about equally. Make one square's likelihood ambiguous (poorly lit) and it is ignored: the streetlight effect, named for the man who looks for his lost keys under the lamp, not because they are likely to be there but because that is where unambiguous information is. Make another square's transitions volatile and it is revisited again and again, because something may have changed since you last looked.

A short note on precision, which recurs on every sheet (Box 7.1). Any distribution can be made sharper or flatter by an inverse-temperature parameter that multiplies its log-probabilities: a gain control, amplifying rather than adding. Precision can itself be inferred, like any latent variable, and the growing suspicion, from Sheet V, is that the neuromodulators are its substrate.

4. Learning by counting

So far the matrices were fixed. Let the model hold beliefs about them too, with Dirichlet priors (the conjugate partner of categorical distributions, meaning the posterior keeps the same form), and learning becomes inference on a slower timescale. The update rule is a tally: each time an outcome is observed together with a belief about the state that caused it, add to the count for that pair; the probabilities are the normalized counts. This is activity-dependent plasticity, derived. Counts also encode confidence. Five heads in a row are alarming from a fresh coin and negligible after a hundred balanced flips. And once parameters are uncertain, expected free energy grows a third term, novelty, the expected information gain about parameters. "Salience is to inference what novelty is to learning." Given two coins, flip the one you have flipped less.

The book's demonstration is a myopic creature on a floor of black and white tiles. It can see only the tile beneath it and move only one tile at a time. Instead of a hidden state for each tile's colour, it keeps its knowledge in the likelihood: what I would see if I went there, learned by counting. Given no preferences at all, so that novelty alone drives it, it never revisits a tile it has already seen. Inhibition of return, a classic of attention research, emerges without being asked for.

Beyond parameters lies structure learning: should the model grow, or prune, a part of itself? That is model comparison, and Bayesian model reduction (Box 7.3) does it cheaply, scoring reduced models from a single inversion of the full one. The book proposes it as a metaphor for sleep, when no data arrive but the model can be simplified, and for the spontaneous activity of the resting brain.

5. Deep time: sentences and words

The last extension nests timescales. A state at the higher level changes slowly and generates a whole short trajectory at the lower level; for every tick above there are several ticks below. Reading is the example: words inside sentences. The book's toy language has three words (flee, feed, wait) and six four-word sentences. The higher level predicts which word comes next and hands that prediction down as an empirical prior for the lower level; the lower level foveates parts of the word to accumulate evidence, exactly as in the visual-search simulation; evidence flows back up, and sentence by sentence the alternatives are eliminated until one remains and the reader is "rewarded with correct feedback". The same architecture has simulated delay-period working memory, empirical priors for vision, and hierarchies of goals and motivation, and the learning models of the previous section are hierarchical in the same sense, slow parameters sitting above fast states. Stack as many levels as the world has timescales.

1. Movement as prophecy

In a continuous model, action does not appear in the generative model at all. The model says how states generate data and how states evolve; action belongs to the generative process, the world, and emerges from minimizing free energy. The simplest possible model makes the point. Let the hidden state x be drawn toward a point v (its rate of change is v − x), and let the data equal x. In the process, let the rate of change of x be whatever action sets it to. Then minimizing free energy through action makes the world fulfil the model: the limb moves to v. This is the equilibrium-point hypothesis of motor control, with the descending "set-points" of that theory reinterpreted as proprioceptive predictions. Make the model Newtonian, so that a force changes velocity and a spring (Hooke's law) pulls toward v, and the same logic produces a realistic reach. No inverse model is required.

Precision returns here in its most natural setting (Box 8.1), because in a continuous model it appears directly as the gain on each prediction error. Higher precision means more vigorous updating and larger evoked responses, and "from the perspective of active inference, precision and attention are synonyms"; cueing one location with higher precision reproduces the faster responses of the Posner paradigm. But precision has a second role that is the mirror of the first. To move you must, for a moment, ignore the evidence that you are still. An accurate belief that "I am not moving" can never generate the predictions that would move you; with precise proprioception the belief "I am moving" is falsified the instant it is formed. So a movement begins with sensory attenuation, a drop in precision, and ends with precision restored, in a cycle of attenuating and moving that the saccadic system displays plainly. Experiment 6 is a toy of this idea.

2. Sequences from ecology

Richer behaviour needs richer dynamics, and the book borrows two systems. The first is Lotka and Volterra's model of predators and prey. When predators are few, prey multiply; the predators then feed and multiply; the prey crash; the predators follow. Generalized to three populations (plant, herbivore, carnivore) the model produces a repeating sequence of peaks, which is to say a continuous system that behaves like a sequence of discrete steps. Two uses: in a model of eyeblink conditioning, the animal learns how many peaks separate a tone from the puff of air that follows it and blinks on time, a task of the cerebellum; and in a model of handwriting, each peak selects the next attracting point and the pen visits them in order. POMDPs have largely replaced these dynamics in practice, but they show how the bead-string of Sheet VII might be built from continuous stuff.

3. Chaos and conversation

The second system is Lorenz's model of atmospheric convection, three equations whose behaviour is chaotic and itinerant, orbiting one lobe of its attractor and then, unpredictably, the other. The book uses it to make songbirds. At the higher level a slow Lorenz attractor evolves; one of its dimensions sets a parameter of a faster Lorenz attractor at the lower level, whose variables map onto the sound spectrogram. Action drives the larynx to resolve prediction error, with a consequence that is easy to miss: a bird that hears the song it is predicting has no error to resolve and need not sing, while a bird that predicts a song it does not hear must start singing. Put two such birds together and turn-taking appears. While one sings, the other listens; when the first stops, the second continues the same song from where it left off: "singing from the same hymn sheet". Why alternate at all? Because of sensory attenuation: to act, a bird must turn down the precision of its own predicted consequences, so at any moment it is either listening or singing, as the eye is either looking or moving.

For this to work the two birds must synchronize their hidden states. Before they have learned each other's parameters, a plot of one bird's high-level expectations against the other's is a cloud. After learning, it lies along a line: identical synchronization of chaos, and a primitive theory of mind, since each bird can now infer what is going on in the other's head. In the general case the joint behaviour lies on a curved, low-dimensional surface, a synchronization manifold, and the book notes that a failure of this generalized synchrony has been proposed as a way of understanding autism. Experiment 7 shows the cloud becoming a line.

4. From decisions to movements, and back

The chapter ends by joining the two model shapes. A hybrid model puts a POMDP on top of a predictive coding scheme. Each discrete outcome is associated with a distribution over continuous causes (a target's coordinates); the prior for the continuous level is an average of these, weighted by the discrete beliefs; and the continuous level returns, for each discrete option, the evidence the continuous data provide for it (computed by the same Bayesian model reduction as in Sheet VII). The example is a monkey's delay-period task: fixate a cross, see a target flash in one of four places, remember it through a delay, then saccade to it on a signal. The top level infers which stage of the trial it is in and which location was cued; the bottom level makes the eye movement, and the movement, once observed, sharpens the discrete beliefs that caused it. Decisions become movements and movements become evidence for decisions.

1. Two inferences at once

Until now the models belonged to the creatures. In this chapter they belong to the scientist as well. A subject's brain uses a subjective generative model, with parameters we do not know, to produce behaviour. We build an objective model of that: prior beliefs about the parameters, and a likelihood that says how behaviour would look under each setting, obtained by simulating the subject with Active Inference and asking how probable the observed choices would be. Inverting the objective model on the data gives posteriors over the subject's priors. This is meta-Bayesian inference, "inferences about an inferential process", and its product is a computational phenotype: a description of an individual in terms of the beliefs that would make their behaviour Bayes-optimal. It serves two purposes. Estimating parameters, which can then be related to imaging or used to forecast responses to treatment; and comparing hypotheses, expressed as alternative models, by their evidence. The machinery is generic: any normative model of behaviour, reinforcement learning included, can be put in the likelihood's place.

The mechanics, briefly. The likelihood is the probability of the observed sequence of actions under the simulated subject's posterior beliefs about policies, with a softmax temperature to absorb "shaky-handedness": at temperature one the subject samples actions from its beliefs (matching behaviour); at high temperature it always takes the likeliest. Variational Laplace fits each subject, assuming Gaussian posteriors over parameters and climbing the log-likelihood plus prior by gradient ascent (finite differences suffice); it returns a posterior and an estimate of evidence. Parametric empirical Bayes (PEB) then models the fitted parameters across subjects as a linear function of explanatory variables (age, drug, diagnosis), and Bayesian model reduction tests which of those effects the data support without refitting anything.

1. Collect behaviour: the choices, the stimuli that prompted them, and whatever else you will want to explain differences with (patient or control, age, questionnaire scores).
2. Write the subjective model as a POMDP whose priors depend on the parameters of interest.
3. Specify the likelihood: a function that solves the POMDP for given parameters and stimuli and returns the probability of the observed actions.
4. Specify priors over the parameters, as expectations and precisions.
5. Invert with variational Laplace to get each subject's posterior and model evidence.
6. Analyse the group with PEB, or with ordinary statistics on the fitted parameters.

Before any real data, two checks: face validity (does the model generate plausible fictive behaviour?) and parameter recovery (given fictive data from known parameters, does inversion get them back? If not, they are not identifiable).

2. Two studies

Adams and colleagues had subjects track a moving target with their eyes, either along a predictable sinusoid or with noise added, while magnetoencephalography recorded their brains. The subjective model was continuous (Sheet VIII): the point of fixation attracted to the target. Inverting it gave each subject's precision parameters; dynamic causal modelling of the MEG gave each subject's synaptic gain in the superficial pyramidal cells of primary visual cortex; and the two were correlated, which is the sort of link between belief updating and its biology that the whole enterprise exists to make. Mirza and colleagues had subjects categorize a visual scene whose quadrants (a cat here, a bird there) were revealed only when fixated, so that several saccades were needed to gather enough evidence. The subjective model was a POMDP with a preference for being told one was correct. Models that included the epistemic term of expected free energy had higher evidence than models without it, in every subject; PEB showed the subjects' beliefs changing across trials, that is, active learning; and a canonical covariates analysis tied the phenotypes to accuracy and reaction time.

3. False inference

Men in general are quick to believe that which they wish to be true. — Julius Caesar, the section's epigraph

The hypothesis behind computational psychiatry, as the book states it, is that psychopathologies are disorders of inference. In most of the studies it surveys the inferential machinery works perfectly; what has gone wrong is the generative model, which holds aberrant priors, and those priors are exactly what the method above can recover from behaviour.

The book offers a way to organize this. The prior over policies depends on posterior beliefs, on the potential for information gain, on preferences and on habits, and each can fail. Posterior beliefs fail through the balance of precisions: trust the senses too much and you overfit noise; trust your priors too much and your percepts become immune to data; both are routes to hallucination, and both can coexist in a hierarchy, which fits the cholinergic loss of Lewy body dementia and the dopaminergic abnormalities of schizophrenia. Information gain fails when the precision of likelihoods (how much can be resolved) or of priors (how much there is to resolve) is wrong; a missing synapse is an unmodifiable, overconfident prior, so synaptic disconnection becomes a disorder of curiosity, as in neglect. Preferences and habits fail into addiction, and into the apathy of a creature that has lost its optimism.

Daniel Dennett once complained that cognitive science modelled isolated subsystems whose boundaries were arbitrary, and suggested modelling "the whole iguana" instead: a complete creature, perhaps a simple one, and a niche for it to cope with. Active Inference's promise is exactly a complete solution to what an organism must do, from which the familiar functions are read off afterward rather than assembled from parts. The textbook categories, the "Jamesian" ones (perception, memory, attention, intention, emotion), may not carve nature at its joints; they may be how our generative models explain our engagement with the world rather than what explains it. "The solipsistic hypothesis that 'I am perceiving' is just my explanation for current states of affairs that include my belief updating." A normative theory also finds analogies across domains, such as the exploration–exploitation trade-off appearing in foraging and again in searching one's memory under time pressure, and it ties every function to a neural signature through the process theory. The chapter then walks the contents page of a psychology textbook. Here are its entries as specimen cards.

The last page

The book ends with the reader. "Active Inference is not something that can be learned purely in theory." Write down a generative model; suffer the simulation that misbehaves; learn from your violated priors. And notice it in daily life: the compulsion to move your eyes toward the thing at the edge of vision, to resolve uncertainty; the choice of a favourite restaurant, to fulfil prior gustatory preferences; and reaching to turn down a shower that is too hot, "to ensure the temperature conforms to your model of how the world should be". Its final epigraph is Tolkien's: Home is behind, the world ahead, and there are many paths to tread.