Hybrid predictive coding: Inferring, fast and slow

Abstract

Predictive coding is an influential model of cortical neural activity. It proposes that perceptual beliefs are furnished by sequentially minimising “prediction errors”—the differences between predicted and observed data. Implicit in this proposal is the idea that successful perception requires multiple cycles of neural activity. This is at odds with evidence that several aspects of visual perception—including complex forms of object recognition—arise from an initial “feedforward sweep” that occurs on fast timescales which preclude substantial recurrent activity. Here, we propose that the feedforward sweep can be understood as performing amortized inference (applying a learned function that maps directly from data to beliefs) and recurrent processing can be understood as performing iterative inference (sequentially updating neural activity in order to improve the accuracy of beliefs). We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner by describing both in terms of a dual optimization of a single objective function. We show that the resulting scheme can be implemented in a biologically plausible neural architecture that approximates Bayesian inference utilising local Hebbian update rules. We demonstrate that our hybrid predictive coding model combines the benefits of both amortized and iterative inference—obtaining rapid and computationally cheap perceptual inference for familiar data while maintaining the context-sensitivity, precision, and sample efficiency of iterative inference schemes. Moreover, we show how our model is inherently sensitive to its uncertainty and adaptively balances iterative and amortized inference to obtain accurate beliefs using minimum computational expense. Hybrid predictive coding offers a new perspective on the functional relevance of the feedforward and recurrent activity observed during visual perception and offers novel insights into distinct aspects of visual phenomenology.

Abstract

Predictive coding is an influential model of cortical neural activity. It proposes that perceptual beliefs are furnished by sequentially minimising “prediction errors”—the differences between predicted and observed data. Implicit in this proposal is the idea that successful perception requires multiple cycles of neural activity. This is at odds with evidence that several aspects of visual perception—including complex forms of object recognition—arise from an initial “feedforward sweep” that occurs on fast timescales which preclude substantial recurrent activity. Here, we propose that the feedforward sweep can be understood as performing amortized inference (applying a learned function that maps directly from data to beliefs) and recurrent processing can be understood as performing iterative inference (sequentially updating neural activity in order to improve the accuracy of beliefs). We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner by describing both in terms of a dual optimization of a single objective function. We show that the resulting scheme can be implemented in a biologically plausible neural architecture that approximates Bayesian inference utilising local Hebbian update rules. We demonstrate that our hybrid predictive coding model combines the benefits of both amortized and iterative inference—obtaining rapid and computationally cheap perceptual inference for familiar data while maintaining the context-sensitivity, precision, and sample efficiency of iterative inference schemes. Moreover, we show how our model is inherently sensitive to its uncertainty and adaptively balances iterative and amortized inference to obtain accurate beliefs using minimum computational expense. Hybrid predictive coding offers a new perspective on the functional relevance of the feedforward and recurrent activity observed during visual perception and offers novel insights into distinct aspects of visual phenomenology.

Author summary

Predictive Coding Networks (PCNs) are a neurobiologically plausible model of cortical processing that can be applied to machine learning tasks. However, they require a computationally costly inference phase to generate predictions. We propose adding an amortized feedforward network to the model which learns to predict the outcome of iterative inference, and uses these predictions to initialize the predictive coding network—an approach we call hybrid predictive coding. This allows our hybrid model to perform simultaneous classification and generation and can be trained much faster and with less data than a standard PCN. Our model can also naturally and adaptively vary its computation time according to task demands, and may also help shed light on the neurocomputational basis of some otherwise difficult-to-understand aspects of visual phenomenology, thus suggesting that the brain may utilize a similar hybrid inference approach in visual processing.

Citation: Tscshantz A, Millidge B, Seth AK, Buckley CL (2023) Hybrid predictive coding: Inferring, fast and slow. PLoS Comput Biol 19(8): e1011280. https://doi.org/10.1371/journal.pcbi.1011280

Editor: Ulrik R. Beierholm, Durham University, UNITED KINGDOM

Received: August 25, 2022; Accepted: June 20, 2023; Published: August 2, 2023

Copyright: © 2023 Tscshantz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All data are in the manuscript and/or supporting information files. Our code is available on github online at https://github.com/alec-tschantz/pybrid.

Funding: This work was supported by the European Research Council (Advanced Investigator Grant 101012954 to AKS), the Canadian Institute for Advanced Research (Program in Brain, Mind, and Consciousness to AKS), the Dr. Mortimer and Theresa Sackler Foundation (part PhD studentship to AT via the Centre for Consciousness Science), the School of Engineering and Informatics at the University of Sussex (part PhD studentship to AT). It was additionally supported by the Biotechnology and Biological Sciences Research Council (Grant BB/P022197/1 to CLB) and (Grant BB/S006338/1 to BM). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

1 Introduction

A classical view of perception is as a primarily bottom-up pipeline, whereby signals are processed in a feed-forward manner from low-level sensory inputs to high-level conceptual representations [1–3]. In apparent contrast with this classical bottom-up view, a family of influential theories—originating with von Helmholtz in the 19th Century—have cast perception as a process of (approximate) Bayesian inference, in which prior expectations are combined with incoming sensory data to form perceptual representations [4–8]. Under this Bayesian perspective, the loci of perceptual content reside predominantly in top-down predictions rather than in the sequential refinement of bottom-up sensory data.

In visual perception, top-down signalling has long been recognised as playing several important functional roles [9]—for instance, in attentional modulation [10], in the goal-directed shaping of stimulus selection [11, 12], and in establishing recurrent loops that have been associated with conscious experience [13]. At the same time, bottom-up signalling has been convincingly linked to rapid perceptual phenomena such as gist perception and context-independent object recognition [9, 14–17]. These and other disparate findings have fueled a long-standing debate over the respective contributions of bottom-up and top-down signals to visual perceptual content [9, 18–21]. Although classical bottom-up perspectives are often contrasted with Bayesian top-down theories in this debate, more nuanced pictures have also been proposed in which bottom-up and top-down signals both contribute to perceptual content, but in distinct ways [21–23]. Such proposals, however, have largely remained conceptual. Here, we develop, and illustrate with simulations, a novel computational architecture in which top-down and bottom-up signalling is adaptively combined to bring about perceptual representations within an extended predictive coding paradigm. We call this architecture hybrid predictive coding (HPC). We show that while both bottom-up and top-down signals convey predictions about perceptual beliefs, they implement different approaches to inference (amortized and iterative inference, respectively). Our model retains the benefits of both approaches to inference in a principled manner, and helps explain several empirical observations that have, until now, evaded a parsimonious explanation in terms of Bayesian inference.

Predictive coding is a highly influential framework in theoretical neuroscience which originated in signal processing theory and proposes that top-down signals in perceptual hierarchies convey predictions about the causes of sensory data [6, 8, 24–32]. Predictive coding is usually considered in systems with multiple hierarchical layers [30, 33–35], where each layer learns to predict (or generate) the activity of the layer below it (with the lowest layer predicting sensory data). In this setting, bottom-up signals convey prediction errors—the difference between predictions and data—whereas top-down signals convey the predictions (although see alternative predictive coding schemes which depart from this template [36], although in different ways to our proposed hybrid predictive coding model.). By minimising prediction errors, a system can both learn a generative model of its sensory data and infer the most likely causes of that data in a hierarchical fashion [37]. The resulting scheme can be interpreted as performing variational inference [29], an optimisation procedure that approximates Bayesian inference [38–40].

Predictive coding can account for a wide range of neurophysiological evidence and provides a compelling account of top-down signals in visual perception [41]. However, to extract meaningful representations from sensory data, predictive coding must iteratively minimise prediction errors over multiple sequential steps [42, 43]. We refer to such procedures as iterative inference, as they require multiple iterations to furnish perceptual beliefs [44–46]. In neural terms, this would imply that multiple cycles of recurrent activity are required to perceive a stimulus [17, 33, 47]. However, empirical studies have consistently demonstrated that many aspects of human visual perception can occur remarkably rapidly, often within 150ms of stimulus onset [14, 48–50]. This evidence of rapid perception—such as in gist perception or context-free object recognition—is difficult to reconcile with a computational process that requires several sequential steps to arrive at perceptual representations.

In machine learning, amortised inference provides an elegant alternative to iterative inference [51, 52] which is well suited to rapid processing. Rather than iteratively updating beliefs for each stimulus, amortised inference learns a parameterised function (such as a neural network) that maps directly from the data to (the parameters of) beliefs. The parameters of this function are optimised across the available dataset, and once learned, inference proceeds simply by applying the learned function to new data. Thus, amortised inference provides a plausible mechanism for extracting beliefs rapidly and efficiently via feed-forward, bottom-up processing [47, 53].

Our novel neural architecture—hybrid predictive coding (HPC)—combines (standard) iterative predictive coding with amortised inference, so that both bottom-up and top-down signals convey predictions (and where prediction errors also flow in both directions) (see Fig 1). The architecture comprises several hierarchical layers, with top-down signals predicting the activity of the subordinate layer and bottom-up signals predicting the superordinate layer’s activity. As with predictive coding, top-down predictions learn to generate the data hierarchically, implementing a ‘generative model’ of the data. The model augments standard predictive coding by also implementing bottom-up predictions which learn to generate (the parameters of) beliefs at higher layers. Crucially, these bottom-up predictions learn to generate beliefs that have been optimised by iterative inference, i.e. they learn to generate approximately posterior beliefs. In this way, our model casts bottom-up processing as amortised inference and top-down processing as iterative inference. This means that both bottom-up and top-down signals convey transformations of errors and transformations of activity. This contrasts with traditional predictive coding, where bottom-up signals convey a transformation of errors while top-down signs convey a transformation of activity (predictions).

[Figure omitted. See PDF.]

Fig 1. Bottom-up and top-down perception.

One classical view of perception is as a primarily bottom-up process, where sensory data x is transformed into perceptual representations z through a cascade of feedforward feature detectors. In contrast, predictive coding suggests that the brain solves perception by modelling how perceptual representations z generate sensory data x, which is a fundamentally top-down process. Although there is bottom-up information flow [in PC] in terms of errors, this bottom-up information conveys errors (transformed by the synaptic weights). However, in our hybrid predictive coding models, bottom-up information conveys both predictions and errors (along with top-down information). In HPC, sensory data x predicts perceptual representations at fast, amortized time scales, and perceptual representations z predict sensory data x at slow, iterative time scales. Our “fast and slow” model casts this integration of bottom-up and top-down signals in a common framework, allowing derivation of a testable process theory.

https://doi.org/10.1371/journal.pcbi.1011280.g001

At stimulus onset, bottom-up predictions rapidly provide an initial “best guess” at perceptual beliefs, which are then refined by minimising prediction errors iteratively in a top-down fashion. Both the bottom-up and top-down processes operate using the same set of biologically plausible Hebbian learning rules, and all layers of the network operate in unison to infer a single set of consistent beliefs. Altogether, the model offers a unified inference algorithm that inherits the rapid processing of amortised inference while maintaining the flexibility, robustness, and context sensitivity of iterative inference [8, 46, 51, 54]. Hybrid predictive coding provides a method for performing inference that is both fast (amortized) and slow (iterative) [55].

The remainder of the paper is structured as follows. Section 2 provides an overview of iterative and amortised inference and describes the hybrid predictive coding (HPC) architecture. In Section 3, we present results from a series of experiments that explore several aspects of our model. Specifically, we demonstrate a) that HPC performs supervised and unsupervised learning simultaneously, b) that bottom-up, amortised predictions reduce the number of iterations required to achieve accurate perceptual beliefs, and that the trade-off between amortised and iterative inference is adaptively modulated by uncertainty, and c) that the generative component of the model enables learning with a limited amount of data. Together, these results show the benefits of inferential process theories that incorporate bottom-up predictions which convey perceptual content rather than just errors. Conversely, they demonstrate that bottom-up approaches to perception should benefit from incorporating top-down generative feedback. Finally, we argue that our model provides a powerful computational framework for interpreting the contributions of bottom-up and top-down signalling in terms of different aspects of visual perception.

1.1 Related work

Many works in machine learning have considered the notion of iterative and amortized (variational) inference [40, 51, 56], with recent work combining iterative and amortized inference in the context of perception [46] and control [44, 45, 57, 58]. This idea of combining model-free and model-based planning methods (interpreted as iterative and amortized inference) in reinforcement learning also has a long history [59, 60]. Moreover, in perception, [61] also consider a feedforward amortized sweep to initialize an iterative inference algorithm in the context of contrastive Hebbian learning algorithms which are another proposed family of biologically-plausible learning algorithms which could potentially be implemented in neural circuitry [62, 63]. Contrastive Hebbian methods differ from predictive coding in that they require both a ‘free phase’ where the network output is unclamped and a ‘fixed phase’ where the network output is clamped and then the weight update is proportional to the difference between the two phases. In a similar approach to ours, [61] show that the fixed and free phase equilibria can be amortized and predicted in a feedforward pass and that this reduces the number of inference iterations required. However, to our knowledge, our approach is the first to combine iterative and amortized inference within a predictive coding architecture, and the resulting network has many favourable theoretical properties such as requiring only local Hebbian updates and that all dynamics and weight changes can be derived from a joint optimization on a unified energy function. In addition, given that predictive coding has been proposed as biological process theory of perception [8, 41, 42, 64], and as a way to explain the phenomenology of perceptual experience in terms of neural mechanisms [65, 66], our novel architecture also offers insights into why gist perception and focal perception have the characteristic phenomenological properties that they do. Our work is closely related to symmetric predictive estimators (SPE) [67], a model that learns separate feedforward and feedback weights using local learning rules. Similar to our work, these feedforward and feedback weights learn bottom-up and top-down projections of activity and are learned in a biologically plausible manner using local error units. Our work differs in several ways, both in technical content and theoretical motivation. In terms of technical content, the update equations have the neuron’s states continually updated based on bottom-up and top-down information, rather than the temporal asymmetry found in HPC. These differences mean that the SPE does not enable the interplay of bottom-up and top-down signals of HPC. On a theoretical level, the SPE is not directly motivated by the Bayesian brain hypothesis and does not look to provide a common framework for feedforward and recurrent activity in the brain.

2 Methods

2.1 Approximate Bayesian inference

Bayesian inference.

To support adaptive behaviour, the brain must overcome the ambiguous relationship between sensory data and their underlying (hidden) causes in the world. For example, suppose an object reflects some pattern of light onto the retina, the brain must recover this object’s identity, despite the fact that the sensory data is inherently noisy, and that multiple objects could have caused the same pattern of retinal stimulation. Such considerations have motivated the popular view that the brain uses a version of Bayesian inference [7, 68], which describes the process of forming probabilistic beliefs about the causes of data, to accomplish perception.

Formally, we can denote sensory data (e.g., the pattern of retinal stimulation) as x and the hidden cause of this data (e.g., the object causing the retinal stimulation) as z. Rather than directly calculating the most likely hidden cause, a Bayesian perspective would propose that the brain infers a conditional distribution over possible causes p(z|x), referred to as the posterior distribution. Bayesian inference then prescribes a method for updating the posterior distribution in light of new sensory data:(1)where p(x|z) is referred to as the likelihood distribution, describing the probabilistic relationship between hidden causes and sensory data, p(z) is the prior distribution, describing the prior probability of hidden causes, and p(x) = ∫ p(x|z)p(z)dz is the evidence, describing the probability of some sensory data averaged over all possible hidden causes. Bayesian inference prescribes a normative and mathematically optimal method for updating beliefs when faced with uncertainty and provides a principled approach for integrating prior knowledge and data into inferences about the world [69–71].

Variational inference.

While Bayesian inference provides an elegant framework for describing perception, the computations it entails are generally mathematically intractable [38]. Therefore, it has been suggested that the brain may implement approximations to Bayesian inference. In particular, it has been suggested that the brain utilises variational inference [38–40, 56, 72–74], which converts the intractable inference problem into a tractable optimisation problem. Variational inference posits the existence of an approximate posterior qλ(z) with parameters λ, which serves as an approximation to the true posterior distribution p(z|x). The goal of variational inference is then to minimise the difference between the true and approximate posteriors, with the difference being quantified in terms of the KL-divergence DKL. The KL-divergence is an asymmetric measure of dissimilarity between two probability distributions.:(2)

However, to minimise Eq 2, it is still necessary to evaluate the true posterior distribution p(z|x). Variational inference circumvents this issue by instead minimising an upper bound on Eq 2, i.e., a quantity which is always greater than or equal to the quantity of interest. In particular, it minimises the variational free energy :(3)

Minimising variational free energy will ensure that the qλ(z) tends towards an approximation of the true posterior (see the final line of Eq 3), thus implementing an approximate form of Bayesian inference. This minimisation takes place with respect to the parameters of the approximate posterior λ, and can be achieved through methods such as gradient descent.

Learning.

Eq 3 introduces an additional joint distribution over hidden causes and sensory data p(z, x), which is referred to as the generative model and is expressed in terms of a likelihood and a prior p(z, x) = p(x|z)p(z). It is common to parameterise the generative model with a set of parameters θ, e.g pθ(z, x). These parameters can then be optimised (over a slower timescale) with respect to variational free energy, thereby providing a tractable method for learning [39, 75, 76]. Intuitively, this is because the variational free energy provides a bound on the marginal-likelihood of observations pθ(x), such that minimising free energy with respect to θ will maximise pθ(x) [77]. The subscript θ highlights that the marginal likelihood is evaluated under the generative model. Minimising with respect to θ will thus cause the generative model to distill statistical contingencies from the data, and by doing so, encode information about the environment. In summary, variational inference provides a method for implementing both inference and learning using a single objective—the minimisation of variational free energy.

Iterative inference.

Variational inference provides a general scheme for approximating Bayesian inference. In practice, it is necessary to specify the approximate posterior and generative model, as well as the optimisation scheme for minimising variational free energy. A standard method is to optimise the parameters of the variational distribution λ for each data point. Given some data point x, we look to solve the following optimisation procedure:(4)

Generally, this is achieved using iterative procedures such as gradient descent. Therefore, we refer to this mode of optimisation as iterative inference [44, 46], as it requires multiple iterations to converge, and the optimisation is performed for each data point individually. Heuristically, for each new data point x, iterative inference randomly initialises λ, and then uses standard optimization algorithms such as gradient descent to iteratively minimise Eq 4. This method underwrites a number of popular inference methods, such as stochastic variational inference [78] and black box variational inference [79], and can be considered to be the ‘classical’ approach to variational inference. In what follows, we specify how predictive coding can be considered as a form of iterative inference.

2.2 Predictive coding

The predictive coding algorithm [6, 8] operates on a hierarchy of layers, where each layer tries to predict the activity of the layer below it (with the lowest layer predicting the sensory data). These predictions are iteratively refined by minimising the prediction errors, (i.e., the difference between predictions and the actual activity) of each layer. In predictive coding, the approximate posterior is defined to be a Gaussian distribution:(5)where λ = {μ, σ2}. In a similar fashion, we assume the factors of the generative model pθ(z, x) = p(x|z)p(z) to also be Gaussian:(6)where fθ(⋅) is some non-linear function with parameters θ, and is the mean of the prior distribution p(z). In hierarchical models, would not be fixed but would instead act as an empirical prior. Given these assumptions, we can rewrite variational free energy as ([30]):(7)where εl and εp are the prediction errors. The term fθ(μ) can be construed as a prediction about the sensory data x, such that εl quantifies the disagreement between this prediction and the data. The same logic applies for εp, which will be discussed further in the context of hierarchical models. Crucially, variational free energy is now written in terms of the sufficient statistics of qλ(z), i.e., the objective is now rather than (see [30] for a full explanation).

The goal is now to find the value of μ which minimises . This can be achieved through gradient descent (with some step size κ)(8)

While these updates may look complicated, they can be straightforwardly implemented in biologically-plausible networks composed of prediction units and error units (see [29]). The same scheme can be applied to learning the parameters of the generative model θ, where the goal is now to find the value of θ which minimises :(9)where κ is the learning rate. For each data point, μ is iteratively updated using Eq 8 until convergence, and then Eq 9 is updated based on the converged value of μ, here denoted μ* in considerations of HPC below. Crucially, Eq 9 can be implemented using simple Hebbian plasticity [29, 42]. Predictive coding is usually implemented in networks with L hierarchical layers, where each layer tries to predict the activity of the layer below it (besides the lowest layer, which predicts the data):(10)

When considered as a neural process theory, predictive coding posits the existence of two neuronal populations: prediction units (which compute predictions) and prediction error units (which compute the difference between a predictions and the actual input) [80, 81]. To make predictions match input data, the dynamics described by Eqs 8 and 9 prescribe that prediction errors are minimised over time. This ensures that contextual information from superordinate layers are integrated into the representations of lower layers during inference, thereby helping to disambiguate ambiguous stimuli [9, 24, 82]. As variational free energy is equal to the sum of (precision-weighted) prediction errors (Eq 7), minimising prediction errors is equivalent to minimising variational free energy, and thus equivalent to performing inference on a generative model. It is for this reason that predictive coding is often considered a ‘probabilistic’ theory of learning (as the model variables can be read as sufficient statistics of posterior beliefs), even when all of the distributions are point mass or with fixed second-order statistics.

Predictive coding models the world in a top-down manner—e.g., it learns to predict features from objects, rather than predicting objects from features. It is this aspect which makes predictive coding generative (While predictive coding is usually considered to be unsupervised algorithm, it is straightforward to extend the scheme into a supervised setting [29, 83–86]. This can be achieved by turning the predictive coding network on its head, so that the model tries to generate hidden states (e.g., labels) from data.)—as it is able to generate data without any external input [33]. Inference is achieved by ‘inverting’ this model—i.e., going from data (stimuli) to hidden states (latent features). [81]. This inversion process is achieved by iteratively applying Eq 8 for a given number of steps. The generative nature of predictive coding means that it is able to take context into account during inference, and it can work with relatively small amounts of data. On the other hand, its inherently iterative nature is computationally costly and temporally slow, and—in addition—in standard implementations predictive coding is also memoryless [53], meaning that inference is repeated afresh for each stimulus, even if that stimulus has been encountered previously, Note that memoryless inference can be useful, in so far as it ignores any biases which may have been introduced by previous data points.

Amortised inference.

Amortised inference provides an alternative approach to performing variational inference which has recently gained prominence in machine learning [51]. Rather than optimising the variational parameters λ directly, amortised inference learns a function fϕ(x) which maps from the data to the variational parameters. The parameters ϕ of this function are then optimised over the whole dataset, rather than on a per-example basis. Once this function has been learned, inference is achieved via a single forward pass through fϕ(x), making amortised inference extremely efficient once learned. Amortised inference also retains some notion of memory [52], as the amortised parameters ϕ are shared across the available dataset. However, amortised inference cannot take contextual information into account, and suffers from the amortisation gap [54], which describes the decrease in performance incurred from sharing parameters across the dataset rather than optimizing them individually for each data point.

Formally, amortised inference solves the following optimisation problem:(11)where is the available dataset, and fϕ(x) is the amortised function which maps from the data x to the variational parameters λ. The goal is then to optimise the parameters of this amortised function ϕ to minimize the variational free energy on average over the entire dataset.

In contrast with predictive coding, amortised inference is fundamentally a bottom-up process: it predicts objects from features. In the current context, fϕ(x) acts as a discriminative model which learns the parameters of the variational posterior q(z). Moreover, during inference, the amortised parameters are fixed. Several methods have been proposed for implementing amortised inference [51, 87], and these usually rely on some form of backpropagation or stochastic sampling to compute or approximate average gradients of the amortisation function with respect to the free energy. In the following section, we present an extension of predictive coding which incorporates amortised inference in a biologically plausible manner.

2.3 Hybrid predictive coding

Our novel hybrid predictive coding (HPC) model combines both amortised and iterative inference into a single biologically plausible network architecture. Our claims of biological plausibility inherit from the predictive coding framework, which adheres to the principles of local computation and local plasticity [29–31, 64]. We do not claim this model to be any more (or less) biologically plausible than networks that meet these criteria. However, we do claim that our models have an increased degree of biological relevance due to the fact that they can account for empirical observations that evade explanation in a traditional predictive coding framework. We consider a model composed of L hierarchical layers, where each layer i is composed of a variable unit μi and an error unit εi. In the same manner as predictive coding, each layer tries to predict the activity of the layer below it: μi−1 = fθ(μi), besides the lowest layer, which tries to predict the data x directly. The error units measure the disagreement between these predictions and the actual input .

In contrast with predictive coding, we assume an additional set of amortised parameters ϕ, which correspond to bottom-up connections. The amortised parameters define non-linear functions which map activity at one layer to activity at the layer above, thereby implementing a bottom-up prediction: . Here, the lowest layer operates directly on sensory data: . We refer to these bottom-up functions fϕ(⋅) as being amortised since they directly map from data x to the parameters of a distribution μ. Crucially, both the top-down fθ(⋅) and bottom-up fϕ(⋅) functions try to predict the same set of variables {μ}.

Inference proceeds in two stages. The first ‘amortised phase’ takes the current sensory data x and propagates it up the hierarchy in a feedforward manner, utilising the amortised functions fϕ(⋅). This produces a set of initial ‘guesses’ for each μ in the hierarchy and is analogous to the feedforward sweep observed in neuroscience [19] and in artificial neural network architectures [88]. The second ‘iterative’ phase refines each μ by generating top-down predictions and iteratively applying Eq 8.

Once inference is completed for the current input, generative and amortized parameters are updated via learning. In order to learn the generative parameters θ, Eq 9 is applied to the converged values of μ*. In order to implement learning for the amortised parameters ϕ, we introduce an additional set of error units which quantify the difference between the amortised predictions and the values of μ at convergence in the iterative phase, which we call μ*. These amortised prediction errors are defined as . Given these errors, we can update the values for ϕ using Eq 9, where fθ(μ) is now replaced by fϕ(μ). By constructing the model in this way, the process of amortised inference is symmetric to the original predictive coding model, except that predictions now also flow in the opposite (bottom-up) direction.

A key aspect of the model is that the amortised predictions learn to predict beliefs at higher layers, after the beliefs have been optimised by iterative inference. In effect, the amortised predictions learn to ‘shortcut’ the costly process of iterative inference, allowing for fast and efficient mapping from data to refined beliefs. Fig 2 provides a schematic of the model, and Algorithm 1 presents the details of the inference and learning procedure.

[Figure omitted. See PDF.]

Fig 2. Hybrid predictive coding combines two phases of inference as follows.

(A) At stimulus onset, data x is propagated up the hierarchy in a feedforward manner, utilising the amortised functions fϕ(⋅). These predictions set the initial conditions for μ, which parameterise posterior beliefs about the sensory data. These predictions are associated with error units that track the difference between variables at one level and the variables at the level above under transforms fϕ(⋅). These errors are not utilised for inference but are used to update the amortised parameters ϕ during learning (weight updates). (B) The initial values for μ are then used to predict the activity at the layer below, transformed by the generative functions fθ(⋅). These predictions incur prediction errors ε, which are then used to update beliefs μ. This process is repeated N times, after which perceptual inference is complete.

https://doi.org/10.1371/journal.pcbi.1011280.g002

Algorithm 1 Hybrid predictive coding

Input: Generative parameters θ—Amortised parameters ϕ—Data x—Step size κ—Learning rate α

Amortised Inference:

for i = 1…L − 1 do

end

Iterative Inference:

for optimisation iteration j = 1…N do

 for i = 1…L do

 end

end

Learning of Parameters:

for i = 0…L do

end

3 Results

To illustrate the performance of HPC, we present a series of simulations on the MNIST dataset, which consists of images of handwritten digits (from 0-9) and their corresponding labels. We first establish that HPC can simultaneously perform classification and generation tasks on the MNIST dataset. We then show that the model enables fast inference, in that the number of iterations required to achieve perceptual certainty reduces over repeated inference cycles. Moreover, we show that the novelty of the data adaptively modulates the number of iterations enabling more rapid adaptation to nonstationary environments and distribution shift. We then demonstrate the practical benefit of fast inference by plotting the accuracy of hybrid and standard predictive coding against the number of iterations, which demonstrates that our model can retain high performance with minimal iterations relative to standard predictive coding. To demonstrate the benefits of the top-down, generative component of our model, we compare the accuracy of HPC inference as a function of the dataset size and show that it can learn with substantially fewer data items than a purely amortized scheme. Finally, we investigate additional beneficial properties of our model. We show that the iterative inference phase can be accurately described as refining beliefs since it decreases the entropy of the initial amortized prediction. We show that our network can adaptively reduce computation time for well-learned stimuli but increase it again for novel data, as well as that combining the iterative and amortized components substantially reduces the number of inference iterations required throughout training.

3.1 Simulation details

The MNIST database consists of 50,000 training examples and 10,000 test examples. Each example is composed of an image and a corresponding label between 0 and 9, where each image is black and white and of size 28 x 28 pixels, which is fed into the predictive coding network via an input layer consisting of 784 nodes. In the context of both hybrid and standard predictive coding, labels are encoded as priors at the highest level of the hierarchy L. Specifically, the model’s highest layer is composed of 10 nodes (one for each label). During training, these nodes are fixed to the corresponding label: the node which corresponds to the label is fixed at one, while the remaining nodes are set to zero. The bottom (sensory) layer is fixed to the current image during training. During testing, while the bottom layer remains fixed to the image, the highest layer is left unconstrained. To obtain a classification during testing, we return the label which corresponds to the top-layer node with the largest activity at the end of inference. For generation, we fix the top layer of the network to a desired label and leave the input nodes unconstrained. We then perform inference throughout the network until convergence and read out the inferred image at the bottom layer.

Both the hybrid and standard predictive coding models are composed of 4 layers of nodes (L = 4). The lowest layer, which is fixed during training and testing, comprises 784 nodes and corresponds to the current image. The next two layers are composed of 500 nodes each, and the highest layer is formed of 10 nodes, which correspond to the current label and are constrained during training. For both the hybrid and standard predictive coding models, the generative, top-down functions fθ(⋅) use tanh activation functions for all layer besides the lowest, which do not use an activation function. Weight normalisation, where the absolute values of the weights in a layer are divided by the mean of the weights of the layer, is used for the generative parameters θ, which we found crucial for maintaining classification accuracy in the standard predictive coding network. For the amortised, bottom-up functions fϕ(⋅) (only used in the hybrid model), a tanh activation is used for all layers besides the highest, which does not use an activation function. All weights are updated using the ADAM optimiser [89] with a learning rate of α = 0.01, and κ = 0.01 is used for iterative inference. Th ADAM optimiser is used as a generic optimiser for updating parameters via gradient descent. While the ADAM optimiser is not biologically plausible, we found that the relative results between different models identified in the paper could be recreated with alternative optimisers, such that the specific optimiser used could be treated as a black box for our key results. Unless specified otherwise, we use N = 100 iterations during iterative inference. To demonstrate the adaptive computation properties of HPC we also use an adaptive threshold which cuts off inference if the average sum (across layers) of mean squared prediction errors is less than 0.005.

In contrast with standard presentations of MNIST results, we do not measure accuracy over entire epochs (e.g., the test set accuracy after the model has been trained on all 60,000 examples in a batched fashion) but instead measure accuracy as a function of batches. Specifically, we compute train and test accuracy after every 100 batches, where the batch size is set to 64 for all experiments. This strategy was chosen due to the speed at which our models converge (often within 600 batches, or approximately 38,000 examples), thereby allowing us to visualise convergence in a more fine-grained manner.

We calculate amortised accuracy by an amortised feedforward sweep and using the resulting activations of the final layer to calculate the loss, and the amortised model is then updated based on the bottom-up errors between results from this initial feedforward sweep and the final activation derived from iterative processing.

3.2 Unsupervised and supervised learning within a single algorithm

The first set of simulations illustrate that the model can perform both classification and generation simultaneously, meaning that it can naturally utilise both supervised and unsupervised learning signals. This property is desirable for a perceptual inference algorithm, since in many situations, training labels may only be available occasionally. The unsupervised capabilities of HPC derive from learning the top-down generative parameters θ. In the absence of labels (or priors in the current framework), these parameters distil statistical regularities in the data, forming a generative model which can be used for various downstream tasks. The supervised learning capabilities derive from both the Bayesian model ‘inversion’ provided by iterative inference, and the bottom-up initialisation provided by amortised inference. Our model captures the relationship between data and labels in a probabilistic manner by constraining the highest layer’s nodes to the relevant labels during training. It is important to note that the ability to utilise both supervised and unsupervised signals is not unique to hybrid predictive coding—standard predictive coding can also learn from both supervised and unsupervised signals. However, as we will show in the following experiments, our hybrid architecture affords several additional benefits which are not provided by standard predictive coding.

We first demonstrate the classification accuracy and generative capabilities of HPC. We compare the results of hybrid and standard predictive coding on the MNIST dataset, and additionally, compare these results to the accuracy of the amortised component alone. Recall that the amortised accuracy corresponds to the initial ‘best guess’ provided by the amortised forward sweep, which is then refined by iterative inference to give the final accuracy of hybrid predictive coding. The present analysis therefore allow us to determine the influence that iterative inference has on the hybrid predictive coding model.

Results are shown in Fig 3. There is no significant difference between the classification accuracy of the hybrid and standard predictive coding (Fig 3A). This is to be expected, as the iterative inference procedure (shared by both hybrid and standard predictive coding) ‘trains’ the amortised component (as the amortized connections learn to minimize the prediction error between their own predicted beliefs and the beliefs eventually converged to in the process of iterative inference), meaning that the amortised component’s accuracy cannot be higher than that provided by iterative inference alone. This ‘training’ can be seen in Fig 3A, where the amortised accuracy converges to the hybrid model’s accuracy over time. The accuracy of amortised inference increases at a slower rate. This result is likely due to the fact that amortised inference is trying to predict the equilibrium point of iterative inference, which is changing as the parameters of the network converge. Similarly, Fig 3B shows that the hybrid and standard predictive coding models are equivalent in terms of their ability to generate data, and Fig 3C & 3D show that samples generated from each of these models are qualitatively similar. Again, these results are expected, as the process of amortised inference should have little influence on the learning of the generative parameters. As illustrated later, the predictive coding and hybrid predictive coding architectures possess a powerful capacity to learn from limited data samples, which may account for the superior performance of the predictive coding network during initial training stages. Nevertheless, the predictive coding network’s accuracy shows a marginal decline due to training procedure instabilities that are absent in hybrid predictive coding. It is worth noting that the asymptotic performance is somewhat lower than usually reported on the MNIST dataset. This discrepancy is explained by the fact that we are using models that are fundamentally generative, i.e., their objective is to generate the data, not perform classification.

[Figure omitted. See PDF.]

Fig 3. Simultaneous classification and generation.

(A) Classification accuracy on the MNIST dataset for hybrid predictive coding, standard predictive coding and amortised inference. Each line is the average classification accuracy across three seeds; the shaded area corresponds to the standard deviation. The x-axis denotes the number of batches. (B) Generative loss. The panel shows the averaged mean-squared error between the lowest level of the hierarchy (which is fixed to the sensory data during testing) and the top-down predictions from the superordinate layer, plotted against batches, for HPC and standard PC. This metric provides a measure of how well each model is able to generate data. The seeds used are the same as those used in panel (A) (i.e., the data is from the same run). (C) Illustrative samples taken from HPC at the end of learning. These images are generated by activating a single nodes in the highest layer (corresponding to a single digit), and performing top-down predictions in a layer-wise fashion. The images correspond to the predicted nodes at the lowest layer. (D) As in (C) but for standard predictive coding.

https://doi.org/10.1371/journal.pcbi.1011280.g003

3.3 Fast inference

In the previous section, we demonstrated that hybrid predictive coding retains standard predictive coding’s classification accuracy and generative capabilities. We next show that the inclusion of a bottom-up, amortised component facilitates fast inference, by which we mean the ability to reach some level of perceptual certainty in a reduced number of iterations. To operationalise perceptual certainty, we introduce an arbitrary threshold (here 0.005) and stop iterative inference once the sum of average squared prediction errors (i.e., the free energy) has fallen below this threshold. The averaged prediction error can be thought of as a proxy for perceptual certainty because it is equivalent to variational free energy in the current context, thereby providing a principled measure of model fit since, after the minimization of the variational free energy is complete, it will come to approximate the log model evidence for a particular setting of the generative model parameters. Continuing with the same experimental setup, Fig 4B shows that the number of iterations required to reach perceptual certainty decreases over batches. Once converged, asymptotic accuracy is achieved without requiring any iterations at all, meaning that accurate perceptual beliefs are furnished through a single amortised forward sweep without the need for any expensive iterative inference steps, thus furnishing rapid and computationally cheap perception for commonly encountered data. Fig 4A demonstrates that this reduction in iterations has no detrimental effect on the accuracy of hybrid predictive coding.

[Figure omitted. See PDF.]

Fig 4. Fast inference.

(A) Classification accuracy of the hybrid predictive coding model and the bottom-up, amortised predictions as a function of number of batches. The asymptotic convergence demonstrates that placing an uncertainty-aware threshold on the number of iterations has no influence on (asymptotic) model performance. Plotted are average accuracies over 5 seeds and shaded regions are the standard deviation. (B) Average number of iterations (for iterative inference) as a function of test batch. Amortised predictions provide increasingly accurate estimates of model variables, reducing the need for costly iterative inference.

https://doi.org/10.1371/journal.pcbi.1011280.g004

To demonstrate the practical benefit of fast inference, we compare the accuracy of hybrid and standard predictive coding when using a fixed number of iterations (i.e., no ‘perceptual certainty’ threshold). Specifically, we compare classification accuracies when using 10, 25, 50 and 100 iterations for iterative inference. Results are shown in Fig 5, using the same simulation setup as in previous experiments, apart from the number of iterations. Hybrid predictive coding can obtain equivalent performance with a little as 10 iterations (Fig 5A), whereas standard predictive coding fails to learn at all under these conditions. At 25 iterations (Fig 5B), we see that the performance of standard predictive coding slowly decreases over batches. Notably, we observed no such general performance decrease for hybrid predictive coding suggesting that the amortized bottom up connection help to ‘stabilize’ learning in the hybrid network. Together, these results further illustrate that hybrid predictive coding facilitates fast inference by bypassing the need for costly iterative inference when amortized inferences are sufficiently accurate.

[Figure omitted. See PDF.]

Fig 5. Classification accuracy under fixed iterations.

(A) 10 iterations. The accuracy of HPC and the amortised predictions is mostly unaffected by the reduced number of iterations, whereas standard predictive coding fails to classify at all. (B) 25 iterations. The classification accuracy of standard predictive coding slowly decreases over batches, illustrating a common pathology observed in these simulations. (C) 50 iterations. Standard predictive coding approximately matches the performance of hybrid predictive coding, but begins to decline later in training. (D) 100 iterations. There are no significant differences between the accuracies of hybrid and standard predictive coding. Together, these results demonstrate that hybrid predictive coding enables effective inference and maintains higher performance with a substantially fewer amount of inference iterations required than standard predictive coding. Plotted are mean accuracies over 5 random network initializations. Shaded areas are the standard deviation.

https://doi.org/10.1371/journal.pcbi.1011280.g005

Another interesting phenomenon is the relative difference in accuracy between the full hybrid model and its amortised component as a function of iterations. When the number of iterations is lower (e.g., Fig 5A), the relative difference between these accuracies is far less pronounced since the accuracy of the pure iterative inference predictive coding network is unstable and decreases over time when there are an insufficient number of inference iterations. The amortized feedforward pass in the hybrid model, by providing an approximately ‘correct’ initialization, enables the network to furnish accurate beliefs within many fewer inference steps. These results suggest that, when the number of iterations is limited, amortised learning progresses at a faster rate and, in the limit, can enable progress even in situations where purely iterative learning fails.

We identify several unintuitive results that require further explanation, and which may be suitable predictions for empirical explorations. First, the accuracy of the amortised network improves slower as the number of inference iterations increases. We hypothesise that this is due to the targets for the amortised network being more difficult to predict (due to an increased number of iterations, leading to increased accuracy). Second, the amortised component can achieve high accuracy while the iterative component achieves low accuracy. This is due to the final layer loss—which is a function of the ground truth labels and which provides a learning signal in lieu of accurate iterative inference. Finally, we observe instabilities in the predictive coding network that occur at a low number of iterations, leading to a decline in accuracy. We hypothesize that these result from instabilities in the training procedure, which crucially are not found in the full hybrid network [90].

3.4 Learning with limited data

Having illustrated the benefits of incorporating a bottom-up, amortised component into predictive coding, we next consider the benefits of the top-down, iterative component of the model. With slight modifications [86], the amortised predictions of our model can approximate the backpropagation algorithm, meaning that the bottom-up connectivity implements something akin to a multi-layer perceptron. This might lead to the worry that the top-down component of hybrid predictive coding is superfluous, and a purely bottom-up scheme would suffice. There are several reasons why this is not the case. First, the top-down, generative component provides the training data from which the amortised component learns. Second, learning a generative model is generally more data-efficient compared to learning discriminative models. Here, we demonstrate the speed at which the network learns by plotting accuracy as a function of the amount of data the network has been exposed to. Specifically, we compare the accuracy of hybrid predictive coding compared to the amortised predictions using datasets with 100, 500, 1000 and 5000 examples (recall that the full dataset contains 60,000 examples). Note that we still use the complete test set of 10,000 images for testing. As Fig 6 shows, hybrid predictive coding retains good performance with as few as 100 training examples. By contrast, the speed at which the amortised predictions converge is significantly affected by the dataset size, such that the amortised predictions give poor accuracy in low data regimes. These results show that incorporating a top-down, generative component substantially increases the speed at which the network learns. It is also notable that performance of HPC is not negatively affected by the poor accuracy of the amortised predictions, again demonstrating an adaptive trade-off between amortised and iterative inference which allows for the iterative inference procedure to overcome a poor initialization by the amortized predictions.

[Figure omitted. See PDF.]

Fig 6. Accuracy as a function of dataset size.

(A) 100 examples. The accuracy of hybrid predictive coding is lower than with the full dataset, but still high given the relatively small amount of data the network has been exposed to (0.17 percent). The accuracy of the amortised predictions is significantly worse (B) 500 examples (C) 1000 examples. (D) 5000 examples. Together, these results demonstrate that bottom-up, amortised inference is far more sensitive to the time spent training, compared to the full hybrid architecture. Importantly, the poor performance of amortised inference at the start of learning does not negatively impact the speed at which iterative inference learns. Plotted are the mean accuracies over 5 seeds. Shaded areas represent the standard deviation.

https://doi.org/10.1371/journal.pcbi.1011280.g006

3.5 Additional properties of HPC

To gain further intuition for the functioning of the HPC model, we investigate several other properties of the model. Firstly, we investigate the degree to which the model’s own uncertainty evolves during the inference process. We quantify the model’s uncertainty as to the correct label by the entropy of its distribution over the predicted labels 0 − 9. This is achieved by normalizing the values for the final layer to obtain an output probability distribution and calculating the Shannon entropy of this distribution. In Fig 7A, we show that this entropy begins high and monotonically decreases through an inference iteration, thus suggesting that in general the iterative inference process serves to sharpen and clarify beliefs. Secondly, we investigated in more detail the computational savings the hybrid model achieves through its accurate initialization of the iterative inference via the amortized model. In Fig 7B, we plotted the number of inference iterations utilized for each batch during learning for the hybrid and the standard predictive coding model. We used an early stopping criterion by which iterations are terminated when an absolute error threshold β is reached. We see that for HPC the number of iterations required rapidly drops off during learning, due to the successful bootstrapping of the amortized model while for standard predictive coding the number of inference iterations only start declining towards the end of training when the network weights have adapted to become good at explaining the data. Since the iterative inference steps are the main computational cost of the model (the weight updates cost at most the same as a single inference iteration) HPC achieves a substantial computational saving over standard predictive coding while also obtaining equal or higher performance, as shown in previous figures.

[Figure omitted. See PDF.]

Fig 7. Additional properties of the HPC model.

(A) Example evolution of the label entropy over the course of an inference phase. The initial amortized guess has relatively high entropy (uncertainty over labels) which progressively reduces during iterative inference. This is consistent with the viewpoint that the iterative inference phase refines the initial amortized guesses. (B) The number of inference steps required over an example training run. Due to the superior initialization provided by the amortized connections, far fewer iterative inference steps are required. (C) Adaptive computation time based on task difficulty. On a well learned task, the number of inference iterations required decays towards 0. However, when there is a change in data distribution, additional iterative inference iterations are adaptively utilized to classify the new, more challenging, stimuli.

https://doi.org/10.1371/journal.pcbi.1011280.g007

Finally, the ability for amortised inference to ‘shortcut’ iterative inference is facilitated by the stationary data distribution used so far. Therefore, we investigated whether changes in the data distribution modulate the number of iterations to reach perceptual certainty. To do this, we split the dataset into two halves—one composed of labels 0 through 5, and the other composed of labels 5 through 9. Initially, we train and test using only the first of the two datasets. As shown in Fig 7C, the number of iterations (for iterative inference) decreases towards zero during this period. To enact a change in data distribution for the second half of training and testing, we utilise only the second half of the dataset. As Fig 7C shows, this dramatically increases the number of iterations required to reach perceptual certainty. This is because the bottom-up, amortised predictions have not learned the predict the relevant model variables, leading to a poor initialisation and an increase in prediction error. Crucially, this increase in iterations is automatically modulated by the data’s novelty (or formally, the log-likelihood of the data under the generative model), highlighting that HPC provides a principled mechanism for trading off speed and accuracy during perceptual processing. Crucially, the relationship between prediction errors and the final accuracy of the beliefs is non-trivial; it is possible for a randomly initialised PC network to achieve zero prediction error in a short number of iterations, and a fully converged PC network to incur a large amount of prediction error after several iterations, while still achieving accurate beliefs. This is demonstrated in experiment Fig 7C, where directly after the change in data distribution, the HPC model cannot classify the previously unseen digits but still manages to achieve perceptual certainty after several iterations. In this context, perceptual certainty refers to the equilibrium point of inference, which may or may not reflect accurate beliefs. As such, this plot tracks the number of iterations required to go from the initial state to the equilibrium state of the network. As demonstrated in Fig 7B, this is mostly facilitated by the role of the amortised network, which provides an initial state closer to the final equilibrium. However, it is partly facilitated by learning the parameters of the generative network (orange line), as learning changes the optimisation landscape that the iterative process operates on, enabling activity to converge in fewer iterations.

4 Discussion

The notion that the brain performs or approximates Bayesian inference has gained significant traction in recent years [7, 8, 68, 69, 91–93]. At the same time, the predictive coding architecture has gained prominence as a process theory which could provide a neurobiological implementation of approximate Bayesian inference [8, 31, 37, 42, 84]. However, there are many ways in which Bayesian inference and learning could be implemented or approximated in the brain. In this paper, we have described a novel architecture—hybrid predictive coding (HPC)—which combines amortised and iterative inference in a principled manner to achieve perceptual inference. In this biologically plausible architecture, predictions (and prediction errors) flow in both top-down and bottom-up directions. The top-down generative aspect of the model allows effective inference in low data regimes through relatively slow iterative inference. The bottom-up amortised (discriminative) aspect allows fast inference in stable data regimes. Hybrid predictive coding inherently balances the contributions of these two components in a data-driven ‘uncertainty aware’ fashion, so that the model inherits the benefits of both. As well as offering a novel machine learning architecture, hybrid predictive coding provides a powerful computational lens through which to understand different forms of visual perception—in particular, differences between fast context-free perception, such as gist perception, and slow, context-sensitive perception, such as detailed object recognition. Further work under the banner of ‘computational neurophenomenology’ (e.g., [94, 95]) is needed to examine this proposition in more detail, and in particular to explore how aspects of iterative/amortized inference correspond respectively to aspects of detailed/gist perceptual phenomenology.

To illustrate HPC, we have presented a number of simulations demonstrating that incorporating bottom-up, amortised predictions into a combined HPC architecture retains the benefits afforded by standard predictive coding (Fig 3), such as the ability to learn in both a supervised and unsupervised manner and work efficiently in low data regimes (Fig 6), while additionally enabling fast inference, a method for shortcutting the costly and time-consuming process of iteratively minimising prediction errors (Fig 4). Crucially, we have shown that the trade-off between fast, bottom-up, amortised inference and slow, top-down iterative inference is automatically modulated based on the model’s uncertainty about the data, enabling the model to utilise the benefits of both in an adaptive manner (Fig 7).

4.1 Properties of hybrid predictive coding

Previous work has described how amortised inference can be construed as learning to infer, as it seeks to learn a function which performs inference, rather than performing inference itself [96] directly. Building upon this insight, hybrid predictive coding can be considered an example of learning to infer through inference, as the learning signal for amortised inference is provided by the outcome of iterative inference. This mechanism provides a straightforward way to generate ‘correct’ layerwise targets internally for the amortized inference process to treat as supervised targets which enables the inference process to be bootstrapped rapidly while only using local learning rules in an unsupervised fashion.

Moreover, by providing a rapid initial ‘best guess’ at representations [9, 16, 97], amortised inference provides informed initial conditions for the subsequent phase of iterative inference, which decreases the number of iterations required to obtain accurate (or useful) beliefs. This dynamic enables rapid responses to novel events while also allowing for improved context-sensitive disambiguation of uncertain stimuli due to the refining effect of additional iterative inference steps. This means that, over time, in statistically stable (stationary) environments, bottom up predictions will learn to accurately capture perceptual beliefs, eschewing the need for costly iterative inference and therefore achieving substantial computational savings. However, if the environment changes, amortization will lead to errors which can be corrected by the iterative inference, allowing a rapid and context-sensitive response at the cost of greater computational expense during the period of adaptation to the novel environment.

Our model also naturally balances top-down and bottom-up contributions to perceptual inference. The degree to which bottom-up and top-down predictions contribute to the final beliefs is adaptively modulated based on their relative uncertainty, equipping the model with a basic notion of ‘confidence’, ‘uncertainty monitoring’, or ‘calibration’ [98]. By combining amortised and iterative inference within a single model, the resulting scheme naturally facilitates an adaptive trade-off between speed and accuracy, which is not possible when utilising either method of inference in isolation.

By building upon predictive coding, our model retains the generative capabilities of this algorithm [64, 99], facilitating broader generalisation and unsupervised learning. But the inclusion of bottom-up predictions also introduces a discriminative element to our model, as these bottom-up predictions map directly from data to beliefs. Because the model retains the neural architecture and Hebbian updates of predictive coding, it inherits the wealth of neurophysiological evidence that has accrued in favour of this theory [41]. Crucially, both the amortised and iterative inference schemes use the same learning rule, allowing us to posit a unified learning algorithm that underwrites both forms of inference. Moreover, by introducing a bottom-up process of amortised inference, our model provides a compelling account of feed-forward activity during perception and accords neatly with empirical results that demonstrate that core object recognition and “gist” perception occur on time scales which preclude the use of recurrent dynamics [48–50].

More generally, our model fits with the growing body of work which aims to describe and understand the relative roles of feed-forward and recurrent activity in the brain. Specifically, abundant evidence suggests that recurrent activity aids the processing of ambiguous, occluded and degraded objects, such that the degree of recurrent activity correlates with the ambiguity of the stimulus [100–105]. Our model can straightforwardly account for these results, as it proposes that recurrent activity implements a process of iterative inference, which refines the beliefs initialised by amortised inference until an accuracy threshold is reached, which would take more iterations for more ambiguous stimuli thus leading to longer response times and more effortful processing.

The model can also explain the effectiveness of the initial feed-forward sweep in core object recognition [19, 106], as simple types of object recognition can be accounted sufficiently for by amortised inference [107–110]. Our work builds upon recent machine learning architectures which have combined amortised and iterative inference in different ways, such as the semi-amortised variational autoencoders [111] and iterative amortised inference [46]. In those previous works, variational autoencoders [51] are augmented with a separate iterative inference phase, helping overcome the shortcomings of amortised inference [54]. Our model is distinguished from them by additionally developing a biologically plausible architecture closely matching neurophysiological observations and framed within the powerful context of predictive coding. Similarly, our model builds upon architectures that incorporate generative feedback into feed-forward neural networks [112]. These previous results demonstrate that generative feedback enables robustness to noise and adversarial attacks. However, these models are trained through backpropagation and are not formed in a biologically plausible manner. Finally, our work builds upon efforts to combine the relative benefits of generative and discriminative models [113].

4.2 The feedforward sweep and beyond

In visual neuroscience, object recognition is often separated into two distinct phases [18, 114]: an initial ‘feedforward’ sweep (lasting around 150ms) [14, 19, 114], in which sensory data is rapidly propagated up the visual hierarchy in a feedforward manner, and a subsequent stage of recurrent processing which persists over longer periods [16]. It has been argued that feedforward processing provides coarse-grained representations sufficient for core object recognition and so-called ‘gist’ perception, while recurrent processing finesses these representations by integrating contextual information [9, 16, 17, 115, 116] and allowing for the resolution of initial ambiguity or uncertainty.

This account of perception is remarkably consistent with our proposed model. In this context, the feedforward sweep corresponds to the amortised ‘best guess’ at perceptual beliefs, which is implemented by feedforward connectivity in our model. Crucially, these amortised predictions are insensitive to current context, as they map directly from data to beliefs. Moreover, amortised predictions suffer from the amortisation gap [54], which arises when parameters are shared across the whole dataset rather than optimized individually for each data point. Taken together, these considerations suggest that the beliefs furnished by amortised inference could lack the ability to successfully underlie perception for challenging (e.g., weak sensory signals), ambiguous, or otherwise unusual situations which require prior contextual knowledge to successfully parse—a result consistent with empirical evidence about the respective roles of feedforward and feedback processing in perception [18, 19, 117]. In line with this view, in our model we see a slower increase in accuracy for amortised inference, compared to the full hybrid predictive coding architecture, for small datasets. The implication here is that these small datasets cannot be modelled well with purely amortized inference, but can be modelled well by the combination of both iterative and amortized inference components. In addition, our model casts the recurrent processing in the visual system as a process of iterative inference, where beliefs are iteratively refined based on top-down predictions interacting with bottom-up beliefs and with sensory input. This iterative refinement integrates contextual information across multiple layers and slowly reduces the ambiguity in perceptual beliefs as the inference process converges to the best explanation. Again, this perspective is in accordance with neurobiologically-informed views suggesting that recurrent processing refines the representations generated during the feedforward sweep [17, 47, 105].

Our model makes several predictions which have been corroborated by empirical evidence. For instance, our model predicts that the amount of recurrent processing will correlate with the difficulty of perceptual processing tasks. In line with this, [116] reported human neuroimaging data suggesting increased recurrent processing for more challenging perceptual tasks. In addition, our model predicts that perceptual difficulty should modulate the relative influence of bottom-up and top-down processing, as has been observed in experimental data [118].

While there have been several proposals for how feedforward and recurrent activity may be integrated in the brain, our model is the first to combine these into a common and biologically plausible predictive-coding architecture. Doing so provides a principled arbitration between speed and accuracy in perceptual processing [119]. In our model, recurrent dynamics are driven by prediction errors. When prediction errors are minimised (i.e., predictions are accurate), recurrent activity is suppressed. This means that when amortised predictions generates accurate beliefs, there will be no prediction errors and no recurrent activity. Alternatively, when amortised predictions generate inaccurate beliefs, prediction errors will be large and iterations of recurrent activity are engaged to finesse beliefs. Crucially, this arbitration arises naturally from the representations within the model. In summary, our model provides a plausible account for both feedforward and recurrent activity in the brain, which can be related to distinct forms of visual perception.

Empirical studies that explore the interaction between feedforward and recurrent activity typically utilize a stimulus-response paradigm, in which a single, static stimulus is introduced, and brain responses are recorded. In such paradigms, distinct feedforward and recurrent activity signatures can be observed [120]. The present study follows this approach by conducting inference on a single, static image and simulating a feedforward pass, followed by recurrent iterations. However, the brain is exposed to continuous dynamic stimuli in natural environments, leading to a less distinct separation between feedforward and recurrent activity. In future research, we aim to expand this model to dynamic stimuli and investigate whether the hybrid predictive coding scheme can be replicated without artificially separating the two stages, such as through interleaved recurrent updates at varying temporal resolutions.

One potential criticism of the hybrid predictive coding model is that it predicts recurrent feedback activity going to zero in static environments. However, the human brain never encounters fully static data—thanks to environmental fluctuations, internal movements such as saccades—or even just ongoing internal neural fluctuations (and there is some circumstantial evidence from phenomena like retinal stabilization [121, 122] that brain activity and perception do go to zero for fixed stimuli). This means that amortised, feedforward predictions should never fully explain the data, so there is always a continual flow of prediction error signals that recurrent activity must explain away.

Interestingly, some prominent theories of consciousness emphasize recurrent (i.e., top-down) signalling as critical for conscious perception ([123, 124], for a review see [125]). While these theories are not always interpreted in terms of perceptual inference, it is natural to do so [126]. This however raises an interesting possibility: could it be that conscious perception depends not on top-down signalling defined anatomically (as in standard recurrent theories) but, instead, defined functionally, in terms of their role in conveying predictions (as opposed to prediction errors)? This would predict that the relationship between conscious status and (anatomical) top-down signalling would depend on the details of the perceptual task and/or aspect of phenomenology being assayed. Future work could usefully investigate this possibility.

4.3 Predictive coding

Predictive coding has been shown to explain a diverse range of perceptual phenomena, such as end stopping [6], bistable perception [82, 127], repetition suppression [128] and illusory motion [129] (see [41] for more). Moreover, recent work has demonstrated that predictive coding provides a local approximation to backpropagation; the algorithm underwriting many of the recent successes in machine learning [84, 86, 130]. As such, it presents one of the leading theories for perception and learning in the brain [131] and by building on the predictive coding framework, our model inherits the wealth of empirical evidence that has been gathered in its favour [41, 42, 132, 133].

While predictive coding has emerged as a promising candidate for understanding cortical function, its iterative nature fits poorly with some established facts about visual perception. Prominent among these is that the visual system can reliably extract a range of features within 150ms of stimulus onset [14, 48–50], a timescale which would seem to preclude the presence of multiple iterations of recurrent dynamics, and in turn, the use of iterative inference. In short, predictive coding struggles to account for rapid “gist” perception [134, 135], an essential component of visual perception. To overcome this shortcoming, our model augments predictive coding with additional bottom-up connectivity, which provides amortised estimates of perceptual beliefs using a single forward pass. The feedforward nature of the amortised connections means that representations can be extracted rapidly without relying on recurrent activity [106]. Although predictions are generally associated with top-down recurrent processing, this bottom-up forward pass can also be interpreted as its own kind of prediction [23]—predicting beliefs directly from data—with its own set of prediction errors that are minimized during learning. This perspective lets us see our model as simply performing bidirectional prediction and prediction error minimization on a unified objective. It is intriguing to consider, from the perspective of “computational phenomenology”, whether the distinct phenomenological character of gist perception (in which an overall context is experienced), compared to detailed focal perception (in which fine details of, for example, visual objects) can be understood in terms of these differing forms of perceptual prediction.

4.4 Generative and discriminative models

In machine learning, a common distinction is made between generative and discriminative methods [136]. Generative methods learn a joint distribution over sensory data and hidden causes p(x, z), whereas discriminative methods learn a conditional mapping from data to hidden causes p(z|x). It is well established that generative methods are more efficient in low data regimes [137], can be used for a wider range of downstream tasks [138], and enable better generalisation [139]. On the other hand, discriminative methods are more efficient when the goal is to predict hidden states, and generally reach higher asymptotic performance [140]. This is because discriminative methods only learn about features relevant for discrimination, whereas generative methods learn about the data distribution itself. In general, generative methods enable unsupervised learning, where hidden states are not known in advance, whereas discriminative methods utilize supervised learning with a known set of ultimate hidden states—the labels.

Our model combines generative and discriminative components within a single architecture. The top-down connectivity implements a generative model, whereas the bottom-up connectivity implements a discriminative classifier (where the labels are now the hidden states of the generative model). The model thus retains the benefits of generative approaches, while also incorporating the benefits of discriminative learning. In contrast to previous proposals which combine generative and discriminative learning [112, 113, 141–143], our model operates within the biologically plausible scheme of predictive coding and automatically arbitrates the relative influence based on the uncertainty of bottom-up and top-down predictions.

An additional benefit of combining generative and discriminative methods is that it enables generative replay [144–146]. This describes the process of generating fake data (using some generative model) which is then used for downstream tasks. For instance, the generated data can be used to overcome ‘catastrophic forgetting’ [147] and enable continual learning [146]. In the context of our work, the generative model can be used to produce data from which the amortised component can learn. This has the benefit of reducing the amount of real-world data required for accurate inference. Another exciting possibility, arising directly from the HPC architecture, is using the discriminative model to generate hidden states which can then be used to train the generative model. These opportunities are afforded by the bi-directional modelling at the heart of our architecture and have been explored extensively in the reinforcement learning literature where the idea of using a learnt model to train an amortized policy or vice versa is common [45, 59, 148]. Finally, the fact that the generative and discriminative connections are implemented in a layer-wise fashion means that replay can operate on a layer-by-layer basis. In brief, the generative model can help train the discriminative model which can help train the generative model. One hypothesis is that this process happens during sleep, when the brain is detached from veridical data [149]. An open question remains as to whether hybrid predictive coding learns different representations, relative to standard predictive coding, or whether the observed effects just influence the manner in which the two networks learn. We hope to explore this question in future work.

4.5 Conclusion

We have proposed a novel predictive coding architecture—hybrid predictive coding—which combines iterative and amortized inference techniques to jointly optimize a single unified energy function using only local Hebbian updates. We demonstrate that this architecture enables both rapid and computationally cheap inference when in stable environments where a good model can be learned, while also flexibly providing more accurate inference in sparse data or early learning regimes. The results of our model additionally suggest that hybrid predictive coding will behave favorably when faced with more difficult stimuli, due to it being able to flexibly deploy additional computation as a function of experience with a given stimulus. Our hybrid model also can learn rapidly from small datasets, and is inherently able to detect its own uncertainty and adaptively respond to changing environments. Hybrid predictive coding thus provides a novel model that unifies disparate perspectives on the biological role of feedforward sweeps and recurrent activity in the visual cortex, offering a parsimonious explanation for several different and often opposing perspectives, while reproducing several well-known experimental effects in this field and potentially even accounting for distinct aspects of visual phenomenology.

Acknowledgments

AT was funded by a PhD studentship from the Dr. Mortimer and Theresa Sackler Foundation and the School of Engineering and Informatics at the University of Sussex. BM was supported by an EPSRC PhD studentship. AKS is supported by ERC Advanced Investigator Grant CONSCIOUS, grant number 101012954, and by the Dr Mortimer and Theresa Sackler Foundation. CLB is supported by BBRSC grant number BB/P022197/1 and by Joint Research with the National Institutes of Natural Sciences (NINS), Japan, program No. 01112005. AKS is additionally grateful to the Canadian Institute for Advanced Research (CIFAR, Program on Brain, Mind, and Consciousness). AT would also like to recognize the support of the Verses Research Lab. BM was additionally supported by Rafal Bogacz with BBSRC grant BB/s006338/1 and MRC grant MC_UU_00003/1.

Citation: Tscshantz A, Millidge B, Seth AK, Buckley CL (2023) Hybrid predictive coding: Inferring, fast and slow. PLoS Comput Biol 19(8): e1011280. https://doi.org/10.1371/journal.pcbi.1011280

References

1. Van Essen DC, Maunsell JH. Hierarchical organization and functional streams in the visual cortex. Trends in neurosciences. 1983;6:370–375.

2. DiCarlo JJ, Zoccolan D, Rust NC. How does the brain solve visual object recognition? Neuron. 2012;73(3):415–434. pmid:22325196

3. Marr D. Vision: A computational investigation into the human representation and processing of visual information. CUMINCAD. 1982;.

4. Dayan P, Hinton GE, Neal RM, Zemel RS. The Helmholtz machine. Neural computation. 1995;7(5):889–904. pmid:7584891

5. Lee TS, Mumford D. Hierarchical Bayesian inference in the visual cortex. JOSA A. 2003;20(7):1434–1448. pmid:12868647

6. Rao RP, Ballard DH. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature neuroscience. 1999;2(1):79–87. pmid:10195184

7. Knill DC, Pouget A. The Bayesian brain: the role of uncertainty in neural coding and computation. TRENDS in Neurosciences. 2004;27(12):712–719. pmid:15541511

8. Friston K. A theory of cortical responses. Philosophical transactions of the Royal Society B: Biological sciences. 2005;360(1456):815–836. pmid:15937014

9. Kveraga K, Ghuman AS, Bar M. Top-down predictions in the cognitive brain. Brain and cognition. 2007;65(2):145–168. pmid:17923222

10. Theeuwes J. Top–down and bottom–up control of visual selection. Acta psychologica. 2010;135(2):77–99. pmid:20507828

11. Weidner R, Krummenacher J, Reimann B, Müller HJ, Fink GR. Sources of top–down control in visual search. Journal of Cognitive Neuroscience. 2009;21(11):2100–2113. pmid:19199412

12. Melloni L, van Leeuwen S, Alink A, Müller NG. Interaction between bottom-up saliency and top-down control: how saliency maps are created in the human brain. Cerebral cortex. 2012;22(12):2943–2952. pmid:22250291

13. Lamme VA. How neuroscience will change our view on consciousness. Cognitive neuroscience. 2010;1(3):204–220. pmid:24168336

14. Thorpe S, Fize D, Marlot C. Speed of processing in the human visual system. nature. 1996;381(6582):520–522. pmid:8632824

15. Delorme A, Rousselet GA, Macé MJM, Fabre-Thorpe M. Interaction of top-down and bottom-up processing in the fast visual analysis of natural scenes. Cognitive Brain Research. 2004;19(2):103–113. pmid:15019707

16. Kreiman G, Serre T. Beyond the feedforward sweep: feedback computations in the visual cortex. Annals of the New York Academy of Sciences. 2020;. pmid:32112444

17. Ahissar M, Hochstein S. The reverse hierarchy theory of visual perceptual learning. Trends in cognitive sciences. 2004;8(10):457–464. pmid:15450510

18. Lamme VA, Roelfsema PR. The distinct modes of vision offered by feedforward and recurrent processing. Trends in neurosciences. 2000;23(11):571–579. pmid:11074267

19. VanRullen R. The power of the feed-forward sweep. Advances in Cognitive Psychology. 2007;3(1-2):167.

20. Roland PE. Six principles of visual cortical dynamics. Frontiers in systems neuroscience. 2010;4:28. pmid:20661451

21. Rauss K, Pourtois G. What is bottom-up and what is top-down in predictive coding? Frontiers in psychology. 2013;4:276. pmid:23730295

22. Awh E, Belopolsky AV, Theeuwes J. Top-down versus bottom-up attentional control: A failed theoretical dichotomy. Trends in cognitive sciences. 2012;16(8):437–443. pmid:22795563

23. Teufel C, Fletcher PC. Forms of prediction in the nervous system. Nature Reviews Neuroscience. 2020;21(4):231–242. pmid:32157237

24. Den Ouden HE, Kok P, De Lange FP. How prediction errors shape perception, attention, and motivation. Frontiers in psychology. 2012;3:548. pmid:23248610

25. Alink A, Schwiedrzik CM, Kohler A, Singer W, Muckli L. Stimulus predictability reduces responses in primary visual cortex. Journal of Neuroscience. 2010;30(8):2960–2966. pmid:20181593

26. Gordon N, Koenig-Robert R, Tsuchiya N, Van Boxtel JJ, Hohwy J. Neural markers of predictive coding under perceptual uncertainty revealed with Hierarchical Frequency Tagging. Elife. 2017;6:e22749. pmid:28244874

27. Murray SO, Kersten D, Olshausen BA, Schrater P, Woods DL. Shape perception reduces activity in human primary visual cortex. Proceedings of the National Academy of Sciences. 2002;99(23):15164–15169. pmid:12417754

28. Summerfield C, De Lange FP. Expectation in perceptual decision making: neural and computational mechanisms. Nature Reviews Neuroscience. 2014;15(11):745–756. pmid:25315388

29. Bogacz R. A tutorial on the free-energy framework for modelling perception and learning. Journal of mathematical psychology. 2017;76:198–211. pmid:28298703

30. Buckley CL, Kim CS, McGregor S, Seth AK. The free energy principle for action and perception: A mathematical review. Journal of Mathematical Psychology. 2017;81:55–79.

31. Millidge B, Seth AK, Buckley CL. Predictive coding: a theoretical and experimental review. arXiv preprint arXiv:210712979. 2021;.

32. Millidge B, Salvatori T, Song Y, Bogacz R, Lukasiewicz T. Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation? arXiv preprint arXiv:220209467. 2022;.

33. Friston K, Kiebel S. Predictive coding under the free-energy principle. Philosophical Transactions of the Royal Society B: Biological Sciences. 2009;364(1521):1211–1221. pmid:19528002

34. Clark A. Surfing uncertainty: Prediction, action, and the embodied mind. Oxford University Press; 2015.

35. Millidge B. Implementing predictive processing and active inference: Preliminary steps and results. PsyArXiv. 2019;.

36. Spratling MW. A review of predictive coding algorithms. Brain and cognition. 2017;112:92–97. pmid:26809759

37. Friston K. Hierarchical models in the brain. PLoS Comput Biol. 2008;4(11):e1000211. pmid:18989391

38. Fox CW, Roberts SJ. A tutorial on variational Bayesian inference. Artificial intelligence review. 2012;38(2):85–95.

39. Beal MJ. Variational algorithms for approximate Bayesian inference; 2003.

40. Hinton GE, van Camp D. Keeping the Neural Networks Simple by Minimizing the Description Length of the Weights. In: Proceedings of the Sixth Annual Conference on Computational Learning Theory. COLT’93. New York, NY, USA: ACM; 1993. p. 5–13. Available from: http://doi.acm.org/10.1145/168304.168306.

41. Walsh KS, McGovern DP, Clark A, O’Connell RG. Evaluating the neurophysiological evidence for predictive processing as a model of perception. Annals of the new York Academy of Sciences. 2020;1464(1):242. pmid:32147856

42. Bastos AM, Usrey WM, Adams RA, Mangun GR, Fries P, Friston K. Canonical microcircuits for predictive coding. Neuron. 2012;76(4):695–711. pmid:23177956

43. Millidge B, Tschantz A, Seth AK, Buckley C. Neural Kalman filtering. arXiv preprint arXiv:210210021. 2021;.

44. Millidge B, Tschantz A, Seth AK, Buckley CL. Reinforcement Learning as Iterative and Amortised Inference. arXiv preprint arXiv:200610524. 2020;.

45. Tschantz A, Millidge B, Seth AK, Buckley CL. Control as hybrid inference. arXiv preprint arXiv:200705838. 2020;.

46. Marino J, Yue Y, Mandt S. Iterative amortized inference. In: International Conference on Machine Learning. PMLR; 2018. p. 3403–3412.

47. van Bergen RS, Kriegeskorte N. Going in circles is the way forward: the role of recurrence in visual inference. arXiv preprint arXiv:200312128. 2020;.

48. Keysers C, Xiao DK, Földiák P, Perrett DI. The speed of sight. Journal of cognitive neuroscience. 2001;13(1):90–101. pmid:11224911

49. Carlson T, Tovar DA, Alink A, Kriegeskorte N. Representational dynamics of object vision: the first 1000 ms. Journal of vision. 2013;13(10):1–1. pmid:23908380

50. Thunell E, Thorpe SJ. Memory for repeated images in rapid-serial-visual-presentation streams of thousands of images. Psychological science. 2019;30(7):989–1000. pmid:31017834

51. Kingma DP, Welling M. Auto-encoding variational bayes. arXiv preprint arXiv:13126114. 2013;.

52. Doersch C. Tutorial on variational autoencoders. arXiv preprint arXiv:160605908. 2016;.

53. Gershman S, Goodman N. Amortized inference in probabilistic reasoning. In: Proceedings of the annual meeting of the cognitive science society. vol. 36; 2014.

54. Cremer C, Li X, Duvenaud D. Inference suboptimality in variational autoencoders. In: International Conference on Machine Learning. PMLR; 2018. p. 1078–1086.

55. Kahneman D. Thinking, fast and slow. macmillan; 2011.

56. Ghahramani Z, Beal MJ. Propagation algorithms for variational Bayesian learning. In: Advances in neural information processing systems; 2001. p. 507–513.

57. Marino J, Piché A, Ialongo AD, Yue Y. Iterative amortized policy optimization. Advances in Neural Information Processing Systems. 2021;34.

58. Millidge B, Tschantz A, Seth AK, Buckley CL. On the relationship between active inference and control as inference. In: International Workshop on Active Inference. Springer; 2020. p. 3–11.

59. Sutton RS. Dyna, an integrated architecture for learning, planning, and reacting. ACM Sigart Bulletin. 1991;2(4):160–163.

60. Schmidhuber J. Making the World Differentiable: On Using Self-Supervised Fully Recurrent Neural Networks for Dynamic Reinforcement Learning and Planning in Non-Stationary Environments. Technical Report: Technical University of Munich. 1990;.

61. Bengio Y, Scellier B, Bilaniuk O, Sacramento J, Senn W. Feedforward initialization for fast inference of deep generative networks is biologically plausible. arXiv preprint arXiv:160601651. 2016;.

62. Xie X, Seung HS. Equivalence of backpropagation and contrastive Hebbian learning in a layered network. Neural computation. 2003;15(2):441–454. pmid:12590814

63. Scellier B, Bengio Y. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in computational neuroscience. 2017;11:24. pmid:28522969

64. Rao RP, Ballard DH. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience. 1999;2(1):79–87. pmid:10195184

65. Hohwy J, Seth AK. Predictive processing as a systematic basis for identifying the neural correlates of consciousness. Philosophy and the Mind Sciences. 2020;1(II).

66. Seth AK, Hohwy J. Predictive processing as an empirical theory for consciousness science. Cognitive Neuroscience. 2021;12(2):89–90. pmid:33280521

67. Xu D, Clappison A, Seth C, Orchard J. Symmetric predictive estimator for biologically plausible neural learning. IEEE Transactions on Neural Networks and Learning Systems. 2017;29(9):4140–4151. pmid:29990028

68. Friston K. The history of the future of the Bayesian brain. NeuroImage. 2012;62(2):1230–1233. pmid:22023743

69. Knill DC, Richards W. Perception as Bayesian inference. Cambridge University Press; 1996.

70. Cox RT. Probability, frequency and reasonable expectation. American journal of physics. 1946;14(1):1–13.

71. Jaynes ET. Probability theory: The logic of science. Cambridge university press; 2003.

72. Wainwright MJ, Jordan MI, et al. Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning. 2008;1(1–2):1–305.

73. Friston K, Kilner J, Harrison L. A free energy principle for the brain. Journal of Physiology-Paris. 2006;100(1-3):70–87. pmid:17097864

74. Friston K, Stephan KE. Free-energy and the brain. Synthese. 2007;159(3):417–458. pmid:19325932

75. Neal RM, Hinton GE. A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Learning in graphical models. Springer; 1998. p. 355–368.

76. Friston K, FitzGerald T, Rigoli F, Schwartenbeck P, O’Doherty J, Pezzulo G. Active inference and learning. Neuroscience and Biobehavioral Reviews. 2016;68:862–879. pmid:27375276

77. Odaibo S. Tutorial: Deriving the standard variational autoencoder (vae) loss function. arXiv preprint arXiv:190708956. 2019;.

78. Hoffman MD, Blei DM, Wang C, Paisley J. Stochastic variational inference. The Journal of Machine Learning Research. 2013;14(1):1303–1347.

79. Ranganath R, Gerrish S, Blei D. Black box variational inference. In: Artificial intelligence and statistics. PMLR; 2014. p. 814–822.

80. Clark A. Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and brain sciences. 2013;36(3):181–204. pmid:23663408

81. Hohwy J. The predictive mind. Oxford University Press; 2013.

82. Weilnhammer V, Stuke H, Hesselmann G, Sterzer P, Schmack K. A predictive coding account of bistable perception-a model-based FMRI study. PLoS computational biology. 2017;13(5):e1005536. pmid:28505152

83. Millidge B, Tschantz A, Seth AK, Buckley CL. Relaxing the constraints on predictive coding models. arXiv preprint arXiv:201001047. 2020;.

84. Whittington JC, Bogacz R. An approximation of the error backpropagation algorithm in a predictive coding network with local Hebbian synaptic plasticity. Neural computation. 2017;29(5):1229–1262. pmid:28333583

85. Sun W, Orchard J. A Predictive-Coding Network That Is Both Discriminative and Generative. Neural Computation. 2020;32(10):1836–1862. pmid:32795234

86. Millidge B, Tschantz A, Buckley CL. Predictive Coding Approximates Backprop along Arbitrary Computation Graphs. arXiv preprint arXiv:200604182. 2020;.

87. Zhang C, Bütepage J, Kjellström H, Mandt S. Advances in variational inference. IEEE transactions on pattern analysis and machine intelligence. 2018;41(8):2008–2026.

88. Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep learning. vol. 1. MIT press Cambridge; 2016.

89. Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980. 2014;.

90. Kinghorn PF, Millidge B, Buckley CL. Preventing Deterioration of Classification Accuracy in Predictive Coding Networks. arXiv preprint arXiv:220807114. 2022;.

91. Doya K, Ishii S, Pouget A, Rao RP. Bayesian brain: Probabilistic approaches to neural coding. MIT press; 2007.

92. Seth AK. The cybernetic Bayesian brain. Open MIND. Frankfurt am Main: MIND Group; 2014.

93. Wolpert DM, Ghahramani Z. Bayes rule in perception, action and cognition. The Oxford Companion to the Mind Oxford University Press (http://eprints.pascal-network.org/archive/00001354/). 2005;.

94. Ramstead MJ, Seth AK, Hesp C, Sandved-Smith L, Mago J, Lifshitz M, et al. From generative models to generative passages: a computational approach to (neuro) phenomenology. Review of Philosophy and Psychology. 2022; p. 1–29. pmid:35317021

95. Suzuki K, Roseboom W, Schwartzman DJ, Seth AK. A deep-dream virtual reality platform for studying altered perceptual phenomenology. Scientific reports. 2017;7(1):1–11. pmid:29167538

96. Dasgupta I, Schulz E, Tenenbaum JB, Gershman SJ. A theory of learning to infer. Psychological Review. 2020;127(3):412. pmid:32223286

97. Bar M. A cortical mechanism for triggering top-down facilitation in visual object recognition. Journal of cognitive neuroscience. 2003;15(4):600–609. pmid:12803970

98. Guo C, Pleiss G, Sun Y, Weinberger KQ. On calibration of modern neural networks. In: International Conference on Machine Learning. PMLR; 2017. p. 1321–1330.

99. Spratling MW. Unsupervised learning of generative and discriminative weights encoding elementary image components in a predictive coding model of cortical function. Neural computation. 2012;24(1):60–103. pmid:22023197

100. Johnson JS, Olshausen BA. The recognition of partially visible natural objects in the presence and absence of their occluders. Vision research. 2005;45(25-26):3262–3276. pmid:16043208

101. Wyatte D, Curran T, O’Reilly R. The limits of feedforward vision: Recurrent processing promotes robust object recognition when objects are degraded. Journal of Cognitive Neuroscience. 2012;24(11):2248–2261. pmid:22905822

102. Rajaei K, Mohsenzadeh Y, Ebrahimpour R, Khaligh-Razavi SM. Beyond core object recognition: Recurrent processes account for object recognition under occlusion. PLoS computational biology. 2019;15(5):e1007001. pmid:31091234

103. Tang H, Schrimpf M, Lotter W, Moerman C, Paredes A, Caro JO, et al. Recurrent computations for visual pattern completion. Proceedings of the National Academy of Sciences. 2018;115(35):8835–8840. pmid:30104363

104. Kar K, Kubilius J, Schmidt K, Issa EB, DiCarlo JJ. Evidence that recurrent circuits are critical to the ventral stream’s execution of core object recognition behavior. Nature neuroscience. 2019;22(6):974–983. pmid:31036945

105. Grossberg S. Adaptive Resonance Theory: How a brain learns to consciously attend, learn, and recognize a changing world. Neural networks. 2013;37:1–47. pmid:23149242

106. Serre T, Oliva A, Poggio T. A feedforward architecture accounts for rapid categorization. Proceedings of the national academy of sciences. 2007;104(15):6424–6429. pmid:17404214

107. Brincat SL, Connor CE. Dynamic shape synthesis in posterior inferotemporal cortex. Neuron. 2006;49(1):17–24. pmid:16387636

108. Rousselet GA, Macé MJM, Fabre-Thorpe M. Is it an animal? Is it a human face? Fast processing in upright and inverted natural scenes. Journal of vision. 2003;3(6):5–5. pmid:12901715

109. Ringach DL, Shapley R. Spatial and temporal properties of illusory contours and amodal boundary completion. Vision research. 1996;36(19):3037–3050. pmid:8917767

110. Sugase Y, Yamane S, Ueno S, Kawano K. Global and fine information coded by single neurons in the temporal visual cortex. Nature. 1999;400(6747):869–873. pmid:10476965

111. Kim Y, Wiseman S, Miller A, Sontag D, Rush A. Semi-amortized variational autoencoders. In: International Conference on Machine Learning. PMLR; 2018. p. 2678–2687.

112. Huang Y, Gornet J, Dai S, Yu Z, Nguyen T, Tsao D, et al. Neural Networks with Recurrent Generative Feedback. Advances in Neural Information Processing Systems. 2020;33.

113. Gordon J, Hernández-Lobato JM. Combining deep generative and discriminative models for Bayesian semi-supervised learning. Pattern Recognition. 2020;100:107156.

114. Grootswagers T, Robinson AK, Carlson TA. The representational dynamics of visual objects in rapid serial visual processing streams. NeuroImage. 2019;188:668–679. pmid:30593903

115. Yoo SA, Tsotsos JK, Fallah M. Feed-forward visual processing suffices for coarse localization but fine-grained localization in an attention-demanding context needs feedback processing. Plos one. 2019;14(9):e0223166. pmid:31557228

116. Mohsenzadeh Y, Qin S, Cichy RM, Pantazis D. Ultra-Rapid serial visual presentation reveals dynamics of feedforward and feedback processes in the ventral visual pathway. Elife. 2018;7:e36329. pmid:29927384

117. Furtak M, Mudrik L, Bola M. The forest, the trees, or both? Hierarchy and interactions between gist and object processing during perception of real-world scenes. PsyArXiv. 2021;.

118. Karimi-Rouzbahani H, Ramezani F, Woolgar A, Rich AN, Ghodrati M. Perceptual difficulty modulates the direction of information flow in familiar face recognition. bioRxiv. 2020;.

119. Spoerer CJ, Kietzmann TC, Mehrer J, Charest I, Kriegeskorte N. Recurrent neural networks can explain flexible trading of speed and accuracy in biological vision. PLoS computational biology. 2020;16(10):e1008215. pmid:33006992

120. Bastos AM, Vezoli J, Bosman CA, Schoffelen JM, Oostenveld R, Dowdall JR, et al. Visual areas exert feedforward and feedback influences through distinct frequency channels. Neuron. 2015;85(2):390–401. pmid:25556836

121. Riggs LA, Ratliff F, Cornsweet JC, Cornsweet TN. The disappearance of steadily fixated visual test objects. JOSA. 1953;43(6):495–501. pmid:13070111

122. Millidge B. Fixational Eye Movements: Data Augmentation for the Brain? PsyArXiv. 2019;.

123. Lamme VA, Supèr H, Landman R, Roelfsema PR, Spekreijse H. The role of primary visual cortex (V1) in visual awareness. Vision research. 2000;40(10-12):1507–1521. pmid:10788655

124. Pascual-Leone A, Walsh V. Fast backprojections from the motion to the primary visual area necessary for visual awareness. Science. 2001;292(5516):510–512. pmid:11313497

125. Seth AK, Bayne T. Theories of consciousness. Nature Reviews Neuroscience. 2022;23(7):439–452. pmid:35505255

126. Seth AK. Being you: A new science of consciousness. Penguin; 2021.

127. Hohwy J, Roepstorff A, Friston K. Predictive coding explains binocular rivalry: An epistemological review. Cognition. 2008;108(3):687–701. pmid:18649876

128. Auksztulewicz R, Friston K. Repetition suppression and its contextual determinants in predictive coding. cortex. 2016;80:125–140. pmid:26861557

129. Lotter W, Kreiman G, Cox D. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv:160508104. 2016;.

130. Song Y, Lukasiewicz T, Xu Z, Bogacz R. Can the Brain Do Backpropagation?—Exact Implementation of Backpropagation in Predictive Coding Networks. Advances in Neural Information Processing Systems. 2020;33.

131. Whittington JC, Bogacz R. Theories of error back-propagation in the brain. Trends in cognitive sciences. 2019;23(3):235–250. pmid:30704969

132. Spratling M. Fitting predictive coding to the neurophysiological data. Brain research. 2019;1720:146313. pmid:31265817

133. Keller GB, Mrsic-Flogel TD. Predictive processing: a canonical cortical computation. Neuron. 2018;100(2):424–435. pmid:30359606

134. Oliva A. Gist of the scene. In: Neurobiology of attention. Elsevier; 2005. p. 251–256.

135. Oliva A, Torralba A. Building the gist of a scene: The role of global image features in recognition. Progress in brain research. 2006;155:23–36. pmid:17027377

136. Bishop CM. Pattern recognition and machine learning. springer; 2006.

137. Chua K, Calandra R, McAllister R, Levine S. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. arXiv preprint arXiv:180512114. 2018;.

138. Kingma DP, Rezende DJ, Mohamed S, Welling M. Semi-supervised learning with deep generative models. arXiv preprint arXiv:14065298. 2014;.

139. Rezende DJ, Mohamed S, Wierstra D. Stochastic backpropagation and approximate inference in deep generative models. In: International conference on machine learning. PMLR; 2014. p. 1278–1286.

140. LeCun Y, Bengio Y, Hinton G. Deep learning. nature. 2015;521(7553):436–444. pmid:26017442

141. Kuleshov V, Ermon S. Deep hybrid models: Bridging discriminative and generative approaches. In: Proceedings of the Conference on Uncertainty in AI (UAI); 2017.

142. Liu H, Abbeel P. Hybrid discriminative-generative training via contrastive learning. arXiv preprint arXiv:200709070. 2020;.

143. Garcia Satorras V, Akata Z, Welling M. Combining Generative and Discriminative Models for Hybrid Inference. Advances in Neural Information Processing Systems. 2019;32:13825–13835.

144. Shin H, Lee JK, Kim J, Kim J. Continual learning with deep generative replay. arXiv preprint arXiv:170508690. 2017;.

145. Van de Ven GM, Tolias AS. Generative replay with feedback connections as a general strategy for continual learning. arXiv preprint arXiv:180910635. 2018;.

146. van de Ven GM, Siegelmann HT, Tolias AS. Brain-inspired replay for continual learning with artificial neural networks. Nature communications. 2020;11(1):1–14. pmid:32792531

147. Kirkpatrick J, Pascanu R, Rabinowitz N, Veness J, Desjardins G, Rusu AA, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences. 2017;114(13):3521–3526. pmid:28292907

148. Schmidhuber J. Making the World Differentiable: On Using Self-Supervised Fully Recurrent Neural Networks for Dynamic Reinforcement Learning and Planning in Non-Stationary Environments; 1990.

149. Friston K, Lin M, Frith CD, Pezzulo G, Hobson JA, Ondobaka S. Active inference, curiosity and insight. Neural computation. 2017;29(10):2633–2683. pmid:28777724