User:Kfriston/sandbox

Generalized filtering is a generic Bayesian filtering scheme for nonlinear state-space models. It is based on a variational principle of least action, formulated in generalized coordinates of motion. Generalized filtering furnishes posterior densities over hidden states (and parameters) generating observed data using a generalized gradient decent on variational free energy, under the Laplace assumption. Unlike classical (e.g., Kalman-Bucy or particle) filtering, generalized filtering eschews Markovian assumptions about random fluctuations. Furthermore, it operates online, assimilating data to approximate the posterior density over unknown quantities, without the need for a backward pass. Special cases include variational filtering, dynamic expectation maximisation and generalised predictive coding.

Definition
Definition: Generalized filtering rests on the tuple $$(\Omega,X,S,U,p,q)$$:
 * A sample space $$\Omega$$ from which random fluctuations $$\omega \in \Omega$$  are drawn
 * Hidden states $$X:X \times U \times \Omega \to \mathbb{R}$$ – that cause sensor states and depend on control states
 * Sensor states $$S:X \times U \times \Omega \to \mathbb{R}$$ – a probabilistic mapping from hidden and control states
 * Control states $$U \in \mathbb{R}$$ – that act as external causes, input or forcing terms
 * Generative density $$p(\tilde{s},\tilde{x},\tilde{u}|m)$$  – over sensory, hidden and control states under a generative model
 * Variational density $$q(\tilde{x},\tilde{u}|m)$$ – over hidden and control states with mean $$\tilde{\mu} \in \mathbb{R} $$

Here ~ denotes a variable in generalised coordinates of motion: $$\tilde{u} = [u,u',u'',...]^T$$

Generalised filtering
The objective is to approximate the posterior density over hidden and control states, given sensor states and a generative model – and estimate the (path integral of) model evidence $$p(\tilde(t)\vert m)$$ to compare different models. This generally involves an intractable marginalization over hidden states, so model evidence (or marginal likelihood) is replaced with a variational free energy bound :

$$\tilde{\mu}(t) = \underset{\tilde{\mu}}{\operatorname{arg\,min}} \{F(\tilde(t),\tilde)\} $$

$$ F(t)=E_{q} [G(\tilde,\tilde,\tilde)]-H[q(\tilde,\tilde\vert \tilde)] =-\ln p(\tilde\vert m)+D_{KL} [q(\tilde,\tilde\vert \tilde{\mu })\vert \vert p(\tilde,\tilde\vert \tilde,m)] $$

$$ G(t)=-\ln p(\tilde,\tilde,\tilde\vert m) $$

The second equality shows that minimizing variational free energy (i) minimizes the Kullback-Leibler divergence between the variational and true posterior density and (ii) renders the variational free energy (a bound approximation to) the negative log evidence (because the divergence can never be less than zero). Under the Laplace assumption $$q(\tilde,\tilde\vert \tilde)=\mathcal{N}(\tilde,C)$$ the variational density is Gaussian and the precision that minimizes free energy is $$C^{-1}=\Pi =\partial_{\tilde\tilde} G(\tilde)$$. This means that free-energy can be expressed in terms of the variational mean (omitting constants):

$$F=G(\tilde)+\textstyle{1 \over 2}\ln \vert \partial_{\tilde\tilde} G(\tilde)\vert$$

The variational means that minimize the (path integral) of free energy can now be recovered by solving the generalized filter:

$$\dot=D\tilde-\partial_{\tilde} F(\tilde,\tilde)$$

Where,$$D$$ is a block matrix derivative operator of identify matrices such that $$D\tilde=[{u}',{u}'',... ]^{T}$$

Variational basis
Generalised filtering is based on the following lemma: The self-consistent solution to $$\dot=D\tilde-\partial_{\tilde} F(s,\tilde)$$ satisfies the variational principle of stationary action, where action is the path integral of variational free energy

$$S=\int {dt} F(\tilde(t),\tilde(t))$$

Proof: self-consistency requires the motion of the mean to be the mean of the motion and (by the fundamental lemma of variational calculus)

$$\dot = D\tilde\Leftrightarrow \partial_{\tilde} F(\tilde,\tilde)=0\Leftrightarrow \delta_{\tilde} S=0$$

Put simply, small perturbations to the path of the mean do not change variational free energy and it has the least action of all possible (local) paths.

Remarks: Heuristically, generalised filtering performs a gradient decent on variational free energy in a moving frame of reference: $$\dot-D\tilde=-\partial_{\tilde} F(s,\tilde)$$, where the frame itself minimises variational free energy. For a related example in statistical physics, see Kerr and Graham who use ensemble dynamics in generalised coordinates to provide a generalised phase-space version of Langevin and associated Fokker-Planck equations.

In practice, generalised filtering uses local linearization over intervals $$\Delta t$$ to recover discrete updates

$$\begin{align} \Delta \tilde & =(\exp (\Delta t\cdot J)-I)J^{-1}\dot \\ J                   & =\partial_{\tilde} \dot=D-\partial _{\tilde\tilde} F(\tilde,\tilde) \end{align}$$

This updates the means of hidden variables at each interval (usually the interval between observations).

Generative (state-space) models in generalised coordinates
Usually, the generative density or model is specified in terms of a nonlinear input-state-output model with continuous nonlinear functions:

$$\begin{align} s & = g(x,u)+\omega_{s} \\ \dot & = f(x,u)+\omega_{x} \\ \end{align}$$

The corresponding generalised model (under local linearity assumptions) obtains the from the chain rule

$$ {\begin{align} \tilde & =\tilde(\tilde,\tilde)+\tilde_{s} \\ \\     s & =g(x,u)+\omega_{s} \\ {s}' & =\partial_{x} g\cdot {x}'+\partial_{u} g\cdot {u}'+{\omega }'_{x} \\ {s} & =\partial_{x} g\cdot {x}+\partial_{u} g\cdot {u}+{\omega }_{x} \\ \vdots \\ \end{align}} \qquad {\begin{align} \tilde & =\tilde(\tilde,\tilde)+\tilde_{x} \\ \\ \dot    & =f(x,u)+\omega_{x} \\ {\dot}' & =\partial_{x} f\cdot {x}'+\partial_{u} f\cdot {u}'+{\omega }'_{x} \\ {\dot}& =\partial_{x} f\cdot {x}+\partial_{u} f\cdot {u}''+{\omega }''_{x} \\ \vdots \\ \end{align}}$$

Gaussian assumptions about the random fluctuations $$\omega$$ then prescribe the likelihood and empirical priors on the motion of hidden states

$${\begin{align} p\left( {\tilde,\tilde,\tilde\vert m} \right) & = p\left( {\tilde\vert \tilde,\tilde,m} \right)p\left( {D\tilde\vert x,\tilde,m} \right)p(x\vert m)p(\tilde\vert m) \\ p\left( {\tilde\vert \tilde,\tilde,m} \right) & = {\mathcal{N}}(\tilde(\tilde,\tilde),\tilde(\tilde,\tilde)_{s} ) \\ p\left( {D\tilde\vert x,\tilde,m} \right)         & = {\mathcal{N}}(\tilde(\tilde,\tilde),\tilde(\tilde,\tilde)_{x} ) \\ \end{align}}$$

The covariances $$\tilde=V\otimes \Sigma$$ factorise into a covariance among variables and correlations $$V$$ among generalised fluctuations that encodes their autocorrelation:

$$V=\begin{bmatrix} 1 & 0  & {\ddot(0)}  & \cdots  \\ 0 & {-\ddot(0)}  & 0 \ & \ \\ {\ddot(0)} \ & 0 \ & {\ddot(0)} \ & \ \\ \vdots \ & \ & \ & \ddots \ \\ \end{bmatrix}$$

Here, $$\ddot(0)$$ is the second derivative of the autocorrelation function evaluated at zero. This is a ubiquitous measure of roughness in the theory of stochastic processes. Crucially, the precision (inverse variance) of high order derivatives fall to zero fairly quickly, which means it is only necessary to model relatively low order generalised motion (usually between two and eight) for any given or parameterized autocorrelation function.

Filtering discrete time series
When time series are observed as a discrete sequence of $$N$$ observations, the implicit sampling is treated as part of the generative process, where (using Taylor's theorem)

$${[s_{1} ,\dots ,s_{N} ]^{T} = (E\otimes I)\cdot \tilde(t)}: \qquad {E_{ij} =\frac{(i-t)^{(j-1)}}{(j-1)!}} $$

In principle, the entire sequence could be used to estimate hidden variables at each point in time. However, the precision of samples in the past and future falls quickly and can be ignored. This allows the scheme to assimilate data online, using local observations around each time point (typically between and eight).

Generalised filtering and model parameters
For any slowly varying model parameters of the equations of motion $$f(x,u,\theta )$$ or precision $$\tilde(x,u,\theta )$$ generalised filtering takes the following form (where $$\mu$$ corresponds to the variational mean of the parameters)

$$\begin{align} \dot   & = {\mu }' \\ \dot & = -\partial_{\mu } F(\tilde,\mu )-\kappa {\mu }' \\ \end{align}$$

Here, the solution $$\dot=0$$ minimizes variational free energy, when the motion of the mean is small. This can be seen by noting $$\dot={\dot}'=0\Rightarrow \partial_{\mu } F=0\Rightarrow \delta_{\mu } S=0$$. It is straightforward to show that this solution corresponds to a classical Newton update.

Generalised filtering and Kalman filtering
Classical filtering under Markovian or Weiner assumptions is equivalent to assuming the precision of the motion of random fluctuations is zero. In this limiting case, one only has to consider the states and their first derivative $$\tilde=(\mu ,{\mu }')$$. This means generalised filtering takes the form of a Kalman-Bucy filter, with prediction and correction terms:

$$\begin{align} \dot    & ={\mu }'-\partial_{\mu } F(s,\tilde) \\ \dot & =-\partial_{{\mu }'} F(s,\tilde) \\ \end{align}$$

Substituting this first-order filtering into the discrete update scheme above gives the equivalent of (extended) Kalman filtering.

Generalised filtering and particle filtering
Particle filtering is a sampling-based scheme that relaxes assumptions about the form of the variational or approximate posterior density. The corresponding generalised filtering scheme is called variational filtering. In variational filtering, an ensemble of particles diffuse over the free energy landscape in a frame of reference that moves with the expected (generalised) motion of the ensemble. This provides a relatively simple scheme that eschews Gaussian (unimodal) assumptions. Unlike particle filtering it does not require proposal densities -- or the elimination or creation of particles.

Generalised filtering and variational Bayes
Variational Bayes rests on a mean field partition of the variational density:

$$q(\tilde,\tilde,\theta \dots \vert \tilde,\mu )=q(\tilde,\tilde\vert \tilde)q(\theta \vert \mu )\dots$$

This partition induces a variational update or step for each marginal density -- that is usually solved analytically using conjugate priors. In generalised filtering, this leads to dynamic expectation maximisation. that comprises a D-step that optimises the sufficient statistics of unknown states, an E-step for parameters and an M-step for precisions.

Generalised filtering and predictive coding
Generalised filtering is usually used to invert hierarchical models of the following form

$$\begin{align} \tilde              & =\tilde^{(1)}(\tilde^{(1)},\tilde^{(1)})+\tilde_{s}^{(1)} \\ \dot^{(1)} & =f^{(i)}(\tilde^{(1)},\tilde^{(1)})+\tilde_{x}^{(1)} \\ \vdots \\ \tilde^{(i-1)}      & =g^{(i)}(\tilde^{(i)},\tilde^{(i)})+\tilde_{u}^{(i)} \\ \dot^{(i)} & =f^{(i)}(\tilde^{(i)},\tilde^{(i)})+\tilde_{x}^{(i)} \\ \vdots \\ \end{align}$$

The ensuing generalised gradient descent on free energy can then be expressed compactly in terms of prediction errors, where (omitting high order terms):

$$\begin{align} \dot_{u}^{(i)} & =D\tilde^{(u,i)}-\partial_{u} \tilde^{(i)}\cdot \Pi^{(i)}\tilde^{(i)}-\Pi ^{(i+1)}\tilde_{u}^{(i+1)} \\ \dot_{x}^{(i)} & =D\tilde^{(x,i)}-\partial_{x} \tilde^{(i)}\cdot \Pi^{(i)}\tilde^{(i)} \\ \\ \tilde_{u}^{(i)} & =\tilde_{u}^{(i-1)} -\tilde^{(i)} \\ \tilde_{x}^{(i)} & =D\tilde_{x}^{(i)} -\tilde^{(i)} \\ \end{align}$$

Here, $$\Pi^{(i)}$$ is the precision of random fluctuations at the$ i$-th level. This is known as generalised predictive coding [11], with linear predictive coding as a special case.

Applications
Generalised filtering has been primarily applied to biological timeseries -- in particular functional magnetic resonance imaging and electrophysiological data. This is usually in the context of dynamic causal modelling to make inferences about the underlying architectures of (neuronal) systems generating data. It is also used to simulate inference in terms of generalised (hierarchical) predictive coding in the brain.