Normal-Wishart distribution

In probability theory and statistics, the normal-Wishart distribution (or Gaussian-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and precision matrix (the inverse of the covariance matrix).

Definition
Suppose


 * $$ \boldsymbol\mu|\boldsymbol\mu_0,\lambda,\boldsymbol\Lambda \sim \mathcal{N}(\boldsymbol\mu_0,(\lambda\boldsymbol\Lambda)^{-1}) $$

has a multivariate normal distribution with mean $$\boldsymbol\mu_0$$ and covariance matrix $$(\lambda\boldsymbol\Lambda)^{-1}$$, where


 * $$\boldsymbol\Lambda|\mathbf{W},\nu \sim \mathcal{W}(\boldsymbol\Lambda|\mathbf{W},\nu)$$

has a Wishart distribution. Then $$(\boldsymbol\mu,\boldsymbol\Lambda) $$ has a normal-Wishart distribution, denoted as
 * $$ (\boldsymbol\mu,\boldsymbol\Lambda) \sim \mathrm{NW}(\boldsymbol\mu_0,\lambda,\mathbf{W},\nu).

$$

Probability density function

 * $$f(\boldsymbol\mu,\boldsymbol\Lambda|\boldsymbol\mu_0,\lambda,\mathbf{W},\nu) = \mathcal{N}(\boldsymbol\mu|\boldsymbol\mu_0,(\lambda\boldsymbol\Lambda)^{-1})\ \mathcal{W}(\boldsymbol\Lambda|\mathbf{W},\nu)$$

Marginal distributions
By construction, the marginal distribution over $$\boldsymbol\Lambda$$ is a Wishart distribution, and the conditional distribution over $$\boldsymbol\mu$$ given $$\boldsymbol\Lambda$$ is a multivariate normal distribution. The marginal distribution over $$\boldsymbol\mu$$ is a multivariate t-distribution.

Posterior distribution of the parameters
After making $$n$$ observations $$\boldsymbol{x}_1, \dots, \boldsymbol{x}_n$$, the posterior distribution of the parameters is
 * $$(\boldsymbol\mu,\boldsymbol\Lambda) \sim \mathrm{NW}(\boldsymbol\mu_n,\lambda_n,\mathbf{W}_n,\nu_n),$$

where
 * $$\lambda_n = \lambda + n,$$
 * $$\boldsymbol\mu_n = \frac{\lambda \boldsymbol\mu_0 + n\boldsymbol{\bar{x}}}{\lambda + n},$$
 * $$\nu_n = \nu + n,$$
 * $$\mathbf{W}_n^{-1} = \mathbf{W}^{-1} + \sum_{i=1}^n (\boldsymbol{x}_i - \boldsymbol{\bar{x}})(\boldsymbol{x}_i - \boldsymbol{\bar{x}})^T + \frac{n \lambda}{n + \lambda} (\boldsymbol{\bar{x}} - \boldsymbol\mu_0)(\boldsymbol{\bar{x}} - \boldsymbol\mu_0)^T.$$

Generating normal-Wishart random variates
Generation of random variates is straightforward:
 * 1) Sample $$\boldsymbol\Lambda$$ from a Wishart distribution with parameters $$\mathbf{W}$$ and $$\nu$$
 * 2) Sample $$\boldsymbol\mu$$ from a multivariate normal distribution with mean $$\boldsymbol\mu_0$$ and variance $$(\lambda\boldsymbol\Lambda)^{-1}$$

Related distributions

 * The normal-inverse Wishart distribution is essentially the same distribution parameterized by variance rather than precision.
 * The normal-gamma distribution is the one-dimensional equivalent.
 * The multivariate normal distribution and Wishart distribution are the component distributions out of which this distribution is made.