Covariance operator

In probability theory, for a probability measure P on a Hilbert space H with inner product $$\langle \cdot,\cdot\rangle $$, the covariance of P is the bilinear form Cov: H &times; H → R given by


 * $$\mathrm{Cov}(x, y) = \int_{H} \langle x, z \rangle \langle y, z \rangle \, \mathrm{d} \mathbf{P} (z)$$

for all x and y in H. The covariance operator C is then defined by


 * $$\mathrm{Cov}(x, y) = \langle Cx, y \rangle$$

(from the Riesz representation theorem, such operator exists if Cov is bounded). Since Cov is symmetric in its arguments, the covariance operator is self-adjoint. When P is a centred Gaussian measure, C is also a nuclear operator. In particular, it is a compact operator of trace class, that is, it has finite trace.

Even more generally, for a probability measure P on a Banach space B, the covariance of P is the bilinear form on the algebraic dual B#, defined by


 * $$\mathrm{Cov}(x, y) = \int_{B} \langle x, z \rangle \langle y, z \rangle \, \mathrm{d} \mathbf{P} (z)$$

where $$ \langle x, z \rangle $$ is now the value of the linear functional x on the element z.

Quite similarly, the covariance function of a function-valued random element (in special cases is called random process or random field) z is


 * $$\mathrm{Cov}(x, y) = \int z(x) z(y) \, \mathrm{d} \mathbf{P} (z) = E(z(x) z(y))$$

where z(x) is now the value of the function z at the point x, i.e., the value of the linear functional $$ u \mapsto u(x) $$ evaluated at z.