Cross-covariance matrix

In probability theory and statistics, a cross-covariance matrix is a matrix whose element in the i, j position is the covariance between the i-th element of a random vector and j-th element of another random vector. A random vector is a random variable with multiple dimensions. Each element of the vector is a scalar random variable. Each element has either a finite number of observed empirical values or a finite or infinite number of potential values. The potential values are specified by a theoretical joint probability distribution. Intuitively, the cross-covariance matrix generalizes the notion of covariance to multiple dimensions.

The cross-covariance matrix of two random vectors $$\mathbf{X}$$ and $$\mathbf{Y}$$ is typically denoted by $$\operatorname{K}_{\mathbf{X}\mathbf{Y}}$$ or $$\Sigma_{\mathbf{X}\mathbf{Y}}$$.

Definition
For random vectors $$\mathbf{X}$$ and $$\mathbf{Y}$$, each containing random elements whose expected value and variance exist, the cross-covariance matrix of $$\mathbf{X}$$ and $$\mathbf{Y}$$ is defined by

where $$\mathbf{\mu_X} = \operatorname{E}[\mathbf{X}]$$ and $$\mathbf{\mu_Y} = \operatorname{E}[\mathbf{Y}]$$ are vectors containing the expected values of $$\mathbf{X}$$ and $$\mathbf{Y}$$. The vectors $$\mathbf{X}$$ and $$\mathbf{Y}$$ need not have the same dimension, and either might be a scalar value.

The cross-covariance matrix is the matrix whose $$(i,j)$$ entry is the covariance


 * $$\operatorname{K}_{X_i Y_j} = \operatorname{cov}[X_i, Y_j] = \operatorname{E}[(X_i - \operatorname{E}[X_i])(Y_j - \operatorname{E}[Y_j])]$$

between the i-th element of $$\mathbf{X}$$ and the j-th element of $$\mathbf{Y}$$. This gives the following component-wise definition of the cross-covariance matrix.



\operatorname{K}_{\mathbf{X}\mathbf{Y}}= \begin{bmatrix} \mathrm{E}[(X_1 - \operatorname{E}[X_1])(Y_1 - \operatorname{E}[Y_1])] & \mathrm{E}[(X_1 - \operatorname{E}[X_1])(Y_2 - \operatorname{E}[Y_2])] & \cdots & \mathrm{E}[(X_1 - \operatorname{E}[X_1])(Y_n - \operatorname{E}[Y_n])] \\ \\ \mathrm{E}[(X_2 - \operatorname{E}[X_2])(Y_1 - \operatorname{E}[Y_1])] & \mathrm{E}[(X_2 - \operatorname{E}[X_2])(Y_2 - \operatorname{E}[Y_2])] & \cdots & \mathrm{E}[(X_2 - \operatorname{E}[X_2])(Y_n - \operatorname{E}[Y_n])] \\ \\ \vdots & \vdots & \ddots & \vdots \\ \\ \mathrm{E}[(X_m - \operatorname{E}[X_m])(Y_1 - \operatorname{E}[Y_1])] & \mathrm{E}[(X_m - \operatorname{E}[X_m])(Y_2 - \operatorname{E}[Y_2])] & \cdots & \mathrm{E}[(X_m - \operatorname{E}[X_m])(Y_n - \operatorname{E}[Y_n])] \end{bmatrix} $$

Example
For example, if $$\mathbf{X} = \left( X_1,X_2,X_3 \right)^{\rm T}$$ and $$\mathbf{Y} = \left( Y_1,Y_2 \right)^{\rm T}$$ are random vectors, then $$ \operatorname{cov}(\mathbf{X},\mathbf{Y}) $$ is a $$3 \times 2$$ matrix whose $$(i,j)$$-th entry is $$\operatorname{cov}(X_i,Y_j)$$.

Properties
For the cross-covariance matrix, the following basic properties apply:


 * 1) $$ \operatorname{cov}(\mathbf{X},\mathbf{Y}) = \operatorname{E}[\mathbf{X} \mathbf{Y}^{\rm T}] - \mathbf{\mu_X} \mathbf{\mu_Y}^{\rm T} $$
 * 2) $$ \operatorname{cov}(\mathbf{X},\mathbf{Y}) = \operatorname{cov}(\mathbf{Y},\mathbf{X})^{\rm T}$$
 * 3) $$ \operatorname{cov}(\mathbf{X_1} + \mathbf{X_2},\mathbf{Y}) = \operatorname{cov}(\mathbf{X_1},\mathbf{Y}) + \operatorname{cov}(\mathbf{X_2}, \mathbf{Y})$$
 * 4) $$\operatorname{cov}(A\mathbf{X}+ \mathbf{a}, B^{\rm T}\mathbf{Y} + \mathbf{b}) = A\, \operatorname{cov}(\mathbf{X}, \mathbf{Y}) \,B$$
 * 5) If $$\mathbf{X}$$ and $$\mathbf{Y}$$ are independent (or somewhat less restrictedly, if every random variable in $$\mathbf{X}$$ is uncorrelated with every random variable in $$\mathbf{Y}$$), then $$\operatorname{cov}(\mathbf{X},\mathbf{Y}) = 0_{p\times q}$$

where $$\mathbf{X}$$, $$\mathbf{X_1}$$ and $$\mathbf{X_2}$$ are random $$p \times 1$$ vectors, $$\mathbf{Y}$$ is a random $$q \times 1$$ vector, $$\mathbf{a}$$ is a $$q \times 1$$ vector, $$\mathbf{b}$$ is a $$p \times 1$$ vector, $$A$$ and $$B$$ are $$q \times p$$ matrices of constants, and $$0_{p\times q}$$ is a $$p \times q$$ matrix of zeroes.

Definition for complex random vectors
If $$\mathbf{Z}$$ and $$\mathbf{W}$$ are complex random vectors, the definition of the cross-covariance matrix is slightly changed. Transposition is replaced by Hermitian transposition:


 * $$\operatorname{K}_{\mathbf{Z}\mathbf{W}} = \operatorname{cov}(\mathbf{Z},\mathbf{W}) \stackrel{\mathrm{def}}{=}\ \operatorname{E}[(\mathbf{Z}-\mathbf{\mu_Z})(\mathbf{W}-\mathbf{\mu_W})^{\rm H}]$$

For complex random vectors, another matrix called the pseudo-cross-covariance matrix is defined as follows:


 * $$\operatorname{J}_{\mathbf{Z}\mathbf{W}} = \operatorname{cov}(\mathbf{Z},\overline{\mathbf{W}}) \stackrel{\mathrm{def}}{=}\ \operatorname{E}[(\mathbf{Z}-\mathbf{\mu_Z})(\mathbf{W}-\mathbf{\mu_W})^{\rm T}]$$

Uncorrelatedness
Two random vectors $$\mathbf{X}$$ and $$\mathbf{Y}$$ are called uncorrelated if their cross-covariance matrix $$\operatorname{K}_{\mathbf{X}\mathbf{Y}}$$ matrix is a zero matrix.

Complex random vectors $$\mathbf{Z}$$ and $$\mathbf{W}$$ are called uncorrelated if their covariance matrix and pseudo-covariance matrix is zero, i.e. if $$\operatorname{K}_{\mathbf{Z}\mathbf{W}} = \operatorname{J}_{\mathbf{Z}\mathbf{W}} = 0$$.