Multidimensional Chebyshev's inequality

In probability theory, the multidimensional Chebyshev's inequality is a generalization of Chebyshev's inequality, which puts a bound on the probability of the event that a random variable differs from its expected value by more than a specified amount.

Let $$X$$ be an $$N$$-dimensional random vector with expected value $$\mu=\operatorname{E}[X] $$ and covariance matrix


 * $$V=\operatorname{E} [(X - \mu) (X - \mu)^T]. \, $$

If $$V$$ is a positive-definite matrix, for any real number $$t>0$$:

\Pr \left( \sqrt{( X-\mu)^T V^{-1} (X-\mu) } > t\right) \le \frac N {t^2} $$

Proof
Since $$V$$ is positive-definite, so is $$V^{-1}$$. Define the random variable



y = (X-\mu)^T V^{-1} (X-\mu). $$

Since $$y$$ is positive, Markov's inequality holds:



\Pr\left( \sqrt{(X-\mu)^T V^{-1} (X-\mu) } > t\right) = \Pr( \sqrt{y} > t) = \Pr(y > t^2) \le \frac{\operatorname{E}[y]}{t^2}. $$

Finally,


 * $$\begin{align}

\operatorname{E}[y] &= \operatorname{E}[(X-\mu)^T V^{-1} (X-\mu)]\\[6pt] &=\operatorname{E}[ \operatorname{trace} ( V^{-1} (X-\mu) (X-\mu)^T )]\\[6pt] &= \operatorname{trace} ( V^{-1} V ) = N \end{align}.$$

Infinite dimensions
There is a straightforward extension of the vector version of Chebyshev's inequality to infinite dimensional settings. Let $X$ be a random variable which takes values in a Fréchet space $$\mathcal X$$ (equipped with seminorms $⋅ _{α}$). This includes most common settings of vector-valued random variables, e.g., when $$\mathcal X$$ is a Banach space (equipped with a single norm), a Hilbert space, or the finite-dimensional setting as described above.

Suppose that $X$ is of "strong order two", meaning that


 * $$ \operatorname{E}\left(\| X\|_\alpha^2 \right) < \infty $$

for every seminorm $⋅ _{α}$. This is a generalization of the requirement that $X$ have finite variance, and is necessary for this strong form of Chebyshev's inequality in infinite dimensions. The terminology "strong order two" is due to Vakhania.

Let $$\mu \in \mathcal X$$ be the Pettis integral of $X$ (i.e., the vector generalization of the mean), and let


 * $$\sigma_a := \sqrt{\operatorname{E}\|X - \mu\|_\alpha^2}$$

be the standard deviation with respect to the seminorm $⋅ _{α}$. In this setting we can state the following:


 * General version of Chebyshev's inequality. $$\forall k > 0: \quad \Pr\left( \|X - \mu\|_\alpha \ge k \sigma_\alpha \right) \le \frac{1}{ k^2 }.$$

Proof. The proof is straightforward, and essentially the same as the finitary version. If $σ_{α} = 0$, then $X$ is constant (and equal to $μ$) almost surely, so the inequality is trivial.

If


 * $$\|X - \mu\|_\alpha \ge k \sigma_\alpha^2$$

then $X − μ_{α} > 0$, so we may safely divide by $X − μ_{α}$. The crucial trick in Chebyshev's inequality is to recognize that $$1 = \tfrac{\|X - \mu\|_\alpha^2}{\|X - \mu\|_\alpha^2}$$.

The following calculations complete the proof:


 * $$\begin{align}

\Pr\left( \|X - \mu\|_\alpha \ge k \sigma_\alpha \right) &= \int_\Omega \mathbf{1}_{\|X - \mu\|_\alpha \ge k \sigma_\alpha} \, \mathrm d\Pr \\ & = \int_\Omega \left ( \frac{\|X - \mu\|_\alpha^2}{\|X - \mu\|_\alpha^2} \right ) \cdot \mathbf{1}_{\|X - \mu\|_\alpha \ge k \sigma_\alpha} \, \mathrm d\Pr \\[6pt] &\le \int_\Omega \left (\frac{\|X - \mu\|_\alpha^2}{(k\sigma_\alpha)^2} \right ) \cdot \mathbf{1}_{\|X - \mu\|_\alpha \ge k \sigma_\alpha} \, \mathrm d\Pr \\[6pt] &\le \frac{1}{k^2 \sigma_\alpha^2} \int_\Omega \|X - \mu\|_\alpha^2 \, \mathrm d\Pr && \mathbf{1}_{\|X - \mu\|_\alpha \ge k \sigma_\alpha} \le 1\\[6pt] &= \frac{1}{k^2 \sigma_\alpha^2} \left (\operatorname{E}\|X - \mu\|_\alpha^2 \right )\\[6pt] &= \frac{1}{k^2 \sigma_\alpha^2} \left (\sigma_\alpha^2 \right )\\[6pt] &= \frac{1}{k^2} \end{align}$$