Uniformly most powerful test

In statistical hypothesis testing, a uniformly most powerful (UMP) test is a hypothesis test which has the greatest power $$1 - \beta$$ among all possible tests of a given size α. For example, according to the Neyman–Pearson lemma, the likelihood-ratio test is UMP for testing simple (point) hypotheses.

Setting
Let $$X$$ denote a random vector (corresponding to the measurements), taken from a parametrized family of probability density functions or probability mass functions $$f_{\theta}(x)$$, which depends on the unknown deterministic parameter $$\theta \in \Theta$$. The parameter space $$\Theta$$ is partitioned into two disjoint sets $$\Theta_0$$ and $$\Theta_1$$. Let $$H_0$$ denote the hypothesis that $$\theta \in \Theta_0$$, and let $$H_1$$ denote the hypothesis that $$\theta \in \Theta_1$$. The binary test of hypotheses is performed using a test function $$\varphi(x)$$ with a reject region $$R$$ (a subset of measurement space).
 * $$\varphi(x) =

\begin{cases} 1 & \text{if } x \in R \\ 0 & \text{if } x \in R^c \end{cases}$$ meaning that $$H_1$$ is in force if the measurement $$ X \in R$$ and that $$H_0$$ is in force if the measurement $$X\in R^c$$. Note that $$R \cup R^c$$ is a disjoint covering of the measurement space.

Formal definition
A test function $$\varphi(x)$$ is UMP of size $$\alpha$$ if for any other test function $$\varphi'(x)$$ satisfying
 * $$\sup_{\theta\in\Theta_0}\; \operatorname{E}[\varphi'(X)|\theta]=\alpha'\leq\alpha=\sup_{\theta\in\Theta_0}\; \operatorname{E}[\varphi(X)|\theta]\,$$

we have
 * $$ \forall \theta \in \Theta_1, \quad \operatorname{E}[\varphi'(X)|\theta]= 1 - \beta'(\theta) \leq 1 - \beta(\theta) =\operatorname{E}[\varphi(X)|\theta].$$

The Karlin–Rubin theorem
The Karlin–Rubin theorem can be regarded as an extension of the Neyman–Pearson lemma for composite hypotheses. Consider a scalar measurement having a probability density function parameterized by a scalar parameter θ, and define the likelihood ratio $$ l(x) = f_{\theta_1}(x) / f_{\theta_0}(x)$$. If $$l(x)$$ is monotone non-decreasing, in $$x$$, for any pair $$\theta_1 \geq \theta_0$$ (meaning that the greater $$x$$ is, the more likely $$H_1$$ is), then the threshold test:
 * $$\varphi(x) =

\begin{cases} 1 & \text{if } x > x_0 \\ 0 & \text{if } x < x_0 \end{cases}$$
 * where $$x_0$$ is chosen such that $$\operatorname{E}_{\theta_0}\varphi(X)=\alpha$$

is the UMP test of size α for testing $$ H_0: \theta \leq \theta_0 \text{ vs. } H_1: \theta > \theta_0 .$$

Note that exactly the same test is also UMP for testing $$ H_0: \theta = \theta_0 \text{ vs. } H_1: \theta > \theta_0 .$$

Important case: exponential family
Although the Karlin-Rubin theorem may seem weak because of its restriction to scalar parameter and scalar measurement, it turns out that there exist a host of problems for which the theorem holds. In particular, the one-dimensional exponential family of probability density functions or probability mass functions with
 * $$f_\theta(x) = g(\theta) h(x) \exp(\eta(\theta) T(x))$$

has a monotone non-decreasing likelihood ratio in the sufficient statistic $$T(x)$$, provided that $$\eta(\theta)$$ is non-decreasing.

Example
Let $$X=(X_0 ,\ldots, X_{M-1})$$ denote i.i.d. normally distributed $$N$$-dimensional random vectors with mean $$\theta m$$ and covariance matrix $$R$$. We then have


 * $$\begin{align}

f_\theta (X) = {} & (2 \pi)^{-MN/2} |R|^{-M/2} \exp \left\{-\frac 1 2 \sum_{n=0}^{M-1} (X_n - \theta m)^T R^{-1}(X_n - \theta m) \right\} \\[4pt] = {} & (2 \pi)^{-MN/2} |R|^{-M/2} \exp \left\{-\frac 1 2 \sum_{n=0}^{M-1} \left (\theta^2 m^T R^{-1} m \right ) \right\} \\[4pt] & \exp \left\{-\frac 1 2 \sum_{n=0}^{M-1} X_n^T R^{-1} X_n \right\} \exp \left\{\theta m^T R^{-1} \sum_{n=0}^{M-1}X_n \right\} \end{align}$$

which is exactly in the form of the exponential family shown in the previous section, with the sufficient statistic being


 * $$T(X) = m^T R^{-1} \sum_{n=0}^{M-1}X_n.$$

Thus, we conclude that the test
 * $$\varphi(T) = \begin{cases} 1 & T > t_0 \\ 0 & T < t_0 \end{cases} \qquad \operatorname{E}_{\theta_0} \varphi (T) = \alpha$$

is the UMP test of size $$\alpha$$ for testing $$H_0: \theta \leqslant \theta_0$$ vs. $$H_1: \theta > \theta_0$$

Further discussion
Finally, we note that in general, UMP tests do not exist for vector parameters or for two-sided tests (a test in which one hypothesis lies on both sides of the alternative). The reason is that in these situations, the most powerful test of a given size for one possible value of the parameter (e.g. for $$\theta_1$$ where $$\theta_1 > \theta_0$$) is different from the most powerful test of the same size for a different value of the parameter (e.g. for $$\theta_2$$ where $$\theta_2 < \theta_0$$). As a result, no test is uniformly most powerful in these situations.