Lehmann–Scheffé theorem

In statistics, the Lehmann–Scheffé theorem is a prominent statement, tying together the ideas of completeness, sufficiency, uniqueness, and best unbiased estimation. The theorem states that any estimator that is unbiased for a given unknown quantity and that depends on the data only through a complete, sufficient statistic is the unique best unbiased estimator of that quantity. The Lehmann–Scheffé theorem is named after Erich Leo Lehmann and Henry Scheffé, given their two early papers.

If T is a complete sufficient statistic for θ and E(g(T)) = &tau;(&theta;) then g(T) is the uniformly minimum-variance unbiased estimator (UMVUE) of τ(&theta;).

Statement
Let $$\vec{X}= X_1, X_2, \dots, X_n$$ be a random sample from a distribution that has p.d.f (or p.m.f in the discrete case) $$f(x:\theta)$$ where $$\theta \in \Omega$$ is a parameter in the parameter space. Suppose $$Y = u(\vec{X})$$ is a sufficient statistic for θ, and let $$\{ f_Y(y:\theta): \theta \in \Omega\}$$ be a complete family. If $$\varphi:\operatorname{E}[\varphi(Y)] = \theta$$ then $$\varphi(Y)$$ is the unique MVUE of θ.

Proof
By the Rao–Blackwell theorem, if $$Z$$ is an unbiased estimator of θ then $$\varphi(Y):= \operatorname{E}[Z\mid Y]$$ defines an unbiased estimator of θ with the property that its variance is not greater than that of $$Z$$.

Now we show that this function is unique. Suppose $$W$$ is another candidate MVUE estimator of θ. Then again $$\psi(Y):= \operatorname{E}[W\mid Y]$$ defines an unbiased estimator of θ with the property that its variance is not greater than that of $$W$$. Then



\operatorname{E}[\varphi(Y) - \psi(Y)] = 0, \theta \in \Omega. $$

Since $$\{ f_Y(y:\theta): \theta \in \Omega\}$$ is a complete family



\operatorname{E}[\varphi(Y) - \psi(Y)] = 0 \implies \varphi(y) - \psi(y) = 0, \theta \in \Omega $$

and therefore the function $$\varphi$$ is the unique function of Y with variance not greater than that of any other unbiased estimator. We conclude that $$\varphi(Y)$$ is the MVUE.

Example for when using a non-complete minimal sufficient statistic
An example of an improvable Rao–Blackwell improvement, when using a minimal sufficient statistic that is not complete, was provided by Galili and Meilijson in 2016. Let $$X_1, \ldots, X_n$$ be a random sample from a scale-uniform distribution $$X \sim U ( (1-k) \theta, (1+k) \theta),$$ with unknown mean $$\operatorname{E}[X]=\theta$$ and known design parameter $$k \in (0,1)$$. In the search for "best" possible unbiased estimators for $$\theta$$, it is natural to consider $$X_1$$ as an initial (crude) unbiased estimator for $$\theta$$ and then try to improve it. Since $$X_1$$ is not a function of $$T = \left( X_{(1)}, X_{(n)} \right)$$, the minimal sufficient statistic for $$\theta$$ (where $$X_{(1)} = \min_i X_i $$ and $$X_{(n)} = \max_i X_i $$), it may be improved using the Rao–Blackwell theorem as follows:


 * $$\hat{\theta}_{RB} =\operatorname{E}_\theta[X_1\mid X_{(1)}, X_{( n)}] = \frac{X_{(1)}+X_{(n)}} 2.$$

However, the following unbiased estimator can be shown to have lower variance:


 * $$\hat{\theta}_{LV} = \frac 1 {k^2\frac{n-1}{n+1}+1} \cdot \frac{(1-k)X_{(1)} + (1+k) X_{(n)}} 2.$$

And in fact, it could be even further improved when using the following estimator:


 * $$\hat{\theta}_\text{BAYES}=\frac{n+1} n \left[1- \frac{\frac{X_{(1)} (1+k)}{X_{(n)} (1-k)}-1}{ \left (\frac{X_{(1)} (1+k)}{X_{(n)} (1-k)}\right )^{n+1} -1} \right] \frac{X_{(n)}}{1+k}$$

The model is a scale model. Optimal equivariant estimators can then be derived for loss functions that are invariant.