User:Rajagiryes

Invariant Estimator is an intuitively appealing non Bayesian estimator. It is also sometimes called an "equivariant estimator". In the estimation problem we have random vector $$x$$ from space $$X$$ with density function $$f(x|\theta)$$ when $$\theta$$ is from the space $$\Theta$$. We want to estimate $$\theta$$ given set of measurements from the distribution $$f(x|\theta)$$. The estimation is denoted by $$a$$, is a function of the measurements and is in the space $$A$$. The quality of the result is defined by a loss function $$L=L(a,\theta)$$ which determine a risk function $$R=R(a,\theta)=E[L(a,\theta)|\theta]$$. Generally speaking invariant estimator is an estimator that obey the 2 following rules:

1. Principle of Rational Invariance: The action taken in a decision problem should not depend on transformation on the measurement used

2. Invariance Principle: If two decision problems have the same formal structure (in terms of $$X$$, $$\Theta$$, $$f(x|\theta)$$ and $$L$$) then the same decision rule should be used in each problem

To define invariant estimator formally we will first set some definitions about groups of transformations:

Invariant Estimation Problem and Invariant Estimator
A group of transformation of $$X$$, to be denoted by $$G$$ is a set of (measurable) $$1:1$$ and onto transformation of $$X$$ into itself, which satisfies the following conditions:

1. If $$g_1\in G$$ and $$g_2\in G$$ then $$g_1,g_2\in G$$

2. If $$g\in G$$ then $$g^{-1}\in G$$ ($$g^{-1}(g(x))=x)$$

3. $$e\in G$$ ($$e(x)=x$$)

$$x_1$$ and $$x_2$$ in $$X$$ are equivalent if $$x_1=g(x_2)$$ for some $$g\in G$$. All the equivalent points form an equivalence class. Such equivalence class is called orbit (in $$X$$). The $$x_0$$ orbit, $$X(x_0)$$, is the set $$X(x_0)=\{g(x_0):g\in G\}$$. If $$X$$ consist of a single orbit than $$g$$ is said to be transitive.

A family of densities $$F$$ is said to be invariant under the group $$G$$ if, for every $$g\in G$$ and $$\theta\in \Theta$$ there exists a unique $$\theta^*\in \Theta$$ such that $$Y=g(x)$$ has density $$f(y|\theta^*)$$. $$\theta^*$$ will be denoted $$\bar{g}(\theta)$$.

If $$F$$ is invariant under the group $$G$$ than the loss function $$L(\theta,a)$$ is said to be invariant under $$G$$ if for every $$g\in G$$ and $$a\in A$$ there exists an $$a^*\in A$$ such that $$L(\theta,a)=L(\bar{g}(\theta),a^*)$$ for all $$\theta \in \Theta$$. $$a^*$$ will be denoted $$\tilde{g}(a)$$.

$$\bar{G}=\{\bar{g}:g\in G\}$$ is a group of transformations from $$\Theta$$ to itself and $$\tilde{G}=\{\tilde{g}: g \in G\}$$ is a group of transformations from $$A$$ to itself.

An estimation problem is invariant under $$G$$ if there exists such three groups $$G, \bar{G}, \tilde{G}$$.

For an estimation problem that is invariant under $$G$$, estimator $$\delta(x)$$ is invariant estimator under $$G$$ if for all $$x\in X$$ and $$g\in G$$ $$\delta(g(x)) = \tilde{g}(\delta(x))$$.

Properties of Invariant Estimators
1. The risk function of an invariant estimator $$\delta$$ is constant on orbits of $$\Theta$$. Equivalently $$R(\theta,\delta)=R(\bar{g}(\theta),\delta)$$ for all $$\theta \in \Theta$$ and $$\bar{g}\in \bar{G}$$.

2. The risk function of an invariant estimator with transitive $$\bar{g}$$ is constant.

For a given problem the invariant estimator with the lowest risk is termed the "best invariant estimator". Best invariant estimator cannot be achieved always. A special case for which it can be achieved is the case when $$\bar{g}$$ is transitive.

Location Parameter Problem Example
$$\theta$$ is a location parameter if the density of $$X$$ is $$f(x-\theta)$$. For $$ \Theta=A=\mathbb{R}^1 $$ and $$L=L(a-\theta)$$ the problem is invariant under $$g=\bar{g}=\tilde{g}=\{g_c:g_c(x)=x+c, c\in \mathbb{R}\}$$. The invariant estimator in this case must satisfy $$\delta(x+c)=\delta(x)+c, \forall c\in \mathbb{R}$$ thus it is of the form $$\delta(x)=x+K$$ ($$K\in \mathbb{R}$$). $$\bar{g}$$ is transitive on $$\Theta$$ so we have here constant risk: $$R(\theta,\delta)=R(0,\delta)=E[L(X+K)|\theta=0]$$. The best invariant estimator is the one that bring the risk $$R(\theta,\delta)$$ to minimum.

In the case that L is squared error $$\delta(x)=x-E[X|\theta=0]$$

Pitman Estimator
Given the estimation problem: $$X=(X_1,\dots,X_n)$$ that has density $$f(x_1-\theta,\dots,x_n-\theta)$$ and loss $$L(|a-\theta|)$$. This problem is invariant under $$G=\{g_c:g_c(x)=(x_1+c, \dots, x_n+c),c\in \mathbb{R}^1\}$$, $$\bar{G}=\{g_c:g_c(\theta)=\theta + c,c\in \mathbb{R}^1\}$$ and $$\tilde{G}=\{g_c:g_c(a)=a + c,c\in \mathbb{R}^1\}$$ (additive groups).

The best invariant estimator $$\delta(x)$$ is the one that minimize $$\frac{\int_{-\infty}^{\infty}{L(\delta(x)-\theta)f(x_1-\theta,\dots,x_n-\theta)d\theta}}{\int_{-\infty}^{\infty}{f(x_1-\theta,\dots,x_n-\theta)d\theta}}$$ (Pitman's estimator, 1939).

For the square error loss case we get that $$\delta(x)=\frac{\int_{-\infty}^{\infty}{\theta f(x_1-\theta,\dots,x_n-\theta)d\theta}}{\int_{-\infty}^{\infty}{f(x_1-\theta,\dots,x_n-\theta)d\theta}}$$

If $$x \sim N(\theta 1_n,I)\,\!$$ than $$\delta_{pitman} = \delta_{ML}=\frac{\sum{x_i}}{n}$$

If $$x \sim C(\theta 1_n,I)\,\!$$ than $$\delta_{pitman} \ne \delta_{ML}$$ and $$\delta_{pitman}=\sum_{k=1}^n{x_k[\frac{Re\{w_k\}}{\sum_{m=1}^{n}{Re\{w_k\}}}]},n>1$$ when $$w_k = \prod_{j\ne k}[\frac{1}{(x_k-x_j)^2+4\sigma^2}][1-\frac{2\sigma}{(x_k-x_j)}i]$$