User:Catalystcf/inference

Hypothesis testing

 * if p-value $$\alpha$$ under $$H_0$$ -> reject; $$\alpha=P(H_0  rejected | H_0  =  true) =P(T \in R | H_0 =true)$$


 * $$ \beta =P( accept H_0| H_a = true)$$. Power of the test = ($$1-\beta$$)  probability of rejecting the null hypothesis when the alternate hypothesis

is true.


 * power function : $$ \pi(\theta)=P(X \in R| \theta) \theta \in \Omega $$

$$ \alpha = sup(\pi(\theta)) \theta \in \Omega_0 $$,

REST OF THE STUFF
Every MLE is AN distributed: $$ \hat{\theta} \rightarrow{}AN( \theta, V(\hat{\theta}) ) $$ ; if $$\hat{\theta}$$ is an MLE of &theta; and if $$g({\dot{}})$$ is a 1-1 function, the g($$\hat{\theta}$$) is an MLE of g(&theta;)
 * MLE

MLE of $$U(\theta)$$ is $$Y_n=max(X_1,X_2,...)$$


 * MSE
 * $$\operatorname{MSE}(\hat{\theta})=\operatorname{E}((\hat{\theta}-\theta)^2)$$
 * $$\operatorname{MSE}(\hat{\theta})=\operatorname{Var}\left(\hat{\theta}\right)+ \left(\operatorname{Bias}(\hat{\theta},\theta)\right)^2.$$


 * LRT

If the null hypothesis is true, the observation is a sequence of $$n$$ independent identically distributed random variables, and $$\theta$$ is not on the boundary of $$\Theta$$, then as the sample size $$n$$ approaches $$\infty$$, the test statistic $$-2 \log(\Lambda)$$ will be asymptotically $\chi^2$ distributed with degrees of freedom equal to the difference in dimensionality of $$\Theta$$ and $$\Theta_0$$.

$$-2 log r(x) ~ \chi^{2}_{df}$$ df- is the difference between parameters being estimated  under $$ \Theta $$ and under $$\theta_0$$
 * $$\Lambda(x)=\frac{\sup\{\,L(\theta\mid x):\theta\in\Theta_0\,\}}{\sup\{\,L(\theta\mid x):\theta\in\Theta\,\}},

\theta$$ is at MLE, at lower boundary for decreasing $$H_0$$

(When domain of X depends on the parameter)
 * order statistic
 * MAX: $$f_{X_{(n)}}(x)= n[F_{X}(x)]^{n-1} f_{X}(x)$$; $$F_{X_{(n)}}(x)= [F_{X}(x)]^{n}$$
 * MIN: $$f_{X_{(1)}}(x) = n[1 - F_X(x)]^{(n-1)}f_X(x)$$;$$F_{X_{(1)}}(x) = 1 - [1 - F_X(x)]^n $$


 * Fisher
 * $$I_n(\beta) = -n\mathbb{E}{(\frac{d^2 log f(y | \beta ) }{d \beta ^2})} = \mathbb{E}_{\beta}{([ \frac{d}{dt} logf(y|\beta) ]^2)}$$
 * $$V[\hat\theta] \geq \, {1} / {\mathcal{I}\left(\theta\right)}$$
 * $$I_n(\theta) = nI(\theta) $$
 * $$E[\sum{X_i}]=n*E[X_i]$$
 * (observed) $$\widehat{I_n(\theta)}=-\frac{1}{n} ln^{''}(\hat\theta)$$


 * Delta Method (CLT)
 * $$ Y_n \xrightarrow{} AN( \mu, \frac{\sigma^2}{n} ) $$


 * $$ g(Y_n) \xrightarrow{} AN( g(\mu), [g^'(\mu)]^2 \frac{\sigma^2}{n})  $$


 * $$ g(\hat\theta)-g(\theta) =N(0,[g^'(\hat{\theta})]^2\frac{1}{I(\hat\theta)} )$$


 * sufficient statistic

Let $$ {X}=(X_1,\ldots,X_n)$$ be a random vector of random variables representing n observations. A statistic $$ T=T(\boldsymbol{X})$$ of $$ \boldsymbol{X}$$  for the parameter $$\theta$$ is called a sufficient statistic, or a sufficient estimator, if the conditional probability distribution of  $$\boldsymbol{X}$$ given $$ T(\boldsymbol{X})= $$ is not a function of $$ \theta$$ (equivalently, does not depend on $$ \theta$$). $$L(\theta|\boldsymbol{x}) = f_n(\boldsymbol{x}|\theta)=u(x)v[r(x),\theta] $$
 * poison, normal ($$\sigma$$ known): $$T(X)=\sum{X_{i}}$$, uniform $$T(X)=max(X_1,\ldots,X_n)$$
 * normal ($$\mu$$ known): ($$\sum{(X_i-\mu)^2}$$)

$$f_{X\mid Y=y}(x)={f_X(x) L_{X\mid Y=y}(x) \over {\int_{-\infty}^\infty f_X(x) L_{X\mid Y=y}(x)\,dx}}$$ $$\pi(\theta! x) \rightarrow f_n(x!\theta) \pi(\theta) $$
 * bayesean
 * Loss Function

Absolute Error LF $$ L(\theta)=!\theta - \hat{\theta}!
 * $$ is min whend d=median (Lebnitz rule)
 * $$E[(X-d)^2]$$ is minimized when d=E(X)

$$\tilde{\beta}= argmin_{ a \in \Omega }= \int(\beta - a)^2 \pi(\beta! x) d\beta)$$

1-a% CI: $$ \theta+\Phi^{-1}(1-\alpha/2)\sqrt{ var(\hat\theta)} $$
 * CI

$$ P(a(x) \leq \sigma^2 \leq b(x) )= 1-\alpha$$
 * pivot


 * F-tesT
 * $$ Y\rightarrow \chi^{2}_{m},Z\rightarrow \chi^{2}_{n}, X=\frac{Y/m}{Z/n} \rightarrow F_{m,n}\rightarrow \frac{1}{F_{n,m}} $$
 * if $$\sigma^{2}_{1}=\sigma^{2}_{2},V\rightarrow F_{m-1,n-1}, V=\frac{S_{x}^{2}}{S_{y}^{2}}$$, reject when $$

V >F_{m-1, n-1}(1-\alpha/2) $$


 * T/Chi
 * $$S^2=\frac{\sum{(X_i-\overline{X})^2}} {n-1}$$
 * $$t_{n-1} \rightarrow \frac{\overline{X}-\mu}{S/\sqrt{n} }$$
 * $$\chi_{n-1}^{2} \rightarrow \frac{\sum{(\overline{X}-X_i)^2} }{\sigma^2 }$$
 * $$\overline{X} +/- t_{n-1,\alpha/2 } \frac{S}{\sqrt{n} } $$
 * $$X_i\rightarrow N(\mu_1, \sigma), Y_i\rightarrow N(\mu_2, \sigma), $$
 * $$ LRT=\frac { \overline{X}_m -\overline{Y}_m } {S_p*sqrt(1/m +1/n)} $$


 * Matrix
 * $$ (ABC)^{-1}=C^{-1}B^{-1}A^{-1} $$
 * $$(A^T)^{-1}=(A^{-1})^{T}$$
 * $$ (ABC)^{T}=C^{T}B^{T}A^{T} $$

$$
 * misc
 * CLT $$\sum{X_i} appx N(n\mu, n\sigma^2)$$
 * Leibniz: $$ {d\over dx}\, \int_{f_1(x)}^{f_2(x)} g(t) \,dt = g(f_2(x)) {f_2'(x)} - g(f_1(x)) {f_1'(x)} $$
 * General Leibntiz : $$ {d\over dx}\, \int_{f_1(x)}^{f_2(x)} g(t,x) \,dt = g(f_2(x),x) {f_2'(x)} - g(f_1(x),x) {f_1'(x)} + \int_{f_1(x)}^{f_2(x)} {\partial \over \partial x}  g(t,x) \,dt $$
 * Taylor: $$ f(a)+\frac{f'(a)}{1!}(x-a)+\frac{f''(a)}{2!}(x-a)^2+\frac{f^{(3)}(a)}{3!}(x-a)^3+\cdots = \sum_{n=0}^{\infin} \frac{f^{(n)}(a)}{n!} (x-a)^{n}
 * $$ \log_b(x) = \frac{\log_k(x)}{\log_k(b)}. $$