Local asymptotic normality

In statistics, local asymptotic normality is a property of a sequence of statistical models, which allows this sequence to be asymptotically approximated by a normal location model, after an appropriate rescaling of the parameter. An important example when the local asymptotic normality holds is in the case of i.i.d sampling from a regular parametric model.

The notion of local asymptotic normality was introduced by and is fundamental in the treatment of estimator and test efficiency.

Definition
A sequence of parametric statistical models {&thinsp;Pn,θ: θ ∈ Θ&thinsp;} is said to be locally asymptotically normal (LAN) at θ if there exist matrices rn and Iθ and a random vector Δn,θ ~ N(0, Iθ) such that, for every converging sequence $h_{n} → h$,

\ln \frac{dP_{\!n,\theta+r_n^{-1}h_n}}{dP_{n,\theta}} = h'\Delta_{n,\theta} - \frac12 h'I_\theta\,h + o_{P_{n,\theta}}(1), $$ where the derivative here is a Radon–Nikodym derivative, which is a formalised version of the likelihood ratio, and where o is a type of big O in probability notation. In other words, the local likelihood ratio must converge in distribution to a normal random variable whose mean is equal to minus one half the variance:

\ln \frac{dP_{\!n,\theta+r_n^{-1}h_n}}{dP_{n,\theta}}\ \ \xrightarrow{d}\ \ \mathcal{N}\Big( {-\tfrac12} h'I_\theta\,h,\ h'I_\theta\,h\Big). $$

The sequences of distributions $$P_{\!n,\theta+r_n^{-1}h_n}$$ and $$P_{n,\theta}$$ are contiguous.

Example
The most straightforward example of a LAN model is an iid model whose likelihood is twice continuously differentiable. Suppose {&thinsp;X1, X2, …, Xn&thinsp;} is an iid sample, where each Xi has density function f(x, θ). The likelihood function of the model is equal to

p_{n,\theta}(x_1,\ldots,x_n;\,\theta) = \prod_{i=1}^n f(x_i,\theta). $$ If f is twice continuously differentiable in θ, then
 * $$\begin{align}

\ln p_{n,\theta+\delta\theta} &\approx \ln p_{n,\theta} + \delta\theta'\frac{\partial \ln p_{n,\theta}}{\partial\theta} + \frac12 \delta\theta' \frac{\partial^2 \ln p_{n,\theta}}{\partial\theta\,\partial\theta'} \delta\theta \\ &= \ln p_{n,\theta} + \delta\theta' \sum_{i=1}^n\frac{\partial \ln f(x_i,\theta)}{\partial\theta} + \frac12 \delta\theta' \bigg[\sum_{i=1}^n\frac{\partial^2 \ln f(x_i,\theta)}{\partial\theta\,\partial\theta'} \bigg]\delta\theta. \end{align}$$

Plugging in $$\delta\theta=h/\sqrt{n}$$, gives

\ln \frac{p_{n,\theta+h/\sqrt{n}}}{p_{n,\theta}} = h' \Bigg(\frac{1}{\sqrt{n}} \sum_{i=1}^n\frac{\partial \ln f(x_i,\theta)}{\partial\theta}\Bigg) \;-\; \frac12 h' \Bigg( \frac1n \sum_{i=1}^n - \frac{\partial^2 \ln f(x_i,\theta)}{\partial\theta\,\partial\theta'} \Bigg) h \;+\; o_p(1). $$ By the central limit theorem, the first term (in parentheses) converges in distribution to a normal random variable Δθ ~ N(0, Iθ), whereas by the law of large numbers the expression in second parentheses converges in probability to Iθ, which is the Fisher information matrix:

I_\theta = \mathrm{E}\bigg[{- \frac{\partial^2 \ln f(X_i,\theta)}{\partial\theta\,\partial\theta'}}\bigg] = \mathrm{E}\bigg[\bigg(\frac{\partial \ln f(X_i,\theta)}{\partial\theta}\bigg)\bigg(\frac{\partial \ln f(X_i,\theta)}{\partial\theta}\bigg)'\,\bigg]. $$ Thus, the definition of the local asymptotic normality is satisfied, and we have confirmed that the parametric model with iid observations and twice continuously differentiable likelihood has the LAN property.