User:PAR/Test

Entropy
The information entropy of the Von Mises distribution is defined as :


 * $$H = -\int_\Gamma f(\theta;\mu,\kappa)\,\ln(f(\theta;\mu,\kappa))\,d\theta\,$$

where $$\Gamma$$ is any interval of length $$2\pi$$. The logarithm of the density of the Von Mises distribution is straightforward:


 * $$\ln(f(\theta;\mu,\kappa))=-\ln(2\pi I_0(\kappa))+ \kappa \cos(\theta)\,$$

The characteristic function representation for the Von Mises distribution is:


 * $$f(\theta;\mu,\kappa) =\frac{1}{2\pi}\left(1+2\sum_{n=1}^\infty\phi_n\cos(n\theta)\right)$$

where $$\phi_n= I_{|n|}(\kappa)/I_0(\kappa)$$. Substituting these expressions into the entropy integral, exchanging the order of integration and summation, and using the orthogonality of the cosines, the entropy may be written:


 * $$H = \ln(2\pi I_0(\kappa))-\kappa\phi_1 = \ln(2\pi I_0(\kappa))-\kappa\frac{I_1(\kappa)}{I_0(\kappa)}$$

Collapsible

 * {| class="collapsible" width="30%" style="text-align:left"

!Table of closed-form stable distribution PDF's where $$S_{\mu,\nu}(z)$$ is a Lommel function. (Reference: Garoni & Frankel .)
 * $$f\left(x;\tfrac{1}{3},0\right) = Re\left( \frac{2 \exp(- i \pi /4)}{3 \sqrt{3} \pi} x^{-3/2} S_{0,1/3} \Bigl(\frac{2 \exp(i \pi /4)}{3 \sqrt{3}} x^{-1/2} \Bigr) \right)$$
 * $$f\left(x;\tfrac{1}{3},0\right) = Re\left( \frac{2 \exp(- i \pi /4)}{3 \sqrt{3} \pi} x^{-3/2} S_{0,1/3} \Bigl(\frac{2 \exp(i \pi /4)}{3 \sqrt{3}} x^{-1/2} \Bigr) \right)$$

where S(x) and C(x) are Fresnel Integrals (Reference: Hopcraft et. al. .)
 * $$f\left(x;\tfrac{1}{2},0\right) = \frac{\vert x \vert^{-3/2}}\left(\sin\left(\frac{1}{4\vert x \vert}\right)\left[\frac{1}{2}-S\left(\sqrt{\frac{1}{2\pi\vert x \vert}}\right)\right]+\cos\left(\frac{1}{4\vert x \vert}\right)\left[\frac{1}{2}-C\left(\sqrt{\frac{1}{2\pi\vert x \vert}}\right)\right]\right)$$


 * $$f\left(x;\tfrac{2}{3},0,\right) = \frac{1}{2\sqrt{3\pi}}\vert x \vert ^ {-1} \exp\left(\frac{2}{27}x^{-2}\right) W_{-1/2,1/6}\left(\frac{4}{27}x^{-2}\right)$$
 * $$f\left(x;\tfrac{2}{3},0,\right) = \frac{1}{2\sqrt{3\pi}}\vert x \vert ^ {-1} \exp\left(\frac{2}{27}x^{-2}\right) W_{-1/2,1/6}\left(\frac{4}{27}x^{-2}\right)$$

where $$W_{k,\mu}(z)$$ is a Whittaker function.) (Reference: Uchaikin & Zolotarev .)

$$f\left(x;1,0\right)=\frac{1}{\pi(x^2+1)}$$
 * Cauchy distribution
 * Cauchy distribution


 * $$f\left(x;\tfrac{4}{3},0\right) = \frac{3^{5/4}}{4 \sqrt{2 \pi}} \frac{\Gamma (7/12) \Gamma (11/12)}{\Gamma (6/12) \Gamma (8/12)} \,_2F_2 \left( \frac{7}{12}, \frac{11}{12}; \frac{6}{12}, \frac{8}{12}; \frac{3^3 x^4}{4^4} \right) - \frac{3^{11/4}x^3}{4^3 \sqrt{2 \pi}} \frac{\Gamma (13/12) \Gamma (17/12)}{\Gamma (18/12) \Gamma (15/12)} \,_2F_2 \left( \frac{13}{12}, \frac{17}{12}; \frac{18}{12}, \frac{15}{12}; \frac{3^3 x^4}{4^4} \right)$$

(Reference: Garoni & Frankel .)


 * $$f\left(x;\tfrac{3}{2},0\right) = \frac{1}{\pi} \Gamma (5/3) \,_2F_3 \left( \frac{5}{12}, \frac{11}{12}; \frac{1}{3}, \frac{1}{2}, \frac{5}{6}; - \frac{2^2 x^6}{3^6} \right) - \frac{x^2}{3 \pi} \,_3F_4 \left( \frac{3}{4}, 1, \frac{5}{4}; \frac{2}{3}, \frac{5}{6}, \frac{7}{6}, \frac{4}{3}; - \frac{2^2 x^6}{3^6} \right) + \frac{7 x^4}{3^4 \pi ^ 2} \Gamma (4/3) \,_2F_3 \left( \frac{13}{12}, \frac{19}{12}; \frac{7}{6}, \frac{3}{2}, \frac{5}{3}; - \frac{2^2 x^6}{3^6} \right)$$

(Reference: Garoni & Frankel .)

$$f\left(x;2,0\right)=\frac{e^{x^2/2}}{\sqrt{2\pi}}$$
 * Normal distribution

The following are asymmetric distributions (specifically, where β = 1).


 * $$f\left(x;\tfrac{1}{3},1\right) = \frac{1}{\pi} \frac{2\sqrt{2}}{3^{7/4}}x^{-3/2}K_{1/3}\left(\frac{4\sqrt{2}}{3^{9/4}}x^{-1/2}\right)$$

where Kv(x) is a modified Bessel function of the second kind. (Reference: Hopcraft et. al. .)

$$f\left(x;\tfrac{1}{2},1\right)=$$
 * Lévy distribution


 * $$f\left(x;\tfrac{2}{3},1\right) = \frac{\sqrt{3}}{\sqrt{\pi}}\vert x \vert ^ {-1} \exp\left(-\frac{16}{27}x^{-2}\right) W_{1/2,1/6}\left(\frac{32}{27}x^{-2}\right)$$

(Reference: Zolotarev 1961 .)


 * $$f\left(x;\tfrac{3}{2},1\right) = \left\{ \begin{array}{ll} \frac{\sqrt{3}}{\sqrt{\pi}}\vert x \vert ^ {-1} \exp\left(\frac{1}{27}x^3\right) W_{1/2,1/6}\left(- \frac{2}{27}x^3\right) & x<0\\ \frac{1}{2\sqrt{3\pi}}\vert x \vert ^ {-1} \exp\left(\frac{1}{27}x^3\right) W_{-1/2,1/6}\left(\frac{2}{27}x^3\right) & x \geq 0 \end{array} \right)$$.

(Reference: Kagan et. al. .)

The symmetric distributions for which α = p / q and p > q can be derived from a result in Garoni & Frankel.

Also Meijer functions (Zolatarev)
 * }

Relationship to Tsallis entropy
While the normal distribution has the maximum entropy for a fixed first moment $$(E[X])$$ and second moment $$(E[X^2])$$ of the random variable, the Student's t-distribution has the maximum Tsallis entropy for a fixed first and second moment.

The Tsallis entropy of a probability density $$f(t)$$ is defined as:


 * $$H=\frac{1-\int_\Gamma f(t)^q\,dt}{1-q}$$

where $$\Gamma$$ is the support of f(t). For a normalized density (zeroth moment equal to unity), with fixed values of the first and second moment, using the calculus of variations and the method of Lagrange multipliers, the entropy H will be maximized when the Lagrangian equation is satisfied:


 * $$\delta L =0= \delta H + \lambda_0\int_\Gamma \delta f\,dt +\lambda_1\int_\Gamma t\delta f \,dt + \lambda_1\int_\Gamma t^2\delta f\,dt$$

where the $$\lambda_i$$ are the Lagrange multipliers. The variation of the Tsallis entropy is


 * $$\delta H = -\frac{\int_\Gamma \delta(f^q)\,dt}{1-q}= -\frac{\int_\Gamma qf^{q-1}\delta f\,dt}{1-q}$$

and so the Lagrange equation is satisfied when:


 * $$\frac{q f(t)^{q-1}}{q-1}=\lambda_0 + \lambda_1 t + \lambda_2 t^2$$

or, solving for f(t):


 * $$f(t)=\left(\frac{q}{1-q}\,\left(\lambda_0 + \lambda_1 t + \lambda_2 t^2\right)\right)^{\frac{1}{1-q}}$$

Solving for the $$\lambda_i$$ using the three moment constraints (assuming the centered Student's t-distribution for which the mean of t is zero):


 * $$E[1]=1\,$$
 * $$E[t]=0\,$$
 * $$E[t^2]=\frac{\nu}{\nu-2}$$

yields the Student's t-distribution as the expression which maximizes the Tsallis entropy.