User:Devansh20/sandbox

Probability mass function
If $$X$$ is a random variable with Bernoulli distribution, then:


 * $$\Pr(X=1) = p $$ and $$ \Pr(X=0) = 1 - p=q.$$

The probability mass function $$f$$ of this distribution, over possible outcomes k, is


 * $$ f(k;p) = \begin{cases}

p & \text{if }k=1, \\ q = 1-p & \text {if } k = 0. \end{cases}$$
 * This can also be expressed as
 * $$f(k;p) = p^k (1-p)^{1-k} \quad \text{for } k\in\{0,1\}$$
 * or as
 * $$f(k;p)=pk+(1-p)(1-k) \quad \text{for } k\in\{0,1\}.$$
 * $$f(k;p)=pk+(1-p)(1-k) \quad \text{for } k\in\{0,1\}.$$

Cumulative distribution function
The cumulative distribution function is:



F(k;p)= \begin{cases} 0 & \text{for }k < 0 \\[8pt] 1-p & \text{for }0 \le k < 1 \\[8pt] 1 & \text{for }k \ge 1 \end{cases} $$

Expected value and variance
$$\operatorname{E}[X] =\sum_{i=1}^k \,p_iX_i$$

For a Bernoulli distributed random variable $$X$$, $$\Pr(X=1)=p$$ and $$\Pr(X=0)=q$$ we find

$$\operatorname{E}[X] = \Pr(X=1)\cdot 1 + \Pr(X=0)\cdot 0 = p \cdot 1 + q\cdot 0 = p.$$

The variance is given by :

$$\begin{align} \operatorname{Var}(X) &= \operatorname{E}\left[X^2 \right] - \operatorname{E}[X]^2 \end{align}$$

We first find


 * $$\operatorname{E}[X^2] = \Pr(X=1)\cdot 1^2 + \Pr(X=0)\cdot 0^2 = p \cdot 1^2 + q\cdot 0^2 = p$$

From this follows


 * $$\operatorname{Var}[X] = \operatorname{E}[X^2]-\operatorname{E}[X]^2 = p-p^2 = p(1-p) = pq$$

Estimation of parameter
The maximum likelihood estimator can be used to estimate the value of parameter $$p$$.

Let$$X = (X_1, X_2, . . ., X_n)$$ represent the outcomes of n independent Bernoulli trials, each with success probability $$p$$. The likelihood function for $$p$$ is given by

$$L(p;x)=\prod_{i=1}^np^{x_i}(1-p)^{1-x_i}$$

The estimator $$\widehat{p\,}$$ can be obtained by differentiating the log of $$L(p ; x)$$ with respect to $$p$$ and setting the derivative to zero. This leads to

$$\widehat{p\,} = \sum_{i=1}^n\frac{x_i}{n}$$

Therefore, the maximum likelihood estimator of $$p$$ is equal to the sample mean.



Calculation from the CDF
For a random variable defined on the domain $$(-\infin,\infin)$$, the variance can be calculated by first deriving the probability density function $$f_X(x)$$ from cumulative distribution function $$F(x)$$.

$$f_X(x) = {dF(x) \over dx}$$

Thereafter, the variance can be calculated using the expression

$$\operatorname{Var}(X) = \int^{+\infty}_{-\infty} x^2 f_X(x) \, dx - \Big(\int^{+\infty}_{-\infty} x f_X(x) \, dx\Big)^2.$$

For a non negative random variable $$X $$, the above expression simplifies and can be expressed in terms of the cumulative distribution function.

$$\operatorname{E}(X^2) = \int^{+\infty}_{0} x^2 f_X(x) \, dx $$

The above equation can be solved using integral by parts.

Let $$u=x^2 $$ and $$dv=f_X(x) $$

$$\operatorname{E}(X^2) = [x^2(F(x)-1)]|_0^\infty + \int^{+\infty}_{0} 2x(1-F(x)) \, dx =\int^{+\infty}_{0} 2x(1-F(x)) \, dx. $$

The first term in the above expression goes to zero when evaluated at the limits of integral.

A similar derivation can be done for the expected value,

$$\operatorname{E}(X) = \int^{+\infty}_{0} xf_X(x) \, dx= \int^{+\infty}_{0} \big(1-F(x)\big)\, dx   $$

The final expression for variance can be obtained by combining the above results.

$$Var(X) = E(X^2)-(E(X))^2=\int^{+\infty}_{0} 2x(1-F(x)) \, dx - \Big(\int^{+\infty}_{0} \big(1-F(x)\big)\, dx\Big)^2. $$