User:SamuelTheGhost/Reversion towards the Mean

Universal definition
Let X, Y be random variables with any joint distribution (discrete or continuous). Reversion towards the Mean is the property defined in the following theorem. Assume means exists and that X and Y have identical marginal distributions. Then for all c in the range of the distribution, so that
 * $$ P[X \ge c] \ne 0 \,,$$

we have that
 * $$E[ Y| X \ge c ] \le E[ X| X \ge c ] \,,$$

with the reverse inequality holding for all $$ X \le c \,\! .$$

Proof
First we look at some probabilities. By elementary laws:
 * $$P[X \ge c] = P[X \ge c \land Y \ge c] + P[X \ge c \land Y < c] $$ and
 * $$P[Y \ge c] = P[X \ge c \land Y \ge c] + P[X < c \land Y \ge c] $$

But the marginal distributions are equal, which implies
 * $$P[X \ge c] = P[Y \ge c] $$

So taking these three equalities together we get
 * $$P[X \ge c \land Y < c] = P[X < c \land Y \ge c] $$

Going on the conditional probabilities we infer that
 * $$P[Y < c | X \ge c ] = \frac {P[X \ge c \land Y < c]}{P[X \ge c]} = \frac {P[X < c \land Y \ge c]}{P[Y \ge c]} = P[X < c | Y \ge c ]$$

Looking now at expected values we have
 * $$E[Y | X \ge c] =$$
 * $$P[Y \ge c | X \ge c]\, \times\, E[Y | X \ge c\, \land\, Y \ge c] \;+\; P[ Y < c | X \ge c]\, \times\, E[Y | X \ge c\, \land\, Y < c]$$

But of course
 * $$E[Y | X \ge c\, \land\, Y < c] < c$$, so
 * $$E[Y | X \ge c] \le $$
 * $$P[Y \ge c | X \ge c]\, \times\, E[Y | X \ge c\, \land\, Y \ge c] \;+\; P[ Y < c | X \ge c]\, \times\, c$$

Similarly we have
 * $$E[Y | Y \ge c] =$$
 * $$P[X \ge c | Y \ge c]\, \times\, E[Y | X \ge c\, \land\, Y \ge c] \;+\; P[ X < c | Y \ge c]\, \times\, E[Y | X < c\, \land\, Y \ge c]$$

and again of course
 * $$E[Y | X < c\, \land\, Y \ge c] \ge c$$, so
 * $$E[Y | Y \ge c] \ge $$
 * $$P[X \ge c | Y \ge c]\, \times\, E[Y | X \ge c\, \land\, Y \ge c] \;+\; P[ X < c | Y \ge c]\, \times\, c$$

Putting these together we have
 * $$E[Y | X \ge c] \le E[Y | Y \ge c] $$

and, since the marginal distributions are equal, we also have
 * $$E[X | X \ge c] = E[Y | Y \ge c] $$, so
 * $$E[Y | X \ge c] \le E[X | X \ge c] $$

which concludes the proof.