Hand's paradox

In statistics, Hand's paradox arises from ambiguity when comparing two treatments. It shows that a comparison of the effects of the treatments applied to two independent groups that can contradict a comparison between the effects of both treatments applied to a single group.

Paradox
Comparisons of two treatments often involve comparing the responses of a random sample of patients receiving one treatment with an independent random sample receiving the other. One commonly used measure of the difference is then the probability that a randomly chosen member of one group will have a higher score than a randomly chosen member of the other group. However, in many situations, interest really lies on which of the two treatments will give a randomly chosen patient the greater probability of doing better. These two measures, a comparison between two randomly chosen patients, one from each group, and a comparison of treatment effects on a randomly chosen patient, can lead to different conclusions.

This has been called Hand's paradox, and appears to have first been described by David J. Hand.

Example 1
Label the two treatments A and B and suppose that:

Patient 1 would have responded values 2 and 3 to A and B respectively. Patient 2 would have responded values 4 and 5 to A and B respectively. Patient 3 would have responded values 6 and 1 to A and B respectively.

Then the probability that the response to A of a randomly chosen patient is greater than the response to B of a randomly chosen patient is 6/9 = 2/3. But the probability that a randomly chosen patient will have a greater response to A than B is 1/3. Thus a simple comparison of two independent groups may suggest that patients have a higher probability of doing better under A, whereas in fact patients have a higher probability of doing better under B.

Example 2
Suppose we have two random variables, $$x_A \sim N(1,1)$$ and $$x_B \sim N(0,1)$$, corresponding to the effects of two treatments. If we assume that $$x_A$$ and $$x_B$$ are independent, then $$ \Pr(x_A > x_B) = 0.76 $$, suggesting that A is more likely to benefit a patient than B. In contrast, the joint distribution which minimizes $$Pr(x_A>x_B )$$ leads to $$Pr(x_A>x_B )\ge 0.38$$. This means that it is possible that in up to 62% of cases treatment B is better than treatment A.