Bennett, Alpert and Goldstein's S

Bennett, Alpert & Goldstein’s S is a statistical measure of inter-rater agreement. It was created by Bennett et al. in 1954.

Rationale for use
Bennett et al. suggested adjusting inter-rater reliability to accommodate the percentage of rater agreement that might be expected by chance was a better measure than simple agreement between raters. They proposed an index which adjusted the proportion of rater agreement based on the number of categories employed.

Mathematical formulation
The formula for S is


 * $$ S = \frac{ Q P_a - 1 } { Q - 1 } $$

where Q is the number of categories and Pa is the proportion of agreement between raters.

The variance of S is


 * $$ \operatorname{Var}(S) = \left( \frac { Q } { Q - 1 } \right)^2 \frac { P_a ( P_a - 1 ) } { n - 1 } $$