User:Deltasun/sandbox

Individual Fairness criteria
An important distinction among fairness definitions is the one between group and individual notions. Roughly speaking, while group fairness criteria compare quantities at a group level, typically identified by sensitive attributes (e.g. gender, ethnicity, age, etc...), individual criteria compare individuals. In words, individual fairness follow the principle that "similar individuals should receive similar treatments".

There is a very intuitive approach to fairness, which usually goes under the name of Fairness Through Unawareness (FTU), or Blindness, that prescribe not to explicitly employ sensitive features $$A$$ when making (automated) decisions. This is effectively a notion of individual fairness, since two individuals differing only for the values of their sensitive attributes would receive the same outcome $$R$$.

However, in general, FTU is subject to several drawbacks, the main being that it does not take into account possible correlations between sensitive attributes and non-sensitive attributes employed in the decision-making process. For example, an agent with the (malignant) intention to discriminate on the basis of gender could introduce in the model a proxy variable for gender (i.e. a variable highly correlated with gender) and effectively using gender information while at the same time being compliant to the FTU prescription.

The problem of what variables correlated to sensitive ones are fairly employable by a model? in the decision-making process is a crucial one, and is relevant for group concept as well: independence metrics require a complete removal of sensitive information, while separation-based metrics allow for correlation, but only as far as the labeled target variable "justify" them.

The most general concept of individual fairness was introduced in the pioneer work by Dwork and collaborators in 2012 and can be thought of as a mathematical translation of the principle that the decision map taking features as input should be built such that it is able to "map similar individuals similarly", that is expressed as a Lipschitz condition on the model map. They call this approach Fairness Through Awareness (FTA), precisely as counterpoint to FTU, since they underline the importance of choosing the appropriate target-related distance metric in order to assess which individuals are similar in specific situations. Again, this problem is very related to the point raised above about what variables can be seen as "legitimate" in particular contexts.

Causality-based metrics
An entire branch of the academic research on fairness metrics is devoted to leverage causal models to assess bias in machine learning models. This approach is usually justified by the fact that the same observational distribution of data may hide different causal relationships among the variables at play, possibly with different interpretations of whether the outcome are affected by some form of bias or not.

Kusner et al. propose to employ counterfactuals, and define a decision-making process counterfactually fair if, for any individual, the outcome does not change in the counterfactual scenario where the sensitive attributes are changed. The mathematical formulation reads:

$$    P(\hat{R}_{A\leftarrow a}=1\ |\ A=a,X=x) = P(\hat{R}_{A\leftarrow b}=1\ |\ A=a,X=x),\quad\forall a,b; $$

that is: taken a random individual with sensitive attribute $$A=a$$ and other features $$X=x$$ and the same individual if she had $$A = b$$, they should have same chance of being accepted. The symbol $$\hat{R}_{A\leftarrow a}$$ represents the counterfactual random variable $$R$$ in the scenario where the sensitive attribute $$A$$ is fixed to $$A=a$$. The conditioning on $$A=a, X=x$$ means that this requirement is at the individual level, in that we are conditioning on all the variables identifying a single observation.