User:AlonDaks/Disrepancy Ratio

Discrepancy ratio (Δ) is a statistic used to measure machine learning model performance in the presence of noisy ground truth labels. It measures performance by comparing model-to-human disagreement relative to human-to-human disagreement. Δ is especially useful for determining whether a model can safely be substituted for a human at tasks where humans are unlikely to perfectly agree with one another given the same sample.

Definition
A defition

Properties

 * 1) Dimensionless: Δ is a dimensionless quantity whose meaning is invariant across different task categories (regression, classification, segmentation, etc).