Algorithmic Fairness Bias Metric Calculator
Interpreting Fairness Metrics
Machine learning systems increasingly influence credit scoring, employment screening, medical diagnostics, and criminal justice. As decisions intersect with sensitive categories like gender or ethnicity, stakeholders examine outcome disparities. Two accessible measures are demographic parity and equal opportunity. They focus on selection rates and true positive rates respectively, allowing you to quantify gaps in model behavior.
Demographic parity compares the share of positive predictions between groups. If and are the true and false positives for a group, the positive prediction rate is calculated via
The demographic parity difference is simply , while the ratio highlights whether Group B receives at least 80% of Group A's positive decisions (the classic four-fifths rule).
Equal opportunity evaluates whether qualified individuals are treated similarly. It compares true positive rates where . The calculator reports the difference between group TPRs and calls out when denominators are zero so you know to revisit data coverage.
Example Confusion Matrices
The table below illustrates how raw counts translate into metrics for a hypothetical hiring screener.
| Outcome | Group A | Group B |
|---|---|---|
| True positives | 50 | 40 |
| False positives | 10 | 15 |
| False negatives | 20 | 30 |
| True negatives | 120 | 100 |
Plugging the values into the formulas yields a demographic parity difference near two percentage points, an 0.94 positive-rate ratio, and an equal opportunity difference of roughly fourteen percentage points. These numbers indicate balanced selection volumes but a clear disparity in recognizing qualified Group B applicants.
Keep Bias Conversations Grounded
Use this tool as a first-pass diagnostic, then explore related resources including the algorithmic fairness metrics calculator, AI ethics compliance cost planner, and the meeting rotation fairness tool to connect quantitative audits with broader governance practices.