Authors
Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H Chi, Alex Beutel
Publication date
2019/1/27
Book
Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society
Pages
219-226
Description
In this paper, we study counterfactual fairness in text classification, which asks the question: How would the prediction change if the sensitive attribute referenced in the example were different? Toxicity classifiers demonstrate a counterfactual fairness issue by predicting that "Some people are gay" is toxic while "Some people are straight" is nontoxic. We offer a metric, counterfactual token fairness (CTF), for measuring this particular form of fairness in text classifiers, and describe its relationship with group fairness. Further, we offer three approaches, blindness, counterfactual augmentation, and counterfactual logit pairing (CLP), for optimizing counterfactual token fairness during training, bridging the robustness and fairness literature. Empirically, we find that blindness and CLP address counterfactual token fairness. The methods do not harm classifier performance, and have varying tradeoffs with group fairness. These …
Total citations
20192020202120222023202420251641506364545
Scholar articles
S Garg, V Perot, N Limtiaco, A Taly, EH Chi, A Beutel - Proceedings of the 2019 AAAI/ACM Conference on AI …, 2019
S Garg, V Perot, N Limtiaco, A Taly - Proceedings of the 2019 AAAI/ACM Conference on AI …