Bias Detection

Bias detection is the task of detecting and measuring racism, sexism and otherwise discriminatory behavior in a model (Source: https://stereoset.mit.edu/)

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 26–50 of 199 papers

Title	Date	Tasks	Status	Score
A Study of Nationality Bias in Names and Perplexity using Off-the-Shelf Affect-related Tweet Classifiers	Jul 1, 2024	Bias Detectioncounterfactual	CodeCode Available	5
Language-Agnostic Bias Detection in Language Models with Bias Probing	May 22, 2023	Bias Detection	CodeCode Available	5
How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contexts	May 22, 2022	Bias DetectionMachine Translation	CodeCode Available	5
IFBiD: Inference-Free Bias Detection	Sep 9, 2021	Bias Detection	CodeCode Available	5
Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Models	Sep 16, 2023	Age/Bias-conflictingBias Detection	CodeCode Available	5
HeteroCorpus: A Corpus for Heteronormative Language Detection	Jul 1, 2022	Bias DetectionGender Bias Detection	CodeCode Available	5
GUS-Net: Social Bias Classification in Text with Generalizations, Unfairness, and Stereotypes	Oct 10, 2024	Bias Detectiontoken-classification	CodeCode Available	5
How Neural Networks Organize Concepts: Introducing Concept Trajectory Analysis for Deep Learning Interpretability	Jun 1, 2025	Bias Detection	CodeCode Available	5
IndiVec: An Exploration of Leveraging Large Language Models for Media Bias Detection with Fine-Grained Bias Indicators	Feb 1, 2024	Bias DetectionInstruction Following	CodeCode Available	5
LOGAN: Local Group Bias Detection by Clustering	Oct 6, 2020	Bias DetectionBIG-bench Machine Learning	CodeCode Available	5
Fine-grained Classification of Political Bias in German News: A Data Set and Initial Experiments	Aug 1, 2021	ArticlesBias Detection	CodeCode Available	5
Fair is Better than Sensational:Man is to Doctor as Woman is to Doctor	May 23, 2019	Bias DetectionDiagnostic	CodeCode Available	5
Forward Composition Propagation for Explainable Neural Reasoning	Dec 23, 2021	Bias DetectionFairness	CodeCode Available	5
Don’t Discard All the Biased Instances: Investigating a Core Assumption in Dataset Bias Mitigation Techniques	Nov 1, 2021	AllBias Detection	CodeCode Available	5
Bipol: Multi-axes Evaluation of Bias with Explainability in Benchmark Datasets	Jan 28, 2023	Bias DetectionDiagnostic	CodeCode Available	5
Bipol: A Novel Multi-Axes Bias Evaluation Metric with Explainability for NLP	Apr 8, 2023	Bias DetectionSentence	CodeCode Available	5
A Domain-adaptive Pre-training Approach for Language Bias Detection in News	May 22, 2022	Bias DetectionDecision Making	CodeCode Available	5
Don't Discard All the Biased Instances: Investigating a Core Assumption in Dataset Bias Mitigation Techniques	Sep 1, 2021	AllBias Detection	CodeCode Available	5
Detection of Puffery on the English Wikipedia	Nov 1, 2021	Bias DetectionInformation Retrieval	CodeCode Available	5
A Unified Comparison of User Modeling Techniques for Predicting Data Interaction and Detecting Exploration Bias	Aug 9, 2022	Bias DetectionData Interaction	CodeCode Available	5
fairmodels: A Flexible Tool For Bias Detection, Visualization, And Mitigation	Apr 1, 2021	Bias DetectionFairness	CodeCode Available	5
A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America	Jul 14, 2022	Bias DetectionDecision Making	CodeCode Available	5
GradBias: Unveiling Word Influence on Bias in Text-to-Image Generative Models	Aug 29, 2024	Bias DetectionFairness	CodeCode Available	5
Automated Dependence Plots	Dec 2, 2019	Bias DetectionModel Selection	CodeCode Available	5
Detecting Media Bias in News Articles using Gaussian Bias Distributions	Oct 20, 2020	ArticlesBias Detection	CodeCode Available	5

Show:10 25 50

← PrevPage 2 of 8Next →

All datasets StereoSet rt-inod-bias ICAT LLM bias PlantVillage_8px Wiki Neutrality Corpus

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	GPT-2 (small)	ICAT Score	72.97	—	Unverified
2	XLNet (large)	ICAT Score	72.03	—	Unverified
3	GPT-2 (medium)	ICAT Score	71.73	—	Unverified
4	BERT (base)	ICAT Score	71.21	—	Unverified
5	GPT-2 (large)	ICAT Score	70.54	—	Unverified
6	BERT (large)	ICAT Score	69.89	—	Unverified
7	RoBERTa (base)	ICAT Score	67.5	—	Unverified
8	GAL 120B	ICAT Score	65.6	—	Unverified
9	XLNet (base)	ICAT Score	62.1	—	Unverified
10	GPT-3 (text-davinci-002)	ICAT Score	60.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GPT-4	Best-of	0.5	—	Unverified
2	Gemma	Best-of	0.41	—	Unverified
3	Baseline	Best-of	0.41	—	Unverified
4	Mistral	Best-of	0.36	—	Unverified
5	Llama2	Best-of	0.34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BAD	ICAT Score	23.44	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RandomForest_default_hyperparameters	Accuracy (%)	49	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RoBERTa+ALBERT	F1	70.4	—	Unverified