Bias Detection

Bias detection is the task of detecting and measuring racism, sexism and otherwise discriminatory behavior in a model (Source: https://stereoset.mit.edu/)

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 176–199 of 199 papers

Title	Date	Tasks	Status
Quantifying Gender Biases Towards Politicians on Reddit	Dec 22, 2021	Bias DetectionGender Bias Detection	CodeCode Available
IndiVec: An Exploration of Leveraging Large Language Models for Media Bias Detection with Fine-Grained Bias Indicators	Feb 1, 2024	Bias DetectionInstruction Following	CodeCode Available
Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions	Oct 3, 2024	Bias Detection	CodeCode Available
The BIAS Detection Framework: Bias Detection in Word Embeddings and Language Models for European Languages	Jul 26, 2024	Bias DetectionDiversity	CodeCode Available
DeNetDM: Debiasing by Network Depth Modulation	Mar 28, 2024	Bias DetectionMitigating Contextual Bias	CodeCode Available
A Domain-adaptive Pre-training Approach for Language Bias Detection in News	May 22, 2022	Bias DetectionDecision Making	CodeCode Available
Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Models	Sep 16, 2023	Age/Bias-conflictingBias Detection	CodeCode Available
Robust Bias Detection in MLMs and its Application to Human Trait Ratings	Feb 21, 2025	Bias Detection	CodeCode Available
Language-Agnostic Bias Detection in Language Models with Bias Probing	May 22, 2023	Bias Detection	CodeCode Available
RuBia: A Russian Language Bias Detection Dataset	Mar 26, 2024	Bias DetectionDiagnostic	CodeCode Available
Towards Detection of Subjective Bias using Contextualized Word Embeddings	Feb 16, 2020	Bias DetectionPropaganda detection	CodeCode Available
Corpora Evaluation and System Bias Detection in Multi-document Summarization	Oct 5, 2020	Bias DetectionDocument Summarization	CodeCode Available
Second Order WinoBias (SoWinoBias) Test Set for Latent Gender Bias Detection in Coreference Resolution	Sep 28, 2021	Bias Detectioncoreference-resolution	CodeCode Available
LOGAN: Local Group Bias Detection by Clustering	Oct 6, 2020	Bias DetectionBIG-bench Machine Learning	CodeCode Available
LUCID-GAN: Conditional Generative Models to Locate Unfairness	Jul 28, 2023	Bias DetectionFairness	CodeCode Available
The Media Bias Taxonomy: A Systematic Literature Review on the Forms and Automated Detection of Media Bias	Dec 26, 2023	Bias DetectionSystematic Literature Review	CodeCode Available
debiaSAE: Benchmarking and Mitigating Vision-Language Model Bias	Oct 17, 2024	BenchmarkingBias Detection	CodeCode Available
Uncovering bias in the PlantVillage dataset	Jun 9, 2022	Bias DetectionDeep Learning	CodeCode Available
Measuring Gender Bias in Word Embeddings across Domains and Discovering New Gender Bias Word Categories	Aug 1, 2019	Bias DetectionClustering	CodeCode Available
The Promises and Pitfalls of LLM Annotations in Dataset Labeling: a Case Study on Media Bias Detection	Nov 17, 2024	Bias DetectionHate Speech Detection	CodeCode Available
Mind Your Bias: A Critical Review of Bias Detection Methods for Contextual Language Models	Nov 15, 2022	Bias DetectionWord Embeddings	CodeCode Available
Mitigating Bias in Queer Representation within Large Language Models: A Collaborative Agent Approach	Nov 12, 2024	Bias DetectionFairness	CodeCode Available
Social Bias in Large Language Models For Bangla: An Empirical Study on Gender and Religious Bias	Jul 3, 2024	BenchmarkingBias Detection	CodeCode Available
TinyEmo: Scaling down Emotional Reasoning via Metric Projection	Oct 9, 2024	Bias DetectionClassification	CodeCode Available

Show:10 25 50

← PrevPage 8 of 8Next →

All datasets StereoSet rt-inod-bias ICAT LLM bias PlantVillage_8px Wiki Neutrality Corpus

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	GPT-2 (small)	ICAT Score	72.97	—	Unverified
2	XLNet (large)	ICAT Score	72.03	—	Unverified
3	GPT-2 (medium)	ICAT Score	71.73	—	Unverified
4	BERT (base)	ICAT Score	71.21	—	Unverified
5	GPT-2 (large)	ICAT Score	70.54	—	Unverified
6	BERT (large)	ICAT Score	69.89	—	Unverified
7	RoBERTa (base)	ICAT Score	67.5	—	Unverified
8	GAL 120B	ICAT Score	65.6	—	Unverified
9	XLNet (base)	ICAT Score	62.1	—	Unverified
10	GPT-3 (text-davinci-002)	ICAT Score	60.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GPT-4	Best-of	0.5	—	Unverified
2	Baseline	Best-of	0.41	—	Unverified
3	Gemma	Best-of	0.41	—	Unverified
4	Mistral	Best-of	0.36	—	Unverified
5	Llama2	Best-of	0.34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BAD	ICAT Score	23.44	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RandomForest_default_hyperparameters	Accuracy (%)	49	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RoBERTa+ALBERT	F1	70.4	—	Unverified