SOTAVerified|Agents Browse Leaderboard About

Bias Detection

Bias detection is the task of detecting and measuring racism, sexism and otherwise discriminatory behavior in a model (Source: https://stereoset.mit.edu/)

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 11–20 of 199 papers

Title	Date	Tasks	Status	Hype	Score
Debiased Visual Question Answering from Feature and Sample Perspectives	Dec 1, 2021	Bias DetectionQuestion Answering	CodeCode Available	1	5
Counterfactual Token Generation in Large Language Models	Sep 25, 2024	Bias Detectioncounterfactual	CodeCode Available	1	5
Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud	Sep 7, 2021	Bias DetectionBIG-bench Machine Learning	CodeCode Available	1	5
Learning to Split for Automatic Bias Detection	Apr 28, 2022	Bias Detectionimage-classification	CodeCode Available	1	5
Introducing MBIB -- the first Media Bias Identification Benchmark Task and Dataset Collection	Apr 25, 2023	Bias Detection	CodeCode Available	1	5
Benchmarking Llama2, Mistral, Gemma and GPT for Factuality, Toxicity, Bias and Propensity for Hallucinations	Apr 15, 2024	BenchmarkingBias Detection	CodeCode Available	1	5
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets	May 29, 2023	Bias DetectionCode Generation	CodeCode Available	1	5
Benchmarking Bias Mitigation Algorithms in Representation Learning through Fairness Metrics	Jun 8, 2021	Age And Gender ClassificationBenchmarking	CodeCode Available	1	5
BiasAsker: Measuring the Bias in Conversational AI System	May 21, 2023	Bias Detection	CodeCode Available	1	5
Neural Media Bias Detection Using Distant Supervision With BABE -- Bias Annotations By Experts	Sep 29, 2022	ArticlesBias Detection	CodeCode Available	1	5

Show:10 25 50

← PrevPage 2 of 20Next →

All datasets StereoSet rt-inod-bias ICAT LLM bias PlantVillage_8px Wiki Neutrality Corpus

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	GPT-2 (small)	ICAT Score	72.97	—	Unverified
2	XLNet (large)	ICAT Score	72.03	—	Unverified
3	GPT-2 (medium)	ICAT Score	71.73	—	Unverified
4	BERT (base)	ICAT Score	71.21	—	Unverified
5	GPT-2 (large)	ICAT Score	70.54	—	Unverified
6	BERT (large)	ICAT Score	69.89	—	Unverified
7	RoBERTa (base)	ICAT Score	67.5	—	Unverified
8	GAL 120B	ICAT Score	65.6	—	Unverified
9	XLNet (base)	ICAT Score	62.1	—	Unverified
10	GPT-3 (text-davinci-002)	ICAT Score	60.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GPT-4	Best-of	0.5	—	Unverified
2	Baseline	Best-of	0.41	—	Unverified
3	Gemma	Best-of	0.41	—	Unverified
4	Mistral	Best-of	0.36	—	Unverified
5	Llama2	Best-of	0.34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BAD	ICAT Score	23.44	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RandomForest_default_hyperparameters	Accuracy (%)	49	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RoBERTa+ALBERT	F1	70.4	—	Unverified