Bias Detection

Bias detection is the task of detecting and measuring racism, sexism and otherwise discriminatory behavior in a model (Source: https://stereoset.mit.edu/)

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 101–125 of 199 papers

Title	Date	Tasks	Status	Hype
Trade-Offs Between Fairness and Privacy in Language Modeling	May 24, 2023	Bias DetectionFairness	CodeCode Available	0
Language-Agnostic Bias Detection in Language Models with Bias Probing	May 22, 2023	Bias Detection	CodeCode Available	0
BiasAsker: Measuring the Bias in Conversational AI System	May 21, 2023	Bias Detection	CodeCode Available	1
BAD: BiAs Detection for Large Language Models in the context of candidate screening	May 17, 2023	Bias DetectionFairness	CodeCode Available	1
Introducing MBIB -- the first Media Bias Identification Benchmark Task and Dataset Collection	Apr 25, 2023	Bias Detection	CodeCode Available	1
Bipol: A Novel Multi-Axes Bias Evaluation Metric with Explainability for NLP	Apr 8, 2023	Bias DetectionSentence	CodeCode Available	0
Disentangling Structure and Style: Political Bias Detection in News by Inducing Document Hierarchy	Apr 5, 2023	ArticlesBias Detection	CodeCode Available	0
Adding Instructions during Pretraining: Effective Way of Controlling Toxicity in Language Models	Feb 14, 2023	Bias DetectionData Augmentation	—Unverified	0
Bipol: Multi-axes Evaluation of Bias with Explainability in Benchmark Datasets	Jan 28, 2023	Bias DetectionDiagnostic	CodeCode Available	0
Auditing Algorithmic Fairness in Machine Learning for Health with Severity-Based LOGAN	Nov 16, 2022	Bias DetectionClustering	—Unverified	0
Galactica: A Large Language Model for Science	Nov 16, 2022	AnachronismsBias Detection	CodeCode Available	4
Mind Your Bias: A Critical Review of Bias Detection Methods for Contextual Language Models	Nov 15, 2022	Bias DetectionWord Embeddings	CodeCode Available	0
Exploiting Transformer-based Multitask Learning for the Detection of Media Bias in News Articles	Nov 7, 2022	ArticlesBias Detection	—Unverified	0
A Keyword Based Approach to Understanding the Overpenalization of Marginalized Groups by English Marginal Abuse Models on Twitter	Oct 7, 2022	Bias DetectionFairness	—Unverified	0
Neural Media Bias Detection Using Distant Supervision With BABE -- Bias Annotations By Experts	Sep 29, 2022	ArticlesBias Detection	CodeCode Available	1
Efficient Gender Debiasing of Pre-trained Indic Language Models	Sep 8, 2022	Bias DetectionCultural Vocal Bursts Intensity Prediction	—Unverified	0
A Unified Comparison of User Modeling Techniques for Predicting Data Interaction and Detecting Exploration Bias	Aug 9, 2022	Bias DetectionData Interaction	CodeCode Available	0
Robots Enact Malignant Stereotypes	Jul 23, 2022	Bias DetectionGender Bias Detection	—Unverified	0
MRCLens: an MRC Dataset Bias Detection Toolkit	Jul 18, 2022	Bias DetectionMachine Reading Comprehension	—Unverified	0
A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America	Jul 14, 2022	Bias DetectionDecision Making	CodeCode Available	0
Incorporating Subjectivity into Gendered Ambiguous Pronoun (GAP) Resolution using Style Transfer	Jul 1, 2022	Bias Detectioncoreference-resolution	—Unverified	0
HeteroCorpus: A Corpus for Heteronormative Language Detection	Jul 1, 2022	Bias DetectionGender Bias Detection	CodeCode Available	0
Personalized Detection of Cognitive Biases in Actions of Users from Their Logs: Anchoring and Recency Biases	Jun 30, 2022	Bias DetectionEthics	—Unverified	0
Towards WinoQueer: Developing a Benchmark for Anti-Queer Bias in Large Language Models	Jun 23, 2022	Bias Detection	—Unverified	0
Uncovering bias in the PlantVillage dataset	Jun 9, 2022	Bias DetectionDeep Learning	CodeCode Available	0

Show:10 25 50

← PrevPage 5 of 8Next →

All datasets StereoSet rt-inod-bias ICAT LLM bias PlantVillage_8px Wiki Neutrality Corpus

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	GPT-2 (small)	ICAT Score	72.97	—	Unverified
2	XLNet (large)	ICAT Score	72.03	—	Unverified
3	GPT-2 (medium)	ICAT Score	71.73	—	Unverified
4	BERT (base)	ICAT Score	71.21	—	Unverified
5	GPT-2 (large)	ICAT Score	70.54	—	Unverified
6	BERT (large)	ICAT Score	69.89	—	Unverified
7	RoBERTa (base)	ICAT Score	67.5	—	Unverified
8	GAL 120B	ICAT Score	65.6	—	Unverified
9	XLNet (base)	ICAT Score	62.1	—	Unverified
10	GPT-3 (text-davinci-002)	ICAT Score	60.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GPT-4	Best-of	0.5	—	Unverified
2	Baseline	Best-of	0.41	—	Unverified
3	Gemma	Best-of	0.41	—	Unverified
4	Mistral	Best-of	0.36	—	Unverified
5	Llama2	Best-of	0.34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BAD	ICAT Score	23.44	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RandomForest_default_hyperparameters	Accuracy (%)	49	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RoBERTa+ALBERT	F1	70.4	—	Unverified