SOTAVerified

Bias Detection

Bias detection is the task of detecting and measuring racism, sexism and otherwise discriminatory behavior in a model (Source: https://stereoset.mit.edu/)

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 199 papers

TitleStatusHype
Galactica: A Large Language Model for ScienceCode4
Explainable AI in Spatial AnalysisCode2
Benchmarking Bias Mitigation Algorithms in Representation Learning through Fairness MetricsCode1
Neural Media Bias Detection Using Distant Supervision With BABE -- Bias Annotations By ExpertsCode1
BiasAsker: Measuring the Bias in Conversational AI SystemCode1
Benchmarking Llama2, Mistral, Gemma and GPT for Factuality, Toxicity, Bias and Propensity for HallucinationsCode1
MT-LENS: An all-in-one Toolkit for Better Machine Translation EvaluationCode1
Counterfactual Token Generation in Large Language ModelsCode1
BAD: BiAs Detection for Large Language Models in the context of candidate screeningCode1
New Job, New Gender? Measuring the Social Bias in Image Generation ModelsCode1
Towards explainable classifiers using the counterfactual approach -- global explanations for discovering bias in dataCode1
Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like BiasesCode1
Debiased Visual Question Answering from Feature and Sample PerspectivesCode1
Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the CloudCode1
Neural Media Bias Detection Using Distant Supervision With BABE - Bias Annotations By ExpertsCode1
Explainable AI for computational pathology identifies model limitations and tissue biomarkersCode1
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark DatasetsCode1
Learning to Split for Automatic Bias DetectionCode1
OpenBias: Open-set Bias Detection in Text-to-Image Generative ModelsCode1
The Hidden Language of Diffusion ModelsCode1
SynthesizRR: Generating Diverse Datasets with Retrieval AugmentationCode1
StereoSet: Measuring stereotypical bias in pretrained language modelsCode1
Introducing MBIB -- the first Media Bias Identification Benchmark Task and Dataset CollectionCode1
Exploring Visual Engagement Signals for Representation LearningCode1
Entropy-based Attention Regularization Frees Unintended Bias Mitigation from ListsCode1
A Review of the Challenges with Massive Web-mined Corpora Used in Large Language Models Pre-Training—0
BENN: Bias Estimation Using Deep Neural Network—0
Adding Instructions during Pretraining: Effective Way of Controlling Toxicity in Language Models—0
Beyond Explanation: A Case for Exploratory Text Visualizations of Non-Aggregated, Annotated Datasets—0
BiasAlert: A Plug-and-play Tool for Social Bias Detection in LLMs—0
BEADs: Bias Evaluation Across Domains—0
Any Large Language Model Can Be a Reliable Judge: Debiasing with a Reasoning-based Bias Detector—0
Constructive Interpretability with CoLabel: Corroborative Integration, Complementary Features, and Collaborative Learning—0
Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema—0
A Novel Method for News Article Event-Based Embedding—0
A Keyword Based Approach to Understanding the Overpenalization of Marginalized Groups by English Marginal Abuse Models on Twitter—0
Reinforcement Learning from Multi-role Debates as Feedback for Bias Mitigation in LLMs—0
Annotating and Analyzing Biased Sentences in News Articles using Crowdsourcing—0
Auditing Predictive Models for Intersectional Biases—0
Accurate Uncertainty Estimation and Decomposition in Ensemble Learning—0
Can we Debias Social Stereotypes in AI-Generated Images? Examining Text-to-Image Outputs and User Perceptions—0
Auditing Algorithmic Fairness in Machine Learning for Health with Severity-Based LOGAN—0
Can We Trust AI Agents? A Case Study of an LLM-Based Multi-Agent System for Ethical AI—0
Cascading Adversarial Bias from Injection to Distillation in Language Models—0
Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2021): Workshop and Shared Task Report—0
ChatGPT v.s. Media Bias: A Comparative Study of GPT-3.5 and Fine-tuned Language Models—0
Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers—0
Cognitive Bias Detection Using Advanced Prompt Engineering—0
Detecting Cross-Geographic Biases in Toxicity Modeling on Social Media—0
Auditing a Dutch Public Sector Risk Profiling Algorithm Using an Unsupervised Bias Detection Tool—0
Show:102550
← PrevPage 1 of 4Next →
All datasetsStereoSetrt-inod-biasICAT LLM biasPlantVillage_8pxWiki Neutrality Corpus

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1GPT-2 (small)ICAT Score72.97—Unverified
2XLNet (large)ICAT Score72.03—Unverified
3GPT-2 (medium)ICAT Score71.73—Unverified
4BERT (base)ICAT Score71.21—Unverified
5GPT-2 (large)ICAT Score70.54—Unverified
6BERT (large)ICAT Score69.89—Unverified
7RoBERTa (base)ICAT Score67.5—Unverified
8GAL 120BICAT Score65.6—Unverified
9XLNet (base)ICAT Score62.1—Unverified
10GPT-3 (text-davinci-002)ICAT Score60.8—Unverified
#ModelMetricClaimedVerifiedStatus
1GPT-4Best-of0.5—Unverified
2GemmaBest-of0.41—Unverified
3BaselineBest-of0.41—Unverified
4MistralBest-of0.36—Unverified
5Llama2Best-of0.34—Unverified
#ModelMetricClaimedVerifiedStatus
1BADICAT Score23.44—Unverified
#ModelMetricClaimedVerifiedStatus
1RandomForest_default_hyperparametersAccuracy (%)49—Unverified
#ModelMetricClaimedVerifiedStatus
1RoBERTa+ALBERTF170.4—Unverified