SOTAVerified

Dialogue Evaluation

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–97 of 97 papers

TitleStatusHype
U-NEED: A Fine-grained Dataset for User Needs-Centric E-commerce Conversational Recommendation—0
Pragmatically Appropriate Diversity for Dialogue Evaluation—0
Improving Open-Domain Dialogue Evaluation with a Causal Inference Model—0
PoE: a Panel of Experts for Generalized Automatic Dialogue Assessment—0
Joint Goal Segmentation and Goal Success Prediction on Multi-Domain Conversations—0
Dialogue Evaluation with Offline Reinforcement Learning—0
SelF-Eval: Self-supervised Fine-grained Dialogue EvaluationCode0
Explaining Dialogue Evaluation Metrics using Adversarial Behavioral Analysis—0
MME-CRS: Multi-Metric Evaluation Based on Correlation Re-Scaling for Evaluating Open-Domain Dialogue—0
AdaCoach: A Virtual Coach for Training Customer Service Agents—0
What is wrong with you?: Leveraging User Sentiment for Automatic Dialog EvaluationCode0
Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges—0
DEAM: Dialogue Coherence Evaluation using AMR-based Semantic ManipulationsCode0
Achieving Reliable Human Assessment of Open-Domain Dialogue SystemsCode0
FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows—0
Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents—0
MDD-Eval: Self-Training on Augmented Data for Multi-Domain Dialogue EvaluationCode0
User Response and Sentiment Prediction for Automatic Dialogue Evaluation—0
GCDF1: A Goal- and Context- Driven F-Score for Evaluating User ModelsCode0
Proxy Indicators for the Quality of Open-domain DialoguesCode0
Investigating the Impact of Pre-trained Language Models on Dialog Evaluation—0
Achieving Reliable Human Assessment of Open-Domain Dialogue Systems—0
A Human-machine Collaborative Framework for Evaluating Malevolence in DialoguesCode0
Enhancing the Open-Domain Dialogue Evaluation in Latent Space—0
Transformers for Headline Selection for Russian News ClustersCode0
Synthesizing Adversarial Negative Responses for Robust Response Ranking and EvaluationCode0
Improving Automated Evaluation of Open Domain Dialog via Diverse Reference AugmentationCode0
Generating Negative Samples by Manipulating Golden Responses for Unsupervised Learning of a Response Evaluation ModelCode0
DCH-2: A Parallel Customer-Helpdesk Dialogue Corpus with Distributions of Annotators' Labels—0
An Adversarially-Learned Turing Test for Dialog Generation ModelsCode0
WeChat AI & ICT's Submission for DSTC9 Interactive Dialogue Evaluation Track—0
Deconstruct to Reconstruct a Configurable Evaluation Metric for Open-Domain Dialogue SystemsCode0
Predicting Ratings of Real Dialogue Participants from Artificial Data and Ratings of Human Dialogue Observers—0
Learning the Human Judgment for the Automatic Evaluation of Chatbot—0
Treating Dialogue Quality Evaluation as an Anomaly Detection Problem—0
How to Evaluate the Next System: Automatic Dialogue Evaluation from the Perspective of Continual Learning—0
Predictive Engagement: An Efficient Metric For Automatic Evaluation of Open-Domain Dialogue SystemsCode0
Towards Best Experiment Design for Evaluating Dialogue System OutputCode0
ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons—0
Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple ReferencesCode0
Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog SystemsCode0
Better Automatic Evaluation of Open-Domain Dialogue Systems with Contextualized Embeddings—0
Evaluating Coherence in Dialogue Systems using EntailmentCode0
Re-evaluating ADEM: A Deeper Look at Scoring Dialogue Responses—0
One "Ruler" for All Languages: Multi-Lingual Dialogue Evaluation with Adversarial Multi-Task Learning—0
Towards an Automatic Turing Test: Learning to Evaluate Dialogue ResponsesCode0
Adversarial Learning for Neural Dialogue GenerationCode0
Show:102550
← PrevPage 2 of 2Next →
All datasetsUSR-TopicalChatUSR-PersonaChat

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MDD-EvalSpearman Correlation0.51—Unverified
2Lin-Reg (all)Spearman Correlation0.49—Unverified
3USRSpearman Correlation0.42—Unverified
4USR - DR (x = c)Spearman Correlation0.32—Unverified
5USR - MLMSpearman Correlation0.31—Unverified
6USR - DR (x = f)Spearman Correlation0.14—Unverified
#ModelMetricClaimedVerifiedStatus
1Lin-Reg (all)Spearman Correlation0.54—Unverified
2USR - DR (x = c)Spearman Correlation0.48—Unverified
3USRSpearman Correlation0.47—Unverified
4USR - MLMSpearman Correlation0.08—Unverified
5USR - DR (x = f)Spearman Correlation-0.05—Unverified