SOTAVerified

Instruction Following

Instruction following is the basic task of the model. This task is dedicated to evaluating the ability of the large model to follow human instructions. It is hoped that the model can generate controllable and safe answers.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1051–1100 of 1135 papers

TitleStatusHype
Cuckoo: An IE Free Rider Hatched by Massive Nutrition in LLM's NestCode0
Alignment-based compositional semantics for instruction followingCode0
Iterative Label Refinement Matters More than Preference Optimization under Weak SupervisionCode0
From Loops to Oops: Fallback Behaviors of Language Models Under UncertaintyCode0
On the Loss of Context-awareness in General Instruction Fine-tuningCode0
IWISDM: Assessing instruction following in multimodal models at scaleCode0
CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing ConstraintsCode0
Cross-lingual Transfer of Reward Models in Multilingual AlignmentCode0
Mapping Instructions to Actions in 3D Environments with Visual Goal PredictionCode0
Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation PredictionCode0
From MTEB to MTOB: Retrieval-Augmented Classification for Descriptive GrammarsCode0
RefuteBench: Evaluating Refuting Instruction-Following for Large Language ModelsCode0
Judging the Judges: Can Large Vision-Language Models Fairly Evaluate Chart Comprehension and Reasoning?Code0
Dancing in Chains: Reconciling Instruction Following and Faithfulness in Language ModelsCode0
Biomedical Visual Instruction Tuning with Clinician Preference AlignmentCode0
A Course Correction in Steerability Evaluation: Revealing Miscalibration and Side Effects in LLMsCode0
MDCure: A Scalable Pipeline for Multi-Document Instruction-FollowingCode0
ReIFE: Re-evaluating Instruction-Following EvaluationCode0
Re-Imagining Multimodal Instruction Tuning: A Representation ViewCode0
Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to GiantCode0
SOTOPIA-Ω: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social AgentsCode0
WangchanLion and WangchanX MRC EvalCode0
The Impact of Demonstrations on Multilingual In-Context Learning: A Multidimensional AnalysisCode0
X-Instruction: Aligning Language Model in Low-resource Languages with Self-curated Cross-lingual InstructionsCode0
Optimal Transport-Based Token Weighting scheme for Enhanced Preference OptimizationCode0
LACMA: Language-Aligning Contrastive Learning with Meta-Actions for Embodied Instruction FollowingCode0
Chasing Ghosts: Instruction Following as Bayesian State TrackingCode0
Repairs in a Block World: A New Benchmark for Handling User Corrections with Multi-Modal Language ModelsCode0
Evaluating the Instruction-Following Robustness of Large Language Models to Prompt InjectionCode0
Language as an Abstraction for Hierarchical Deep Reinforcement LearningCode0
Language-Conditioned Change-point Detection to Identify Sub-Tasks in Robotics DomainsCode0
URL: Universal Referential Knowledge Linking via Task-instructed Representation CompressionCode0
A Reminder of its Brittleness: Language Reward Shaping May Hinder Learning for Instruction Following AgentsCode0
Grade Score: Quantifying LLM Performance in Option SelectionCode0
Order Matters: Investigate the Position Bias in Multi-constraint Instruction FollowingCode0
Don't Copy the Teacher: Data and Model Challenges in Embodied DialogueCode0
Beyond Content Relevance: Evaluating Instruction Following in Retrieval ModelsCode0
CASTILLO: Characterizing Response Length Distributions of Large Language ModelsCode0
HREF: Human Response-Guided Evaluation of Instruction Following in Language ModelsCode0
MIMO: A Medical Vision Language Model with Visual Referring Multimodal Input and Pixel Grounding Multimodal OutputCode0
Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual SettingsCode0
CORDIAL: Can Multimodal Large Language Models Effectively Understand Coherence Relationships?Code0
From Supervised to Generative: A Novel Paradigm for Tabular Deep Learning with Large Language ModelsCode0
FMDLlama: Financial Misinformation Detection based on Large Language ModelsCode0
X-Shot: A Unified System to Handle Frequent, Few-shot and Zero-shot Learning Simultaneously in ClassificationCode0
Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation LearningCode0
Mitigating the Bias of Large Language Model EvaluationCode0
Synthetic Programming Elicitation for Text-to-Code in Very Low-Resource Programming and Formal LanguagesCode0
Self-Judge: Selective Instruction Following with Alignment Self-EvaluationCode0
Learning by Correction: Efficient Tuning Task for Zero-Shot Generative Vision-Language ReasoningCode0
Show:102550
← PrevPage 22 of 23Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1AutoIF (Llama3 70B)Inst-level loose-accuracy90.4—Unverified
2AutoIF (Qwen2 72B)Inst-level loose-accuracy88—Unverified
3GPT-4Inst-level loose-accuracy85.37—Unverified
4PaLM 2 SInst-level loose-accuracy59.11—Unverified