SOTAVerified

Logical Reasoning

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–100 of 747 papers

TitleStatusHype
HF4Rec: Human-Like Feedback-Driven Optimization Framework for Explainable Recommendation—0
Context-Awareness and Interpretability of Rare Occurrences for Discovery and Formalization of Critical Failure Modes—0
LogicTree: Structured Proof Exploration for Coherent and Rigorous Logical Reasoning with Large Language Models—0
Multi-Stage Retrieval for Operational Technology Cybersecurity Compliance Using Large Language Models: A Railway Casestudy—0
LAD-Reasoner: Tiny Multimodal Models are Good Reasoners for Logical Anomaly Detection—0
Enhancing the Geometric Problem-Solving Ability of Multimodal LLMs via Symbolic-Neural IntegrationCode1
d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning—0
PuzzleBench: A Fully Dynamic Evaluation Framework for Large Multimodal Models on Puzzle Solving—0
MediSee: Reasoning-based Pixel-level Perception in Medical Images—0
VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledge—0
Socrates or Smartypants: Testing Logic Reasoning Capabilities of Large Language Models with Logic Programming-based Test OraclesCode0
MovSAM: A Single-image Moving Object Segmentation Framework Based on Deep ThinkingCode0
Alice: Proactive Learning with Teacher's Demonstrations for Weak-to-Strong GeneralizationCode1
Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification—0
Provable Failure of Language Models in Learning Majority Boolean Logic via Gradient Descent—0
Have Large Language Models Learned to Reason? A Characterization via 3-SAT Phase Transition—0
Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual EditingCode2
Adaptive Rectification Sampling for Test-Time Compute ScalingCode0
Exploring the Effect of Reinforcement Learning on Video Understanding: Insights from SEED-Bench-R1Code2
VGRP-Bench: Visual Grid Reasoning Puzzle Benchmark for Large Vision-Language Models—0
QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?Code1
Negation: A Pink Elephant in the Large Language Models' Room?—0
ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning—0
Rosetta-PL: Propositional Logic as a Benchmark for Large Language Model Reasoning—0
A Study on Neuro-Symbolic Artificial Intelligence: Healthcare Perspectives—0
(G)I-DLE: Generative Inference via Distribution-preserving Logit Exclusion with KL Divergence Minimization for Constrained Decoding—0
Enhancing Retrieval Systems with Inference-Time Logical Reasoning—0
MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic WorkflowCode2
LaMOuR: Leveraging Language Models for Out-of-Distribution Recovery in Reinforcement Learning—0
From Chaos to Order: The Atomic Reasoner Framework for Fine-grained Reasoning in Large Language Models—0
Bridging Technology and Humanities: Evaluating the Impact of Large Language Models on Social Sciences Research with DeepSeek-R1—0
Measuring AI Ability to Complete Long TasksCode3
Efficient but Vulnerable: Benchmarking and Defending LLM Batch Prompting Attack—0
3DAxisPrompt: Promoting the 3D Grounding and Reasoning in GPT-4o—0
Reasoning is All You Need for Video Generalization: A Counterfactual Benchmark with Sub-question Evaluation—0
Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models—0
LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLCode4
Towards Superior Quantization Accuracy: A Layer-sensitive Approach—0
SCoRE: Benchmarking Long-Chain Reasoning in Commonsense ScenariosCode0
The Society of HiveMind: Multi-Agent Optimization of Foundation Model Swarms to Unlock the Potential of Collective Intelligence—0
HelpSteer3: Human-Annotated Feedback and Edit Data to Empower Inference-Time Scaling in Open-Ended General-Domain Tasks—0
DB-Explore: Automated Database Exploration and Instruction Synthesis for Text-to-SQL—0
Three tiers of computation in transformers and in brain architecturesCode0
Psy-Insight: Explainable Multi-turn Bilingual Dataset for Mental Health Counseling—0
DeLTa: A Decoding Strategy based on Logit Trajectory Prediction Improves Factuality and Reasoning AbilityCode0
KGCompiler: Deep Learning Compilation Optimization for Knowledge Graph Complex Logical Query AnsweringCode0
HoT: Highlighted Chain of Thought for Referencing Supporting Facts from Inputs—0
Order Doesn't Matter, But Reasoning Does: Training LLMs with Order-Centric Augmentation—0
Nexus: A Lightweight and Scalable Multi-Agent Framework for Complex Tasks AutomationCode2
TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model ReasoningCode0
Show:102550
← PrevPage 2 of 15Next →
All datasetsLingOlyBIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTreeWinograd AutomaticBIG-bench (Logical Fallacy Detection)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Claude OpusDelta_NoContext28.8—Unverified
2GPT-4oDelta_NoContext25.1—Unverified
3Gemini 1.5 ProDelta_NoContext23.4—Unverified
4GPT-4Delta_NoContext21.5—Unverified
5Command R+Delta_NoContext11.6—Unverified
6GPT-3.5Delta_NoContext11.2—Unverified
7Mixtral 8x7BDelta_NoContext6.4—Unverified
8Llama 3 8BDelta_NoContext4.9—Unverified
9Llama 3 70BDelta_NoContext2.9—Unverified
10Gemma 7BDelta_NoContext2.2—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, Direct)Accuracy64.8—Unverified
2PaLM 2 (few-shot, k=3, CoT)Accuracy57.2—Unverified
3OPT 66B (few-shot, k=3)Accuracy54—Unverified
4PaLM 540B (few-shot, k=3)Accuracy53.6—Unverified
5GPT-NeoX 20B (few-shot, k=3)Accuracy52.8—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy52.8—Unverified
7Chinchilla-70B (few-shot, k=5)Accuracy52.1—Unverified
8Bloomberg GPT 50B (few-shot, k=3)Accuracy50.8—Unverified
9Gopher-280B (few-shot, k=5)Accuracy50.7—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy84.9—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy65.8—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy48.7—Unverified
4PaLM 540B (few-shot, k=3)Accuracy44.5—Unverified
5Gopher-280B (few-shot, k=5)Accuracy40.6—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy40.41—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy37.67—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy33.56—Unverified
9OPT 66B (few-shot, k=3)Accuracy28.08—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy91.2—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy61.2—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy59.7—Unverified
4Gopher-280B (few-shot, k=5)Accuracy49.2—Unverified
5PaLM 540B (few-shot, k=3)Accuracy38—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy34.8—Unverified
8OPT 66B (few-shot, k=3)Accuracy31.2—Unverified
9GPT-NeoX (few-shot, k=3)Accuracy26—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy100—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy96.4—Unverified
3PaLM 540B (few-shot, k=3)Accuracy39.6—Unverified
4BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
5Chinchilla-70B (few-shot, k=5)Accuracy32—Unverified
6Bloomberg GPT (few-shot, k=3)Accuracy29.2—Unverified
7OPT 66B (few-shot, k=3)Accuracy23.6—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy21.2—Unverified
9Gopher-280B (few-shot, k=5)Accuracy19—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy44—Unverified
2PaLM-540B (few-shot, k=5)Accuracy42.4—Unverified
3PaLM-62B (few-shot, k=5)Accuracy36.5—Unverified
4Gopher-280B (few-shot, k=5)Accuracy35.1—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM-540B (few-shot, k=5)Accuracy73.9—Unverified
2Chinchilla-70B (few-shot, k=5)Accuracy68.3—Unverified
3PaLM-62B (few-shot, k=5)Accuracy65.4—Unverified
4Gopher-280B (few-shot, k=5)Accuracy61—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy 83.7—Unverified
2RuGPT-3 LargeAccuracy 40.7—Unverified
3RuGPT-3 MediumAccuracy 38—Unverified
4RuGPT-3 SmallAccuracy 34—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy87—Unverified
2RuGPT-3 SmallAccuracy57.9—Unverified
3RuGPT-3 MediumAccuracy57.2—Unverified
4RuGPT-3 LargeAccuracy55.5—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy72.1—Unverified
2Gopher-280B (few-shot, k=5)Accuracy58.9—Unverified