SOTAVerified

Logical Reasoning

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 251–300 of 747 papers

TitleStatusHype
Towards Superior Quantization Accuracy: A Layer-sensitive Approach—0
SCoRE: Benchmarking Long-Chain Reasoning in Commonsense ScenariosCode0
The Society of HiveMind: Multi-Agent Optimization of Foundation Model Swarms to Unlock the Potential of Collective Intelligence—0
DB-Explore: Automated Database Exploration and Instruction Synthesis for Text-to-SQL—0
HelpSteer3: Human-Annotated Feedback and Edit Data to Empower Inference-Time Scaling in Open-Ended General-Domain Tasks—0
Psy-Insight: Explainable Multi-turn Bilingual Dataset for Mental Health Counseling—0
Three tiers of computation in transformers and in brain architecturesCode0
DeLTa: A Decoding Strategy based on Logit Trajectory Prediction Improves Factuality and Reasoning AbilityCode0
KGCompiler: Deep Learning Compilation Optimization for Knowledge Graph Complex Logical Query Answering—0
HoT: Highlighted Chain of Thought for Referencing Supporting Facts from Inputs—0
Order Doesn't Matter, But Reasoning Does: Training LLMs with Order-Centric Augmentation—0
Reversal Blessing: Thinking Backward May Outpace Thinking Forward in Multi-choice Questions—0
TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model ReasoningCode0
Logic Haystacks: Probing LLMs Long-Context Logical Reasoning (Without Easily Identifiable Unrelated Padding)—0
Intermediate Languages Matter: Formal Choice Drives Neurosymbolic LLM Reasoning—0
Autoregressive Image Generation Guided by Chains of Thought—0
Quantifying Logical Consistency in Transformers via Query-Key Alignment—0
Empowering LLMs with Logical Reasoning: A Comprehensive Survey—0
Identifying Features that Shape Perceived Consciousness in Large Language Model-based AI: A Quantitative Study of Human Responses—0
Triangulating LLM Progress through Benchmarks, Games, and Cognitive Tests—0
On the logical skills of large language models: evaluations using arbitrarily complex first-order logic problemsCode0
A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos—0
SPPD: Self-training with Process Preference Learning Using Dynamic Value Margin—0
Inference-Time Computations for LLM Reasoning and Planning: A Benchmark and Insights—0
HopRAG: Multi-Hop Reasoning for Logic-Aware Retrieval-Augmented Generation—0
Integrating Expert Knowledge into Logical Programs via LLMsCode0
Beyond Single-Task: Robust Multi-Task Length Generalization for LLMs—0
Dialogue-based Explanations for Logical Reasoning using Structured Argumentation—0
Quantifying the Capability Boundary of DeepSeek Models: An Application-Driven Performance Analysis—0
The Multilingual Mind : A Survey of Multilingual Reasoning in Language Models—0
Logical Reasoning in Large Language Models: A Survey—0
Logical Lease Litigation: Prolog and LLMs for Rental Law Compliance in New York—0
Logical forms complement probability in understanding language model (and human) performance—0
DMWM: Dual-Mind World Model with Long-Term Imagination—0
Structural Reformation of Large Language Model Neuron Encapsulation for Divergent Information Aggregation—0
S^2-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiency—0
SymAgent: A Neural-Symbolic Self-Learning Agent Framework for Complex Reasoning over Knowledge Graphs—0
Standard Neural Computation Alone Is Insufficient for Logical Intelligence—0
Automating Mathematical Proof Generation Using Large Language Model Agents and Knowledge Graphs—0
ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning—0
Enhancing Large Language Model Efficiencyvia Symbolic Compression: A Formal Approach Towards Interpretability—0
Instantiation-based Formalization of Logical Reasoning Tasks using Language Models and Logical Solvers—0
Town Hall Debate Prompting: Enhancing Logical Reasoning in LLMs through Multi-Persona Interaction—0
DBRouting: Routing End User Queries to Databases for Answerability—0
SedarEval: Automated Evaluation using Self-Adaptive RubricsCode0
A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models—0
JustLogic: A Comprehensive Benchmark for Evaluating Deductive Reasoning in Large Language ModelsCode0
VERUS-LM: a Versatile Framework for Combining LLMs with Symbolic Reasoning—0
Assessing the Alignment of FOL Closeness Metrics with Human JudgementCode0
Reasoning with Graphs: Structuring Implicit Knowledge to Enhance LLMs Reasoning—0
Show:102550
← PrevPage 6 of 15Next →
All datasetsLingOlyBIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTreeWinograd AutomaticBIG-bench (Logical Fallacy Detection)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Claude OpusDelta_NoContext28.8—Unverified
2GPT-4oDelta_NoContext25.1—Unverified
3Gemini 1.5 ProDelta_NoContext23.4—Unverified
4GPT-4Delta_NoContext21.5—Unverified
5Command R+Delta_NoContext11.6—Unverified
6GPT-3.5Delta_NoContext11.2—Unverified
7Mixtral 8x7BDelta_NoContext6.4—Unverified
8Llama 3 8BDelta_NoContext4.9—Unverified
9Llama 3 70BDelta_NoContext2.9—Unverified
10Gemma 7BDelta_NoContext2.2—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, Direct)Accuracy64.8—Unverified
2PaLM 2 (few-shot, k=3, CoT)Accuracy57.2—Unverified
3OPT 66B (few-shot, k=3)Accuracy54—Unverified
4PaLM 540B (few-shot, k=3)Accuracy53.6—Unverified
5GPT-NeoX 20B (few-shot, k=3)Accuracy52.8—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy52.8—Unverified
7Chinchilla-70B (few-shot, k=5)Accuracy52.1—Unverified
8Bloomberg GPT 50B (few-shot, k=3)Accuracy50.8—Unverified
9Gopher-280B (few-shot, k=5)Accuracy50.7—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy84.9—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy65.8—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy48.7—Unverified
4PaLM 540B (few-shot, k=3)Accuracy44.5—Unverified
5Gopher-280B (few-shot, k=5)Accuracy40.6—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy40.41—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy37.67—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy33.56—Unverified
9OPT 66B (few-shot, k=3)Accuracy28.08—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy91.2—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy61.2—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy59.7—Unverified
4Gopher-280B (few-shot, k=5)Accuracy49.2—Unverified
5PaLM 540B (few-shot, k=3)Accuracy38—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy34.8—Unverified
8OPT 66B (few-shot, k=3)Accuracy31.2—Unverified
9GPT-NeoX (few-shot, k=3)Accuracy26—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy100—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy96.4—Unverified
3PaLM 540B (few-shot, k=3)Accuracy39.6—Unverified
4BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
5Chinchilla-70B (few-shot, k=5)Accuracy32—Unverified
6Bloomberg GPT (few-shot, k=3)Accuracy29.2—Unverified
7OPT 66B (few-shot, k=3)Accuracy23.6—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy21.2—Unverified
9Gopher-280B (few-shot, k=5)Accuracy19—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy44—Unverified
2PaLM-540B (few-shot, k=5)Accuracy42.4—Unverified
3PaLM-62B (few-shot, k=5)Accuracy36.5—Unverified
4Gopher-280B (few-shot, k=5)Accuracy35.1—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM-540B (few-shot, k=5)Accuracy73.9—Unverified
2Chinchilla-70B (few-shot, k=5)Accuracy68.3—Unverified
3PaLM-62B (few-shot, k=5)Accuracy65.4—Unverified
4Gopher-280B (few-shot, k=5)Accuracy61—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy 83.7—Unverified
2RuGPT-3 LargeAccuracy 40.7—Unverified
3RuGPT-3 MediumAccuracy 38—Unverified
4RuGPT-3 SmallAccuracy 34—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy87—Unverified
2RuGPT-3 SmallAccuracy57.9—Unverified
3RuGPT-3 MediumAccuracy57.2—Unverified
4RuGPT-3 LargeAccuracy55.5—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy72.1—Unverified
2Gopher-280B (few-shot, k=5)Accuracy58.9—Unverified