SOTAVerified

Logical Reasoning

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 151–200 of 747 papers

TitleStatusHype
AbductionRules: Training Transformers to Explain Unexpected InputsCode1
FaiRR: Faithful and Robust Deductive Reasoning over Natural LanguageCode1
AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading ComprehensionCode1
A Neuro-vector-symbolic Architecture for Solving Raven's Progressive MatricesCode1
MERIt: Meta-Path Guided Contrastive Learning for Logical ReasoningCode1
ExAIS: Executable AI SemanticsCode1
End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without OverthinkingCode1
VAEL: Bridging Variational Autoencoders and Probabilistic Logic ProgrammingCode1
Quantifying Adaptability in Pre-trained Language Models with 500 TasksCode1
Enhancing Multilingual Language Model with Massive Multilingual Knowledge TriplesCode1
Probabilistic Entity Representation Model for Reasoning over Knowledge GraphsCode1
ConditionalQA: A Complex Reading Comprehension Dataset with Conditional AnswersCode1
AI Descartes: Combining Data and Theory for Derivable Scientific DiscoveryCode1
From LSAT: The Progress and Challenges of Complex ReasoningCode1
Query Embedding on Hyper-relational Knowledge GraphsCode1
Discriminative Reasoning for Document-level Relation ExtractionCode1
SIRE: Separate Intra- and Inter-sentential Reasoning for Document-level Relation ExtractionCode1
NeuralLog: Natural Language Inference with Joint Neural and Logical ReasoningCode1
Fact-driven Logical Reasoning for Machine Reading ComprehensionCode1
Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of TextCode1
DAGN: Discourse-Aware Graph Network for Logical ReasoningCode1
Natural Language Inference in Context -- Investigating Contextual Reasoning over Long TextsCode1
Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge GraphsCode1
Measuring Systematic Generalization in Neural Proof Generation with TransformersCode1
Neural Logic ReasoningCode1
LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical ReasoningCode1
Logical Neural NetworksCode1
ReCO: A Large Scale Chinese Reading Comprehension Dataset on OpinionCode1
Neural Collaborative ReasoningCode1
Logic and the 2-Simplicial TransformerCode1
ReClor: A Reading Comprehension Dataset Requiring Logical ReasoningCode1
Quantum Embedding of Knowledge for ReasoningCode1
Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and KnowledgeCode1
FEVO: Financial Knowledge Expansion and Reasoning Evolution for Large Language Models—0
MiCo: Multi-image Contrast for Reinforcement Visual Reasoning—0
Discrete JEPA: Learning Discrete Token Representations without Reconstruction—0
CAPO: Reinforcing Consistent Reasoning in Medical Decision-Making—0
TeleMath: A Benchmark for Large Language Models in Telecom Mathematical Problem Solving—0
Motion-R1: Chain-of-Thought Reasoning and Reinforcement Learning for Human Motion Generation—0
TTT-Bench: A Benchmark for Evaluating Reasoning Ability with Simple and Novel Tic-Tac-Toe-style Games—0
EviNet: Evidential Reasoning Network for Resilient Graph Learning in the Open and Noisy EnvironmentsCode0
Are LLMs Reliable Translators of Logical Reasoning Across Lexically Diversified Contexts?Code0
Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision StudyCode0
Towards Geometry Problem Solving in the Large Model Era: A Survey—0
Infi-MMR: Curriculum-based Unlocking Multimodal Reasoning via Phased Reinforcement Learning in Multimodal Small Language Models—0
VisualSphinx: Large-Scale Synthetic Vision Logic Puzzles for RL—0
Continuous Chain of Thought Enables Parallel Exploration and Reasoning—0
Climate Finance BenchCode0
SV-TrustEval-C: Evaluating Structure and Semantic Reasoning in Large Language Models for Source Code Vulnerability AnalysisCode0
MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs—0
Show:102550
← PrevPage 4 of 15Next →
All datasetsLingOlyBIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTreeWinograd AutomaticBIG-bench (Logical Fallacy Detection)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Claude OpusDelta_NoContext28.8—Unverified
2GPT-4oDelta_NoContext25.1—Unverified
3Gemini 1.5 ProDelta_NoContext23.4—Unverified
4GPT-4Delta_NoContext21.5—Unverified
5Command R+Delta_NoContext11.6—Unverified
6GPT-3.5Delta_NoContext11.2—Unverified
7Mixtral 8x7BDelta_NoContext6.4—Unverified
8Llama 3 8BDelta_NoContext4.9—Unverified
9Llama 3 70BDelta_NoContext2.9—Unverified
10Gemma 7BDelta_NoContext2.2—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, Direct)Accuracy64.8—Unverified
2PaLM 2 (few-shot, k=3, CoT)Accuracy57.2—Unverified
3OPT 66B (few-shot, k=3)Accuracy54—Unverified
4PaLM 540B (few-shot, k=3)Accuracy53.6—Unverified
5GPT-NeoX 20B (few-shot, k=3)Accuracy52.8—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy52.8—Unverified
7Chinchilla-70B (few-shot, k=5)Accuracy52.1—Unverified
8Bloomberg GPT 50B (few-shot, k=3)Accuracy50.8—Unverified
9Gopher-280B (few-shot, k=5)Accuracy50.7—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy84.9—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy65.8—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy48.7—Unverified
4PaLM 540B (few-shot, k=3)Accuracy44.5—Unverified
5Gopher-280B (few-shot, k=5)Accuracy40.6—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy40.41—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy37.67—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy33.56—Unverified
9OPT 66B (few-shot, k=3)Accuracy28.08—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy91.2—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy61.2—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy59.7—Unverified
4Gopher-280B (few-shot, k=5)Accuracy49.2—Unverified
5PaLM 540B (few-shot, k=3)Accuracy38—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy34.8—Unverified
8OPT 66B (few-shot, k=3)Accuracy31.2—Unverified
9GPT-NeoX (few-shot, k=3)Accuracy26—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy100—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy96.4—Unverified
3PaLM 540B (few-shot, k=3)Accuracy39.6—Unverified
4BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
5Chinchilla-70B (few-shot, k=5)Accuracy32—Unverified
6Bloomberg GPT (few-shot, k=3)Accuracy29.2—Unverified
7OPT 66B (few-shot, k=3)Accuracy23.6—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy21.2—Unverified
9Gopher-280B (few-shot, k=5)Accuracy19—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy44—Unverified
2PaLM-540B (few-shot, k=5)Accuracy42.4—Unverified
3PaLM-62B (few-shot, k=5)Accuracy36.5—Unverified
4Gopher-280B (few-shot, k=5)Accuracy35.1—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM-540B (few-shot, k=5)Accuracy73.9—Unverified
2Chinchilla-70B (few-shot, k=5)Accuracy68.3—Unverified
3PaLM-62B (few-shot, k=5)Accuracy65.4—Unverified
4Gopher-280B (few-shot, k=5)Accuracy61—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy 83.7—Unverified
2RuGPT-3 LargeAccuracy 40.7—Unverified
3RuGPT-3 MediumAccuracy 38—Unverified
4RuGPT-3 SmallAccuracy 34—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy87—Unverified
2RuGPT-3 SmallAccuracy57.9—Unverified
3RuGPT-3 MediumAccuracy57.2—Unverified
4RuGPT-3 LargeAccuracy55.5—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy72.1—Unverified
2Gopher-280B (few-shot, k=5)Accuracy58.9—Unverified