SOTAVerified

Logical Reasoning

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 101–150 of 747 papers

TitleStatusHype
Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and KnowledgeCode1
LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual ContextsCode1
BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMsCode1
Large Language Models are Better Reasoners with Self-VerificationCode1
Improved Logical Reasoning of Language Models via Differentiable Symbolic ProgrammingCode1
AI Descartes: Combining Data and Theory for Derivable Scientific DiscoveryCode1
DAGN: Discourse-Aware Graph Network for Logical ReasoningCode1
Mind Reasoning Manners: Enhancing Type Perception for Generalized Zero-shot Logical Reasoning over TextCode1
Modeling Complex Mathematical Reasoning via Large Language Model based MathAgentCode1
Modeling Hierarchical Reasoning Chains by Linking Discourse Units and Key Phrases for Reading ComprehensionCode1
LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language ModelsCode1
Neural Graph Reasoning: Complex Logical Query Answering Meets Graph DatabasesCode1
Certified Deductive Reasoning with Language ModelsCode1
Chain of Images for Intuitively ReasoningCode1
Advancing Abductive Reasoning in Knowledge Graphs through Complex Logical Hypothesis GenerationCode1
IDOL: Indicator-oriented Logic Pre-training for Logical ReasoningCode1
Enhancing Multilingual Language Model with Massive Multilingual Knowledge TriplesCode1
Logical Message Passing Networks with One-hop Inference on Atomic FormulasCode1
AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading ComprehensionCode1
Automatic Curriculum Expert Iteration for Reliable LLM ReasoningCode1
OMGEval: An Open Multilingual Generative Evaluation Benchmark for Large Language ModelsCode1
ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language ModelsCode1
HAE-RAE Bench: Evaluation of Korean Knowledge in Language ModelsCode1
CHECKWHY: Causal Fact Verification via Argument StructureCode1
GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language ModelsCode1
A Neuro-vector-symbolic Architecture for Solving Raven's Progressive MatricesCode1
Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-AugmentationCode1
From LSAT: The Progress and Challenges of Complex ReasoningCode1
FOLIO: Natural Language Reasoning with First-Order LogicCode1
DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to DeterminacyCode1
AbductionRules: Training Transformers to Explain Unexpected InputsCode1
Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language NavigationCode1
Assessing and Enhancing the Robustness of Large Language Models with Task Structure Variations for Logical ReasoningCode1
GLoRE: Evaluating Logical Reasoning of Large Language ModelsCode1
Exposing Numeracy Gaps: A Benchmark to Evaluate Fundamental Numerical Abilities in Large Language ModelsCode1
ConditionalQA: A Complex Reading Comprehension Dataset with Conditional AnswersCode1
ExAIS: Executable AI SemanticsCode1
Fact-driven Logical Reasoning for Machine Reading ComprehensionCode1
Conditional and Modal Reasoning in Large Language ModelsCode1
Enhancing the Geometric Problem-Solving Ability of Multimodal LLMs via Symbolic-Neural IntegrationCode1
Explicit Planning Helps Language Models in Logical ReasoningCode1
Complex Logical Reasoning over Knowledge Graphs using Large Language ModelsCode1
Counterfactual reasoning: Do language models need world knowledge for causal understanding?Code1
Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4Code1
FaiRR: Faithful and Robust Deductive Reasoning over Natural LanguageCode1
Abstract Meaning Representation-Based Logic-Driven Data Augmentation for Logical ReasoningCode1
ElecBench: a Power Dispatch Evaluation Benchmark for Large Language ModelsCode1
End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without OverthinkingCode1
AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language ModelsCode1
COLLIE: Systematic Construction of Constrained Text Generation TasksCode1
Show:102550
← PrevPage 3 of 15Next →
All datasetsLingOlyBIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTreeWinograd AutomaticBIG-bench (Logical Fallacy Detection)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Claude OpusDelta_NoContext28.8—Unverified
2GPT-4oDelta_NoContext25.1—Unverified
3Gemini 1.5 ProDelta_NoContext23.4—Unverified
4GPT-4Delta_NoContext21.5—Unverified
5Command R+Delta_NoContext11.6—Unverified
6GPT-3.5Delta_NoContext11.2—Unverified
7Mixtral 8x7BDelta_NoContext6.4—Unverified
8Llama 3 8BDelta_NoContext4.9—Unverified
9Llama 3 70BDelta_NoContext2.9—Unverified
10Gemma 7BDelta_NoContext2.2—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, Direct)Accuracy64.8—Unverified
2PaLM 2 (few-shot, k=3, CoT)Accuracy57.2—Unverified
3OPT 66B (few-shot, k=3)Accuracy54—Unverified
4PaLM 540B (few-shot, k=3)Accuracy53.6—Unverified
5GPT-NeoX 20B (few-shot, k=3)Accuracy52.8—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy52.8—Unverified
7Chinchilla-70B (few-shot, k=5)Accuracy52.1—Unverified
8Bloomberg GPT 50B (few-shot, k=3)Accuracy50.8—Unverified
9Gopher-280B (few-shot, k=5)Accuracy50.7—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy84.9—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy65.8—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy48.7—Unverified
4PaLM 540B (few-shot, k=3)Accuracy44.5—Unverified
5Gopher-280B (few-shot, k=5)Accuracy40.6—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy40.41—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy37.67—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy33.56—Unverified
9OPT 66B (few-shot, k=3)Accuracy28.08—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy91.2—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy61.2—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy59.7—Unverified
4Gopher-280B (few-shot, k=5)Accuracy49.2—Unverified
5PaLM 540B (few-shot, k=3)Accuracy38—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy34.8—Unverified
8OPT 66B (few-shot, k=3)Accuracy31.2—Unverified
9GPT-NeoX (few-shot, k=3)Accuracy26—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy100—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy96.4—Unverified
3PaLM 540B (few-shot, k=3)Accuracy39.6—Unverified
4BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
5Chinchilla-70B (few-shot, k=5)Accuracy32—Unverified
6Bloomberg GPT (few-shot, k=3)Accuracy29.2—Unverified
7OPT 66B (few-shot, k=3)Accuracy23.6—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy21.2—Unverified
9Gopher-280B (few-shot, k=5)Accuracy19—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy44—Unverified
2PaLM-540B (few-shot, k=5)Accuracy42.4—Unverified
3PaLM-62B (few-shot, k=5)Accuracy36.5—Unverified
4Gopher-280B (few-shot, k=5)Accuracy35.1—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM-540B (few-shot, k=5)Accuracy73.9—Unverified
2Chinchilla-70B (few-shot, k=5)Accuracy68.3—Unverified
3PaLM-62B (few-shot, k=5)Accuracy65.4—Unverified
4Gopher-280B (few-shot, k=5)Accuracy61—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy 83.7—Unverified
2RuGPT-3 LargeAccuracy 40.7—Unverified
3RuGPT-3 MediumAccuracy 38—Unverified
4RuGPT-3 SmallAccuracy 34—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy87—Unverified
2RuGPT-3 SmallAccuracy57.9—Unverified
3RuGPT-3 MediumAccuracy57.2—Unverified
4RuGPT-3 LargeAccuracy55.5—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy72.1—Unverified
2Gopher-280B (few-shot, k=5)Accuracy58.9—Unverified